Quality evaluation method and device for in-situ observation data and electronic equipment

Abstract

The invention provides an in-situ observation data quality evaluation method and device and electronic equipment, and the method comprises the steps: carrying out the data compliance detection of a to-be-evaluated in-situ observation data set, and obtaining a data compliance detection score; performing data missing detection on the to-be-evaluated in-situ observation data set to obtain a data missing detection score; obtaining a numerical data set in the in-situ observation data set to be evaluated; performing data error detection on the numerical data set to obtain a data error detection score; performing data forgery detection on the numerical data set to obtain a data forgery detection score; and weighting the data compliance detection score, the data missing detection score, the data error detection score and the data forgery detection score to obtain a quality evaluation result of the to-be-evaluated in-situ observation data set. By means of the method, a set of indexes suitable for the current observation data set are screened out from the multiple indexes of the in-situ observation data, and it is ensured that the indexes can comprehensively cover all aspects of the data quality.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a quality evaluation method, device and electronic equipment for in-situ observation data. Background Art

[0002] In-situ observation data refers to direct measurements of natural phenomena or physical quantities at the data collection point, such as weather stations, ocean buoys, soil moisture sensors, etc., and is widely used in meteorology, environmental science, oceanography and other fields. With the development of the in-situ observation field, a large amount of in-situ observation data has been accumulated in related application fields, but the quality of these data has not been evaluated in multiple quality dimensions. Therefore, evaluating the quality of in-situ observation data not only plays a key role in improving data quality and value, but will also promote the scientific development of the entire observation field.

[0003] However, faced with massive and diverse in-situ observation data, selecting a set of indicators that can fully reflect data quality and adapt to specific observation needs is a complex task. Therefore, how to select a set of indicators that best suits the current observation data set from a large number of indicators while ensuring that these indicators can comprehensively cover all aspects of data quality is an urgent problem to be solved. Summary of the invention

[0004] The present invention provides a quality evaluation method, device and electronic equipment for in-situ observation data, which are used to select a set of indicators that are most suitable for the current observation data set from numerous indicators of in-situ observation data, while ensuring that these indicators can comprehensively cover all aspects of data quality.

[0005] The present invention provides a method for evaluating the quality of in-situ observation data, comprising the following steps: Obtain the in-situ observation data set to be evaluated; Performing a data compliance test on the in-situ observation data set to be evaluated to obtain a data compliance test score; and performing a data missing test on the in-situ observation data set to be evaluated to obtain a data missing test score; Acquire a numerical data set in the in-situ observation data set to be evaluated; perform data misdetection detection on the numerical data set to obtain a data misdetection detection score; perform data forgery detection on the numerical data set to obtain a data forgery detection score; According to the weight of each first-level scoring indicator in the in-situ observation dataset to be evaluated, the data compliance detection score, the data missing detection score, the data misdetection score and the data falsification detection score are weighted to obtain the quality evaluation result of the in-situ observation dataset to be evaluated.

[0006] According to a quality evaluation method for in-situ observation data provided by the present invention, data compliance detection is performed on the in-situ observation data set to be evaluated to obtain a data compliance detection score, specifically comprising: Testing the data uniqueness and data standardization of the in-situ observation data set to be evaluated to obtain a data uniqueness score and a data standardization score; According to the weights of the secondary scoring indicators in the in-situ observation data set to be evaluated, the data uniqueness score and the data standardization score are weighted to obtain the data compliance detection score of the in-situ observation data set to be evaluated.

[0007] According to a quality evaluation method for in-situ observation data provided by the present invention, data missing detection is performed on the in-situ observation data set to be evaluated to obtain a data missing detection score, specifically comprising: Detecting the attribute integrity of the data in the in-situ observation data set to be evaluated to obtain an attribute integrity score; Testing the record integrity of the data in the in-situ observation data set to be evaluated to obtain a record integrity score; Testing the numerical integrity of the data in the in-situ observation data set to be evaluated to obtain a numerical integrity score; According to the weights of the secondary scoring indicators in the in-situ observation data set to be evaluated, the attribute integrity score, the record integrity score and the numerical integrity score are weighted to obtain the data missing detection score of the in-situ observation data set to be evaluated.

[0008] According to a quality evaluation method for in-situ observation data provided by the present invention, data error detection is performed on the numerical data set to obtain a data error detection score, specifically comprising: Testing the data accuracy of the numerical data set to obtain a data accuracy score; Testing the data consistency of the numerical data set to obtain a data consistency score; According to the weights of the secondary scoring indicators in the in-situ observation data set to be evaluated, the data accuracy score and the data consistency score are weighted to obtain the data mismeasurement detection score.

[0009] According to a quality evaluation method for in-situ observation data provided by the present invention, data forgery detection is performed on the numerical data set to obtain a data forgery detection score, specifically comprising: A similarity judgment is performed on the numerical data set and the historical data set to obtain a data similarity score; and based on the data similarity score, the data forgery detection score is obtained.

[0010] According to a quality evaluation method for in-situ observation data provided by the present invention, the data accuracy includes one or more of range accuracy and numerical accuracy, and the data accuracy of the numerical data set is detected to obtain a data accuracy score, specifically including: Testing the range accuracy of the numerical data set to obtain a range accuracy score; Testing the numerical accuracy of the numerical data set to obtain a numerical accuracy score; The data accuracy score is obtained according to the range accuracy score and the numerical accuracy score.

[0011] The present invention also provides a quality evaluation device for in-situ observation data, comprising: A data acquisition module, used to obtain the in-situ observation data set to be evaluated; A first scoring module is used to perform data compliance detection on the in-situ observation data set to be evaluated to obtain a data compliance detection score; and perform data missing detection on the in-situ observation data set to be evaluated to obtain a data missing detection score; The second scoring module is used to obtain a numerical data set in the in-situ observation data set to be evaluated; perform data error detection on the numerical data set to obtain a data error detection score; perform data forgery detection on the numerical data set to obtain a data forgery detection score; The scoring weighting module is used to weight the data compliance detection score, the data missing detection score, the data error detection score and the data falsification detection score according to the weight of each first-level scoring indicator in the in-situ observation data set to be evaluated, so as to obtain the quality evaluation result of the in-situ observation data set to be evaluated.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a quality evaluation method for in-situ observation data as described in any one of the above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the quality assessment method for in-situ observation data as described in any one of the above.

[0014] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the quality assessment method for in-situ observation data as described in any one of the above is implemented.

[0015] The quality evaluation method, device and electronic equipment of in-situ observation data provided by the present invention are achieved through. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flow chart of the quality evaluation method of in-situ observation data provided by the present invention.

[0018] Figure 2 It is a structural schematic diagram of the quality evaluation device for in-situ observation data provided by the present invention.

[0019] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchanged where appropriate, so that the embodiments can be implemented in a sequence other than that illustrated or described in the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of the steps that appear in the present application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of modules in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. In addition, the modules or submodules described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed in multiple circuit units, and some or all of the units may be selected according to actual needs to achieve the purpose of the present application.

[0022] In the embodiment of the present invention, the quality evaluation of the in-situ observation data is aimed at the monitoring mechanism of the in-situ observation data and reflecting the quality status of the data. Among them, the in-situ observation data mainly has three characteristics: 1. The in-situ observation data is real-time, can provide real-time observation results, and reflect the instantaneous state of the physical quantity under specific conditions. 2. The in-situ observation data is dynamic, can observe and record the changes of the physical quantity in the time evolution process, and there is a correlation between the physical quantities in the in-situ observation data. By analyzing whether there is an abnormal continuous state of each physical quantity as time evolves, the quality status of the in-situ observation data is reflected in a deeper level. 3. There are time-type data, text-type data and numerical data in the in-situ observation data. The time-type data has a specific storage form, the text-type data usually records the parameters involved in observing the in-situ observation data, and the numerical data has specific accuracy requirements. Since data quality itself is a subjective issue, by quantifying the data, the data quality can really play a role, so that the data quality has multiple indicators and determining factors, and these elements are integrated into a complete comprehensive evaluation method.

[0023] Combine the following Figure 1-Figure 3 The specific contents of the present invention are described.

[0024] Figure 1 It is a flow chart of a quality evaluation method for in-situ observation data provided by the present invention, including steps S101-S104.

[0025] Step S101, obtaining an in-situ observation data set to be evaluated.

[0026] For example, a week of observation data from a weather station is selected as the in-situ observation data set to be evaluated. The data set includes meteorological elements such as temperature, humidity, wind speed, wind direction, air pressure, and precipitation, which are recorded regularly every day, and each element has an observation value every hour. The data set is stored in CSV file format, which contains the timestamp and the observation value of each meteorological element.

[0027] Step S102, performing data compliance detection on the in-situ observation data set to be evaluated to obtain a data compliance detection score; and performing data missing detection on the in-situ observation data set to be evaluated to obtain a data missing detection score.

[0028] Specifically, standardization refers to the degree to which the data content of the in-situ observation data set to be evaluated complies with relevant specifications and standards. Generally speaking, high-quality in-situ observation data should be non-redundant and the data type should comply with standardization requirements. The degree of data compliance can be measured by checking the degree of non-redundancy and standardization of the data.

[0029] At the same time, since in-situ observation data with high missing rates will have a significant impact on future data analysis, modeling, and research results, when the data missing rate is too high, some variables or observations may be completely excluded from the analysis, reducing the value of the data set. In particular, when the missing rate of key variables is high, the analytical value of the data is significantly reduced. Excessively high missing rates will limit the reuse and sharing of in-situ observation data. Therefore, checking the integrity of in-situ observation data can be used as a part of the missing rate inspection of in-situ observation data and as a factor in evaluating the quality of in-situ observation data.

[0030] In an embodiment of the present invention, data missing detection is performed on the in-situ observation dataset to be evaluated. The in-situ observation dataset to be evaluated can be evaluated from three aspects: attribute integrity, record integrity and value integrity. The specific implementation process can refer to the content of subsequent embodiments.

[0031] In a possible implementation, in step S102, data compliance detection is performed on the in-situ observation data set to be evaluated to obtain a data compliance detection score, which specifically includes: Step S1021 , testing the data uniqueness and data standardization of the in-situ observation data set to be evaluated, and obtaining a data uniqueness score and a data standardization score.

[0032] In an embodiment of the present invention, the in-situ observation data set is composed of multiple attribute data, and the data uniqueness score is obtained by querying the proportion of attribute data with duplicate data to all attribute data. The greater the proportion of attribute data with duplicate data to all attribute data, the lower the data uniqueness score.

[0033] In an embodiment of the present invention, each attribute data includes data standard formats such as time format, data type or data precision. The data standardization score is obtained by querying the proportion of attribute data that meets all data standard formats to all attribute data. The greater the proportion of attribute data that meets all data standard formats to all attribute data, the higher the data standardization score.

[0034] Step S1022, according to the weights of each secondary scoring indicator in the in-situ observation data set to be evaluated, the data uniqueness score and the data standardization score are weighted to obtain the data compliance detection score of the in-situ observation data set to be evaluated.

[0035] In an embodiment of the present invention, the secondary scoring indicators include a data uniqueness score and a data standardization score. For different types of in-situ observation data sets, weights are pre-configured for the data uniqueness score and the data standardization score through expert scoring and the data quality requirements of the in-situ observation data sets. The higher the weight, the more important the scoring indicator is for the data set. The data uniqueness score and the data standardization score obtained above are weighted to obtain the data compliance detection score of the in-situ observation data set to be evaluated.

[0036] In a possible implementation, in step S102, data missing detection is performed on the in-situ observation data set to be evaluated to obtain a data missing detection score, which specifically includes: Step S1023, detecting the attribute integrity of the data in the in-situ observation data set to be evaluated, and obtaining an attribute integrity score.

[0037] In an embodiment of the present invention, the attribute integrity reflects the degree of missing of each attribute data in the in-situ observation data set to be evaluated. If there are factors such as observation equipment failure, the collected attribute data (observed physical quantity) may be completely lost, which means that the target data collected by the in-situ observation data set is incomplete. If key attribute data is missing, the representativeness of the data set will be reduced. Therefore, the attribute integrity score is obtained by querying the proportion of attribute data with non-null values ​​in all attribute data. Among them, the greater the proportion of attribute data with non-null values ​​in all attribute data, the higher the attribute integrity score.

[0038] Step S1024, detecting the record integrity of the data in the in-situ observation data set to be evaluated, and obtaining a record integrity score.

[0039] In an embodiment of the present invention, record integrity reflects the degree of missing observation records in the in-situ observation data set to be evaluated. If the collection of in-situ observation data is specified to have a start time, an end time, and a sampling frequency, if there is a data transmission error, the entire record may be missing. It should be noted that a record is considered a complete record only when all non-missing attribute values ​​are present. In an embodiment of the present invention, the record integrity of the in-situ observation data set to be evaluated is either complete or incomplete, and the record integrity score is also full marks or zero accordingly.

[0040] Step S1025, detecting the numerical integrity of the data in the in-situ observation data set to be evaluated, and obtaining a numerical integrity score.

[0041] In an embodiment of the present invention, numerical integrity reflects the degree of missing elements in a data set. Factors such as data omission may lead to the loss of elements in a data set. It should be noted that the numerical integrity here refers to the missing of some elements in all non-missing attributes and non-missing records. That is, for the attribute data without missing attributes and missing records mentioned above, it is queried whether each element in these attribute data is missing. Therefore, by querying the number of missing elements in all elements of the attribute data that meet both attribute integrity and record integrity, the percentage of all elements of the attribute data that meet both attribute integrity and record integrity can be used to obtain a numerical integrity score.

[0042] For example, for an in-situ observation data set to be evaluated with 100 attribute data, there are 98 attribute data that meet attribute integrity and record integrity. Among the above 98 attribute data, the number of all elements is 98*10, that is, each attribute data contains 10 elements. Assuming that there are 8 missing elements in the above 98 attribute data, the numerical integrity score can be expressed as (98*10-8) / 980.

[0043] Step S1026, according to the weights of each secondary scoring indicator in the in-situ observation data set to be evaluated, the attribute integrity score, the record integrity score and the value integrity score are weighted to obtain the data missing detection score of the in-situ observation data set to be evaluated.

[0044] In an embodiment of the present invention, the secondary scoring index also includes an attribute integrity score, a record integrity score, and a numerical integrity score. For different types of in-situ observation data sets, the indicator weights of the attribute integrity score, the record integrity score, and the numerical integrity score are obtained through expert scoring and the data quality requirements of the in-situ observation data sets. And weighted processing is performed to obtain the data missing detection score of the in-situ observation data set to be evaluated.

[0045] Step S103, obtaining a numerical data set in the in-situ observation data set to be evaluated; performing data error detection on the numerical data set to obtain a data error detection score; performing data forgery detection on the numerical data set to obtain a data forgery detection score.

[0046] In one possible implementation, the detection of various quality evaluation indicators of the in-situ observation data must be prioritized. Since the in-situ observation data set contains both time-type data, text-type data and numerical-type data, before dividing the data set, the indicator dimensions shared by the time-type data, text-type data and numerical-type data are detected. After dividing the data set, the indicator dimensions required only for numerical data are detected.

[0047] Specifically, the mismeasurement check of in-situ observation data is one of the important links of data quality control of the present invention. In an embodiment of the present invention, data error observation refers to data measurement errors caused by factors such as instrument failure, human operation error or environmental interference. Checking and processing these erroneous data is crucial to ensure the accuracy and reliability of the analysis results. Since numerical data usually have a specific range of values, there is a specific correlation between the attributes (observed physical quantities) in the in-situ observation data set to be evaluated, so the mismeasurement check process of the in-situ observation data can be further divided into two sub-dimensions, including: detecting the data accuracy of the numerical data set, and detecting the data consistency of the numerical data set.

[0048] In addition, in order to ensure the credibility of data quality, the primary purpose of performing data forgery detection on numerical data sets is to ensure the authenticity of the data and provide a reliable basis for subsequent data analysis, research and decision-making. In an embodiment of the present invention, data forgery detection is performed on numerical data sets to obtain a data forgery detection score, as shown in the specific embodiment of step S1034.

[0049] In a possible implementation, in step S103, data error detection is performed on the numerical data set to obtain a data error detection score, which specifically includes: Step S1031, testing the data accuracy of the numerical data set to obtain a data accuracy score.

[0050] In the embodiments of the present invention, data accuracy is used to characterize the degree to which the data in a numerical data set conforms to the actual conditions or standards of the real world. Specifically, data accuracy measures whether the data correctly reflects the object, event or phenomenon it represents.

[0051] In a possible implementation, data accuracy includes one or more of range accuracy and numerical accuracy. The data accuracy of a numerical data set is tested to obtain a data accuracy score, specifically including: The range accuracy of the numerical data set is tested to obtain a range accuracy score; the numerical accuracy of the numerical data set is tested to obtain a numerical accuracy score; and the data accuracy score is obtained based on the range accuracy score and the numerical accuracy score.

[0052] Specifically, data accuracy can be divided into range accuracy and numerical accuracy according to the nature of numerical data sets and the needs of actual applications.

[0053] In an embodiment of the present invention, range accuracy refers to whether the data in a numerical data set falls within a reasonable and predefined range, which is used for preliminary screening and verification of data to ensure that the data does not contain obviously unreasonable values. For each data in a numerical data set, there is a corresponding data range. The range accuracy score is obtained by querying the data in the numerical data set that meets the data range and the proportion of the data in the numerical data set to all the data.

[0054] In an embodiment of the present invention, numerical accuracy refers to the degree of precision between the data in a numerical data set and the actual value, that is, the authenticity of the data, ensuring that the measured or recorded data is highly close to the actual situation. In a numerical data set, the proportion of the range that falls within a preset data true value interval is determined to obtain a data accuracy score.

[0055] The present invention uses a combination of range accuracy and numerical accuracy to comprehensively evaluate the quality of data, ensuring that the data does not contain obvious errors and meets actual needs in terms of accuracy.

[0056] Step S1032: Check the data consistency of the numerical data set to obtain a data consistency score.

[0057] In an embodiment of the present invention, there is a logical relationship between the physical quantities of the numerical data set; there may be element correlations in different physical quantities of the in-situ observation data, indicating that there is a logical relationship between different physical quantities at the same time point. By pre-defining different logical rules, the degree to which the in-situ observation data conforms to the consistency principle is judged. Exemplarily, for some data in the numerical data set, if there are pre-defined logical rules, it is necessary to judge whether the parts of the data conform to the pre-defined logical rules. If there are N data groups in the numerical data set that define the association relationship of the logical rules. Then obtain the proportion of data groups that meet the pre-defined logical rules in the N data groups to obtain the data consistency score.

[0058] Step S1033, according to the weights of each secondary scoring indicator in the in-situ observation data set to be evaluated, the data accuracy score and the data consistency score are weighted to obtain the data mismeasurement detection score.

[0059] In the embodiment of the present invention, the secondary scoring index also includes a data accuracy score and a data consistency score. The weighted processing of the score can refer to the content of the above embodiment, and the present invention will not be described in detail here.

[0060] In a possible implementation, in step S103, data forgery detection is performed on the numerical data set to obtain a data forgery detection score, specifically including: Step S1034, performing similarity judgment on the numerical data set and the historical data set to obtain a data similarity score; and obtaining a data forgery detection score based on the data similarity score.

[0061] In an embodiment of the present invention, similarity judgment is performed on numerical data sets and historical data sets. The observation record at a certain moment is judged similarly with the observation record at a certain moment in the past, and a similarity score is obtained between them. If the score exceeds a certain threshold and the observation time interval between the two records is greater than the minimum value of the interval between the two records, it is considered that a low-probability event has occurred, and the in-situ observation data set to be evaluated is suspected of forged records. Exemplarily, the data similarity score can be determined by the similarity score between them obtained by similarity judgment.

[0062] Step S104, according to the weight of each first-level scoring indicator in the in-situ observation data set to be evaluated, weighted processing is performed on the data compliance detection score, the data missing detection score, the data error detection score and the data falsification detection score to obtain the quality evaluation result of the in-situ observation data set to be evaluated.

[0063] In an embodiment of the present invention, the first-level scoring indicators include data compliance detection score, data missing detection score, data error detection score and data forgery detection score. The weights of the first-level scoring indicators are obtained by pre-configuration.

[0064] In a possible implementation, the quality evaluation method for in-situ observation data further includes: obtaining weights of each secondary scoring indicator.

[0065] In an embodiment of the present invention, the secondary scoring indicators include, but are not limited to, data uniqueness scoring, data standardization scoring, attribute integrity scoring, record integrity scoring, numerical integrity scoring, data accuracy scoring, data consistency scoring, etc.

[0066] First, by constructing a judgment matrix, the influence of each secondary scoring indicator on the same level on the primary scoring indicator of the previous level is compared, and each secondary scoring indicator is compared pairwise.

[0067] Then, the eigenvector and the maximum eigenvalue of the judgment matrix are calculated.

[0068] Finally, solve the eigenvector corresponding to the maximum eigenvalue of the judgment matrix A w The largest characteristic root The eigenvector of w Satisfy the equation , the eigenvector Represents the relative weight of each indicator. Normalized feature vector: Normalize the feature vector so that the weight vector w The sum of the elements of is 1. The weight of the i-th secondary scoring indicator is determined according to the following formula: In a possible implementation manner, after obtaining the weights of each secondary scoring indicator, the method further includes: calculating a consistency ratio (CR) to determine whether the judgment matrix has consistency.

[0069] Specifically, if the consistency ratio is greater than or equal to 1, the elements of the judgment matrix are adjusted and recalculated until the consistency test passes.

[0070] In the embodiment of the present invention, by weighted summation layer by layer: starting from the secondary scoring index, the weight of each secondary scoring index is multiplied by the weight of its corresponding primary evaluation index, and the weighted summation is performed layer by layer to obtain the comprehensive weight of each secondary index on the overall goal, and finally form a complete weight system. The quality evaluation result of the in-situ observation data set to be evaluated can be obtained through the comprehensive weight of each secondary index on the overall goal and the score value of each secondary index.

[0071] By adopting the above method, the present invention can achieve the following beneficial effects: by obtaining data compliance detection scores, data missing detection scores, data error detection scores and data falsification detection scores and performing weighted processing, the quality evaluation results of the in-situ observation data set to be evaluated are obtained. A set of indicators that are most suitable for the current observation data set is selected from the numerous indicators of the in-situ observation data, while ensuring that these indicators can comprehensively cover all aspects of data quality. By combining a variety of quantitative and qualitative analysis methods, a comprehensive and systematic evaluation of the in-situ observation data is carried out to reduce the influence of human subjective factors, and better cope with the complex relationship between indicators of different dimensions.

[0072] The quality evaluation device for in-situ observation data provided by the present invention is described below. The quality evaluation device for in-situ observation data described below and the quality evaluation method for in-situ observation data described above can be referred to each other.

[0073] Figure 2 A schematic diagram of the structure of a quality evaluation device for in-situ observation data provided by the present invention includes the following parts: The data acquisition module 210 is used to acquire the in-situ observation data set to be evaluated.

[0074] The first scoring module 220 is used to perform data compliance detection on the in-situ observation dataset to be evaluated to obtain a data compliance detection score; and perform data missing detection on the in-situ observation dataset to be evaluated to obtain a data missing detection score.

[0075] The second scoring module 230 is used to obtain a numerical data set in the in-situ observation data set to be evaluated; perform data error detection on the numerical data set to obtain a data error detection score; perform data forgery detection on the numerical data set to obtain a data forgery detection score.

[0076] The scoring weighting module 240 is used to perform weighted processing on the data compliance detection score, data missing detection score, data error detection score and data falsification detection score according to the weight of each first-level scoring indicator in the in-situ observation data set to be evaluated, so as to obtain the quality evaluation result of the in-situ observation data set to be evaluated.

[0077] In a possible implementation, the first scoring module 220 is specifically configured to: The data uniqueness and data standardization of the in-situ observation data set to be evaluated are tested to obtain data uniqueness scores and data standardization scores; According to the weights of each secondary scoring indicator in the in-situ observation dataset to be evaluated, the data uniqueness score and the data standardization score are weighted to obtain the data compliance detection score of the in-situ observation dataset to be evaluated.

[0078] In a possible implementation, the first scoring module 220 is specifically configured to: Detect the attribute integrity of the data in the in-situ observation dataset to be evaluated and obtain the attribute integrity score; Test the record integrity of the data in the in-situ observation dataset to be evaluated and obtain a record integrity score; Test the numerical integrity of the data in the in-situ observation dataset to be evaluated and obtain a numerical integrity score; According to the weights of each secondary scoring indicator in the in-situ observation dataset to be evaluated, the attribute integrity score, record integrity score and numerical integrity score are weighted to obtain the data missing detection score of the in-situ observation dataset to be evaluated.

[0079] In a possible implementation, the second scoring module 230 is specifically configured to: Test the data accuracy of numerical data sets and obtain data accuracy scores; Test the data consistency of numerical data sets and obtain data consistency scores; According to the weights of each secondary scoring indicator in the in-situ observation data set to be evaluated, the data accuracy score and the data consistency score are weighted to obtain the data mismeasurement detection score.

[0080] In a possible implementation, the second scoring module 230 is specifically configured to: The similarity of the numerical data set and the historical data set is judged to obtain a data similarity score; and based on the data similarity score, a data forgery detection score is obtained.

[0081] In a possible implementation, data accuracy includes one or more of range accuracy and value accuracy. The second scoring module 230 is specifically configured to: Test the range accuracy of the numerical data set and obtain the range accuracy score; Test the numerical accuracy of numerical data sets and obtain numerical accuracy scores; Based on the range accuracy score and the numerical accuracy score, the data accuracy score is obtained.

[0082] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communications interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a quality evaluation method for in-situ observation data, the method comprising: Obtain the in-situ observation data set to be evaluated; Perform data compliance detection on the in-situ observation dataset to be evaluated to obtain a data compliance detection score; and perform data missing detection on the in-situ observation dataset to be evaluated to obtain a data missing detection score; Obtain a numerical data set from the in-situ observation data set to be evaluated; perform data mismeasurement detection on the numerical data set to obtain a data mismeasurement detection score; perform data forgery detection on the numerical data set to obtain a data forgery detection score; According to the weight of each first-level scoring indicator in the in-situ observation dataset to be evaluated, the data compliance detection score, data missing detection score, data error detection score and data falsification detection score are weighted to obtain the quality evaluation result of the in-situ observation dataset to be evaluated.

[0083] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0084] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute a quality assessment method for in-situ observation data provided by the above methods, the method comprising: Obtain the in-situ observation data set to be evaluated; Perform data compliance detection on the in-situ observation dataset to be evaluated to obtain a data compliance detection score; and perform data missing detection on the in-situ observation dataset to be evaluated to obtain a data missing detection score; Obtain a numerical data set from the in-situ observation data set to be evaluated; perform data mismeasurement detection on the numerical data set to obtain a data mismeasurement detection score; perform data forgery detection on the numerical data set to obtain a data forgery detection score; According to the weight of each first-level scoring indicator in the in-situ observation dataset to be evaluated, the data compliance detection score, data missing detection score, data error detection score and data falsification detection score are weighted to obtain the quality evaluation result of the in-situ observation dataset to be evaluated.

[0085] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform a quality assessment method for in-situ observation data provided by the above methods, the method comprising: Obtain the in-situ observation data set to be evaluated; Perform data compliance detection on the in-situ observation dataset to be evaluated to obtain a data compliance detection score; and perform data missing detection on the in-situ observation dataset to be evaluated to obtain a data missing detection score; Obtain a numerical data set from the in-situ observation data set to be evaluated; perform data mismeasurement detection on the numerical data set to obtain a data mismeasurement detection score; perform data forgery detection on the numerical data set to obtain a data forgery detection score; According to the weight of each first-level scoring indicator in the in-situ observation dataset to be evaluated, the data compliance detection score, data missing detection score, data error detection score and data falsification detection score are weighted to obtain the quality evaluation result of the in-situ observation dataset to be evaluated.

[0086] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0087] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A quality evaluation method for in-situ observation data, characterized in that: include: Obtain the in-situ observation data set to be evaluated; Performing data compliance detection on the in-situ observation data set to be evaluated to obtain a data compliance detection score; and performing data missing detection on the in-situ observation data set to be evaluated to obtain a data missing detection score; Acquire a numerical data set from the in-situ observation data set to be evaluated; Performing data error detection on the numerical data set to obtain a data error detection score; performing data forgery detection on the numerical data set to obtain a data forgery detection score; According to the weight of each first-level scoring indicator in the in-situ observation dataset to be evaluated, the data compliance detection score, the data missing detection score, the data misdetection score and the data falsification detection score are weighted to obtain the quality evaluation result of the in-situ observation dataset to be evaluated.

2. The quality evaluation method of in-situ observation data according to claim 1, characterized in that: Perform data compliance detection on the in-situ observation data set to be evaluated to obtain a data compliance detection score, which specifically includes: Testing the data uniqueness and data standardization of the in-situ observation data set to be evaluated to obtain a data uniqueness score and a data standardization score; According to the weights of the secondary scoring indicators in the in-situ observation data set to be evaluated, the data uniqueness score and the data standardization score are weighted to obtain the data compliance detection score of the in-situ observation data set to be evaluated.

3. The quality evaluation method of in-situ observation data according to claim 1, characterized in that: Performing data missing detection on the in-situ observation data set to be evaluated to obtain a data missing detection score, specifically including: Detecting the attribute integrity of the data in the in-situ observation data set to be evaluated to obtain an attribute integrity score; Testing the record integrity of the data in the in-situ observation data set to be evaluated to obtain a record integrity score; Testing the numerical integrity of the data in the in-situ observation data set to be evaluated to obtain a numerical integrity score; According to the weights of the secondary scoring indicators in the in-situ observation data set to be evaluated, the attribute integrity score, the record integrity score and the numerical integrity score are weighted to obtain the data missing detection score of the in-situ observation data set to be evaluated.

4. The quality evaluation method of in-situ observation data according to claim 1, characterized in that: Performing data error detection on the numerical data set to obtain a data error detection score specifically includes: Testing the data accuracy of the numerical data set to obtain a data accuracy score; Testing the data consistency of the numerical data set to obtain a data consistency score; According to the weights of the secondary scoring indicators in the in-situ observation data set to be evaluated, the data accuracy score and the data consistency score are weighted to obtain the data mismeasurement detection score.

5. The quality evaluation method of in-situ observation data according to claim 1, characterized in that: Performing data forgery detection on the numerical data set to obtain a data forgery detection score specifically includes: A similarity judgment is performed on the numerical data set and the historical data set to obtain a data similarity score; and based on the data similarity score, the data forgery detection score is obtained.

6. The quality evaluation method of in-situ observation data according to claim 4, characterized in that: The data accuracy includes one or more of range accuracy and numerical accuracy. The data accuracy of the numerical data set is detected to obtain a data accuracy score, which specifically includes: Testing the range accuracy of the numerical data set to obtain a range accuracy score; Testing the numerical accuracy of the numerical data set to obtain a numerical accuracy score; The data accuracy score is obtained according to the range accuracy score and the numerical accuracy score.

7. A quality evaluation device for in-situ observation data, characterized in that: include: A data acquisition module, used to obtain the in-situ observation data set to be evaluated; A first scoring module is used to perform data compliance detection on the in-situ observation data set to be evaluated to obtain a data compliance detection score; and perform data missing detection on the in-situ observation data set to be evaluated to obtain a data missing detection score; A second scoring module is used to obtain a numerical data set from the in-situ observation data set to be evaluated; Performing data error detection on the numerical data set to obtain a data error detection score; performing data forgery detection on the numerical data set to obtain a data forgery detection score; The scoring weighting module is used to weight the data compliance detection score, the data missing detection score, the data error detection score and the data falsification detection score according to the weight of each first-level scoring indicator in the in-situ observation data set to be evaluated, so as to obtain the quality evaluation result of the in-situ observation data set to be evaluated.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the quality assessment method for in-situ observation data according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the quality assessment method for in-situ observation data according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the quality assessment method for in-situ observation data according to any one of claims 1 to 6 is implemented.

Images