A method of monitoring the quality of test data

Abstract

The application discloses a test test data quality monitoring method. The method comprises the following steps: determining quality state data of test data according to analysis characteristic results and business logic which are set by a user in advance, wherein the quality state data comprises low-quality data and valid data; determining an optimization method according to the type of the low-quality data, optimizing the low-quality data by the optimization method, and determining valid test data, wherein the valid test data comprises the valid data and the optimized low-quality data; and transmitting the valid test data and the test data to a terminal display device, and the terminal display device is used for displaying evaluation analysis graphs of the valid test data and the test data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and more particularly, to a test data quality monitoring method. BACKGROUND

[0002] At present, with the gradual increase of test equipment, the test data of test system is also increasing. The test results can be obtained by analyzing and calculating the test data. However, there are low-quality data in the test data, which affects the test results. Moreover, there are various types of tests, and the experimental data is various, and the types of low-quality data are also increasing. At present, it is difficult to solve the problem of processing low-quality data of various test data. SUMMARY

[0003] In view of the deficiencies of the prior art, the present application provides a test data quality monitoring method.

[0004] According to one aspect of the present application, a test data quality monitoring method is provided, comprising: determining the quality state data of the test data according to the analysis characteristic results and the business logic set by the user in advance, wherein the quality state data comprises low-quality data and valid data; determining the optimization method according to the type of the low-quality data, and optimizing the low-quality data by the optimization method to determine the valid test data, wherein the valid test data comprises the valid data and the optimized low-quality data; and transmitting the valid test data and the test data to a terminal display device, and the terminal display device is used to display the evaluation analysis diagram of the valid test data and the test data.

[0005] Optionally, the low-quality data comprises missing data, abnormal data, mutation data and non-standard data, and the method further comprises: scoring the different types of quality state data to determine the scores of the different types of quality state data, wherein the scores are used to indicate the grades of the quality state data, and the operation of scoring the different types of quality state data to determine the scores of the different types of quality state data comprises: determining a first score of the valid data according to the proportion of the valid data in the test data; determining a second score of the missing data according to the proportion of the missing data in the test data; determining a third score of the abnormal data according to the proportion of the abnormal data in the test data; determining a fourth score of the mutation data according to the proportion of the mutation data in the test data; and determining a fifth score of the non-standard data according to the proportion of the non-standard data in the test data.

[0006] Optionally, the method further comprises: detecting the test data by a pre-set threshold test method / feature comparison algorithm to determine the data amount of the abnormal data.

[0007] Optionally, the method further comprises: detecting the test data by a preset static mutation detection method / dynamic mutation detection method, and determining the data amount of the mutation data.

[0008] Optionally, the method further comprises: according to a preset determination logic, alarming the low-quality data, and determining alarm prompt information, wherein the alarm prompt information comprises a pre-warning value and an alarm value, and the determination logic comprises greater than a pre-warning upper limit, greater than a pre-warning lower limit, greater than an alarm upper limit, greater than an alarm lower limit, greater than the pre-warning upper limit and less than the alarm upper limit, greater than the pre-warning lower limit and less than the alarm lower limit, greater than the pre-warning upper limit and greater than the pre-warning lower limit, greater than the alarm upper limit and greater than the alarm lower limit, greater than the pre-warning upper limit and less than the alarm upper limit and greater than the pre-warning lower limit and less than the alarm lower limit, greater than the pre-warning upper limit and greater than the alarm upper limit and greater than the pre-warning lower limit and greater than the alarm lower limit.

[0009] Optionally, the method further comprises: determining the test data by a preset single-channel determination method / multi-channel determination method / embedded algorithm determination method / trigger determination method, and determining whether the data in the test data is low-quality data.

[0010] Optionally, according to the type of the low-quality data, the operation of the optimization method is determined, comprising: determining the parameter type of the low-quality data, wherein the parameter type comprises a calculation parameter and a measurement parameter; determining the low-quality data type of the parameter type, and determining the optimization method according to the low-quality data type, wherein the optimization method comprises a pre-test zero position optimization method, a post-test zero position optimization method, a difference optimization method, a linear optimization method, a post-test slope optimization method, an interpolation algorithm, data alignment, and a test data supplement point.

[0011] Optionally, the evaluation analysis graph comprises a data quality summary evaluation graph, a data quality summary evaluation table, and a data quality detailed evaluation graph.

[0012] According to another aspect of the present application, a test test data quality monitoring device is provided, comprising:

[0013] a first determination module configured to determine the quality state data of the test data according to the analysis characteristic result and the business logic preset by a user, wherein the quality state data comprises low-quality data and valid data;

[0014] a second determination module configured to determine the optimization method according to the type of the low-quality data, and optimize the low-quality data by the optimization method to determine valid test data, wherein the valid test data comprises the valid data and the optimized low-quality data;

[0015] a transmission module configured to transmit the valid test data and the test data to a terminal display device, and the terminal display device is configured to display the evaluation analysis graph of the valid test data and the test data.

[0016] According to still another aspect of the present application, there is provided a computer readable storage medium storing a computer program for performing the method according to any one of the preceding aspects of the present application.

[0017] According to still another aspect of the present application, there is provided an electronic device comprising: a processor; a memory for storing instructions executable by the processor; and the processor configured to read the executable instructions from the memory and execute the instructions to implement the method according to any one of the preceding aspects of the present application.

[0018] Thus, the present application realizes the screening of the individual with relatively poor data quality in the same group of sensors through intelligent analysis of data quality comparison, realizes the automatic optimization or replacement of the low-quality or error critical data filtered in the data processing software through test data optimization, and realizes the visual display of the overall condition of data quality, the summary of data problems and the condition of problem data through visual display of the test data quality of complex test systems. BRIEF DESCRIPTION OF DRAWINGS

[0019] The exemplary embodiments of the present application can be more fully understood with reference to the following drawings:

[0020] Figure 1 is a flowchart of the test data quality monitoring method provided by an exemplary embodiment of the present application;

[0021] Figure 2 is a flowchart of the abnormal data interpretation provided by an exemplary embodiment of the present application;

[0022] Figure 3a is a display diagram of the threshold test method provided by an exemplary embodiment of the present application;

[0023] Figure 3b is a display diagram of the feature comparison analysis method provided by an exemplary embodiment of the present application;

[0024] Figure 4a is a flowchart of the data optimization provided by an exemplary embodiment of the present application;

[0025] Figure 4b is a layout diagram of the 5-point steady-state total pressure comb measurement section of AIP provided by an exemplary embodiment of the present application;

[0026] Figure 5 is a schematic diagram of the data quality summary evaluation provided by an exemplary embodiment of the present application;

[0027] Figure 6 is a data quality summary evaluation table provided by an exemplary embodiment of the present application;

[0028] Figure 7 is a schematic diagram of data quality detailed evaluation provided by an exemplary embodiment of the present application;

[0029] Figure 8 is another schematic diagram of data quality detailed evaluation provided by an exemplary embodiment of the present application;

[0030] Figure 9 is a structural schematic diagram of a test data quality monitoring device provided by an exemplary embodiment of the present application;

[0031] Figure 10 is a structure of an electronic device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0032] In the following, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and the present application is not limited to the described exemplary embodiments.

[0033] It should be noted that: unless otherwise specified, the relative arrangement, numerical expression and values of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0034] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present application are only used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor do they represent the inevitable logical sequence between them.

[0035] It should also be understood that in the embodiments of the present application, "a plurality of" can mean two or more, and "at least one" can mean one, two or more.

[0036] It should also be understood that for any component, data or structure mentioned in the embodiments of the present application, unless specifically limited or given the opposite implication by the context, it can be understood as one or more in general.

[0037] In addition, the term "and / or" in the present application is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. In addition, the character " / " in the present application generally represents an "or" relationship between the front and rear associated objects.

[0038] It should also be understood that the description of each embodiment of the present application emphasizes the differences between each embodiment, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0039] It should be understood, of course, that the drawings are designed for the purpose of illustration only and not as a definition of the limits of the application.

[0040] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the application or its application or uses.

[0041] Techniques, methods, and apparatus known to those of ordinary skill in the relevant art can not be discussed in detail herein. However, where appropriate, techniques, methods, and apparatus should be considered as being part of the description of the application.

[0042] It is to be understood that the same can be employed in the same or similar form or in a modified form in the various embodiments of the application.

[0043] Embodiments of the application can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with numerous other general purpose or special purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations that can be suitable for use with terminal devices, computer systems, servers, etc. include, but are not limited to: personal computers, servers, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputers, mainframe computers, and distributed cloud computing environments that include any of the above systems or the like.

[0044] Terminal devices, computer systems, servers, etc. can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and the like, which perform particular tasks or implement particular abstract data types. Computer systems / servers can be practiced in distributed cloud computing environments with remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules can be located in local or remote computer system storage media including memory storage devices.

[0045] Exemplary method

[0046] Figure 1 FIG. 1 is a flowchart of a test data quality monitoring method according to an example embodiment of the application. The test data quality monitoring method 100 can be applied to electronic devices such as terminal devices, computer systems, servers, etc. Figure 1 As shown in FIG. 1, the test data quality monitoring method 100 includes the following steps:

[0047] In step 101, the quality state data of the test data is determined according to the analysis characteristic result preset by the user and the business logic, wherein the quality state data includes low-quality data and valid data.

[0048] The analysis characteristic result may be, for example, determination of the data quality state according to the average value, the maximum value, the minimum value and the like in the sensor test data in the sensor test data.

[0049] The business logic may be, for example, that the value range of the sensor test data is 1-2, and the data quality state is determined according to whether the value range is exceeded.

[0050] Specifically, the test data evaluation index is referred to Table 1, so that the data quality can be divided into the following eight evaluation indexes according to the data quality state.

[0051] Table 1

[0052]

[0053]

[0054] Thus, through the above-mentioned data quality comparison and intelligent analysis, the screening of the individual with relatively poor data quality in the same group of sensors can be realized.

[0055] Optionally, the low-quality data includes missing data, abnormal data, mutation data and non-standard data, and the method further includes: scoring different types of quality state data to determine the scores of different types of quality state data, wherein the score is used to indicate the grade of the quality state data, and the operation of scoring different types of quality state data to determine the scores of different types of quality state data includes: determining a first score of the valid data according to the proportion of the valid data in the test data; determining a second score of the missing data according to the proportion of the missing data in the test data; determining a third score of the abnormal data according to the proportion of the abnormal data in the test data; determining a fourth score of the mutation data according to the proportion of the mutation data in the test data; and determining a fifth score of the non-standard data according to the proportion of the non-standard data in the test data.

[0056] Specifically, the effective data amount scoring standard: only the effective data amount in the data is used as the data quality evaluation standard. The total data amount depends on the data range selected by the user (the user can select the same group of sensors, the same state and cross-section data, the same type of parameters, certain column parameter data, the same type of parameters in a certain period of time, etc.), and the data read by the user is the total data amount. The effective data amount refers to the amount of data remaining after removing the non-standard data amount, missing data amount, abnormal data amount, and mutation data amount in the selected data range. According to the calculation method in Table 2, the effective data amount in the selected range is scored according to the percentage of the total data amount.

[0057] Note: The total number of data for real-time analysis is about 1s-2s, and the user can set the time range.

[0058] Table 2

[0059]

[0060] The data missing amount scoring standard: only the data missing amount in the data is used as the data quality evaluation standard. The total data amount depends on the data range selected by the user (the user can select the same group of sensors, the same state and cross-section data, the same type of parameters, certain column parameter data, the same type of parameters in a certain period of time, etc.), and the data read by the user is the total data amount. According to the calculation method in Table 3, the missing amount in the selected range is scored according to the percentage of the total data amount.

[0061] Note: The total number of data for real-time analysis is about 1s-2s, and the user can set the time range.

[0062] Table 3

[0063]

[0064]

[0065] The abnormal data amount (data quality comparison intelligent analysis) scoring standard: the abnormal data determination method is the data quality comparison intelligent analysis method, which is divided into threshold test and feature comparison analysis algorithm. The user can select one of them to evaluate the quality of the same group of sensor data, filter out the number of abnormal data that do not meet the determination logic, and realize the filtering of the relatively poor individuals in the same group of sensors. The maximum number of data quality intelligent comparison analysis supporting the same group of sensor parameter comparison analysis is not less than 50, and the abnormal data amount is scored according to the method in Table 4.

[0066] Note: The total number of data for real-time analysis is about 1s-2s, and the user can set the time range.

[0067] Table 4

[0068]

[0069] Mutant data quantity scoring standard: Only the data quality evaluation standard of the mutant data quantity in the data. The total number of data depends on the data range selected by the user (the user can select the same group of sensors, the same state and cross-section data, the same type of parameters, certain column parameter data, the same type of parameters in a certain period of time, etc.) The data read as the total number of data, and according to the calculation method of Table 5, the mutant quantity in the selected range of data is scored according to the percentage of the total number of data.

[0070] Note: The total number of data for real-time analysis is about 1s-2s, and the user can set the time range.

[0071] Table 5

[0072]

[0073]

[0074] Non-standard data quantity scoring standard: Only the data quality evaluation standard of the non-standard data in the data. The total number of data depends on the data range selected by the user (the user can select the same group of sensors, the same state and cross-section data, the same type of parameters, certain column parameter data, the same type of parameters in a certain period of time, etc.) The data read as the total number of data, and according to the calculation method of Table 6, the non-standard data quantity in the selected range of data is scored according to the percentage of the total number of data.

[0075] Note: The total number of data for real-time analysis is about 1s-2s, and the user can set the time range.

[0076] Table 6

[0077]

[0078] Optionally, the method further comprises: detecting the test data by a pre-set threshold test method / feature comparison algorithm to determine the data quantity of abnormal data.

[0079] Specifically, in addition, the interpretation process of abnormal data is as follows (as shown in Figure 2 ).

[0080] 1. Range determination: Preferentially interpret whether the data is out of the sensor range

[0081] 2. Threshold test method or feature comparison algorithm: According to the interpretation method selected by the user, the abnormal data is determined

[0082] 3. Labeling the abnormal data and calculating the abnormal data quantity.

[0083] For example, the range data of the sensor in the sensor library is called (test link intelligent matching technology and verification platform) to determine the upper and lower limits of the sensor range. For data exceeding the upper and lower limits of the sensor, it is marked as abnormal data.

[0084] 1. Threshold test method

[0085] According to the historical data or user-defined upper and lower limits of the sensor data of this section, the data exceeding the threshold is determined as abnormal data. It can be used for real-time data collection or historical data analysis.

[0086] Threshold test is divided into: early warning value determination and alarm value determination, both early warning value and alarm value are hyperbolic curves, and the early warning value hyperbolic curve is contained in the alarm value hyperbolic curve, that is, the envelope curve. According to the comprehensive consideration of data and business, the user sets the early warning value and alarm value of the corresponding data. In the threshold test method, all data exceeding the upper and lower limits of the alarm value is determined as abnormal data.

[0087] Figure 3a That is, the threshold test method visual chart presentation example. The user can manually input the early warning value and alarm value on the right side, and the envelope curve displayed according to the user input is displayed together with the data on the left. The relevant data characteristics are displayed in the lower right.

[0088] All data exceeding the alarm value is determined as abnormal value. Its display list form is as follows table 7:

[0089] Table 7

[0090]

[0091] 2. Feature comparison analysis method

[0092] According to the quality comparison analysis of the same group of sensor change data, the maximum value and minimum value (such as: mean, variance, standard deviation, skewness, kurtosis, etc.) of each feature of the selected historical data under the same state are compared with the current data features. If it is found that it exceeds the maximum value and minimum value of the historical characteristics, it will alarm and be determined as abnormal data.

[0093] Feature comparison analysis method, that is, the user needs to select appropriate and most representative features (one or more) as criteria. The user selects the alarm feature according to the existing criteria. Calculate the data that does not meet the business logic in the same group of sensors. It can only be used for historical data analysis. As shown in the following figure, the user selects the criteria as the "extreme value" and "mean value" of the historical data, that is, generates the envelope curve of the maximum maximum value, minimum minimum value, maximum mean value, and minimum mean value of the corresponding historical data. At the same time, set the alarm value as "extreme value", that is, the data exceeding the maximum maximum value and minimum minimum value is marked as abnormal data.

[0094] Figure 3b That is, the feature contrast analysis method visual chart presentation example. Users can manually select criteria for (historical data) mean, extreme value (maximum and minimum), median, standard deviation, variance and other characteristic values, while setting the alarm basis (for which feature upper and lower limits). According to the user selected criteria to generate the corresponding envelope curve and data display on the right. Related data characteristics are shown in the lower right.

[0095] All data outside the user selected alarm value is determined to be an outlier. Its display list is as shown in Table 8:

[0096] Table 8

[0097]

[0098] Finally, whether the selected features are appropriate or the established threshold values are correct will directly affect the accuracy of the comprehensive evaluation. The construction of evaluation indexes should be widely involved in the industry data and data characteristics of the evaluation index device. The data quality is evaluated through correct and compliant qualitative analysis and quantitative analysis.

[0099] In order to facilitate data quality evaluation, the evaluation results are divided into "abnormal" and "normal" two cases. When the data quality does not meet the algorithm logic and business rules, the abnormal data is highlighted to the user in red alarm mode.

[0100] Regardless of which determination rule, the device will automatically calculate the number of current "abnormal" data, and display the abnormal data amount (compared with intelligent analysis) result to the user.

[0101] Optionally, the method further comprises: detecting the test data by a pre-set static mutation detection method / dynamic mutation detection method to determine the data amount of the mutation data.

[0102] Specifically, for example, monitoring of data quality decay rate of individuals in the same group of sensors, using data mutation detection algorithm, data quality decay rate of individuals is obtained. Existing mutation detection algorithms can be divided into two categories according to their definition of mutation: static mutation detection method and dynamic mutation detection method.

[0103] 1. Static mutation detection method: according to historical data or user-defined threshold, judge whether the data meets the limited threshold, to determine the data exceeding the threshold as low-quality data. It can be used for real-time data acquisition and historical data set. All values exceeding the user-set threshold are determined as mutation data (as shown in Table 8).

[0104] Table 8

[0105]

[0106] 2. Dynamic mutation detection method: i.e. the jump algorithm. Using adaptive mutation threshold for data stream detection, overcoming the lack of the first method. Can detect both upward and downward trends of mutation, and can exclude the interference of the mutation detection of the jitter.

[0107] According to the classification of certain single / sensor group, the quality mutation analysis of the data changing with time is carried out. The jump algorithm is used to calculate the data, and the data records with deviation greater than the index requirement N times are recorded as jump anomalies. The algorithm automatically calculates the jump times, and records the data with jump times greater than 0 as low-quality data, and records the data with jump times equal to 0 as normal data (as shown in Table 9).

[0108] Table 9

[0109]

[0110] Among them, whether the selection of algorithm parameters (index and N) is appropriate or the establishment of threshold value is correct will directly affect the accuracy of comprehensive evaluation. The construction of evaluation index should widely involve the evaluation index device industry data and algorithm characteristics. The data quality is evaluated through correct and compliant qualitative analysis and quantitative analysis.

[0111] In order to facilitate data quality evaluation, the evaluation results are divided into "mutation" and "normal" two cases. When the data quality does not meet the algorithm logic and business rules, the mutation data is highlighted to the user in the form of red alarm.

[0112] Thus, through the intelligent analysis of data quality mutation, the monitoring of the individual with relatively fast data quality decay rate in the same group of sensors is realized.

[0113] Optionally, the method further comprises: according to the pre-set determination logic, alarming and prompting the low-quality data, and determining the alarm prompt information, wherein the alarm prompt information includes a pre-alarm upper limit, a pre-alarm lower limit, an alarm upper limit, an alarm lower limit, the pre-alarm upper limit and the alarm upper limit, the pre-alarm lower limit and the alarm lower limit, the pre-alarm upper limit and the pre-alarm lower limit, the alarm upper limit and the alarm lower limit, the pre-alarm upper limit and the alarm upper limit and the pre-alarm lower limit and the alarm lower limit, the pre-alarm upper limit and the alarm upper limit and the pre-alarm lower limit and the alarm lower limit.

[0114] Specifically, the test data filtering realizes the automatic determination, identification, filtering and processing of low-quality or error data, so as to facilitate subsequent data optimization. The data filtering device can filter up to 200 parameters at the same time.

[0115] The data filtering determination alarm prompt logic is divided into two categories according to the early warning value and the alarm value. The early warning value and the alarm value are two envelope curves, and the upper and lower limits of the early warning value are within the upper and lower limits of the alarm value. If the data is within the upper and lower limits of the early warning value, it is considered normal data. If the data exceeds the upper and lower limits of the early warning value but does not reach the upper and lower limits of the alarm value, the data quality interpretation platform will make a "warning" prompt to the data. If the data exceeds both the early warning value and the upper and lower limits of the alarm value, it is determined to be an "alarm" prompt. Users can select a certain type of alarm prompt method according to the business data state and logic through the drop-down box, or select both types of alarm methods.

[0116] 1) early warning upper limit

[0117] 2) early warning lower limit

[0118] 3) alarm upper limit

[0119] 4) alarm lower limit

[0120] 5) early warning upper limit & alarm upper limit

[0121] 6) early warning lower limit & alarm lower limit

[0122] 7) early warning upper limit & early warning lower limit

[0123] 8) alarm upper limit & alarm lower limit

[0124] 9) early warning upper limit & alarm upper limit & early warning lower limit & alarm lower limit

[0125] 10) early warning upper limit & alarm upper limit & early warning lower limit & alarm lower limit

[0126] Among them, the data can be real-time data, historical data, feature data in the channel, or expressions.

[0127] Thus, the alarm prompt information of low-quality data is realized through uploading the determination logic, and the optimization of the data in the later period is facilitated. And the test data filtering realizes the automatic identification, filtering and processing of low-quality or error data

[0128] Optionally, the method further comprises: determining the test data by a pre-set single-channel determination method / multi-channel determination method / embedded algorithm determination method / trigger determination method, to determine whether the data in the test data is low-quality data.

[0129] Specifically, the determination methods for low-quality or error data include single-channel determination, multi-channel determination, embedded algorithm determination, and trigger determination.

[0130] Single channel judgment: whether the value of one channel is a wild value, to determine whether the data is qualified; unqualified invalid data cannot be output (including cannot be output to the interface to display, also cannot be output to the result file to save), but still needs to be recorded to the original data; each group of data can only configure one single channel judgment rule; single channel judgment rule can work together with multi-channel judgment rule and embedded algorithm judgment rule.

[0131] Multi-channel judgment: in the case that single channel judgment does not meet the requirements, multi-channel judgment is needed; according to multiple channels; according to the logical expression composed of multiple channels to determine whether the frame is valid; select multiple channels, take multiple channels as variables X1, X2, X3... and then edit the expression; the operators that can be used in the expression include +, -, *, / , %, ^, () and the like, and the functions that can be used include sum, average, power, variance, absolute value and the like.

[0132] Embedded algorithm judgment: when simple functions cannot meet the judgment requirements, user-provided judgment algorithms can be used for judgment. Whether there is low-quality or error value in the data is determined according to multiple channels, and the channel data is not fixed; the judgment algorithm is provided by the user's algorithm plug-in; the user algorithm is a simple C function provided by the dynamic library DLL; the user's C function is automatically loaded for judgment; which algorithm plug-in is used is configured in advance on the configuration interface of the data analysis protocol.

[0133] Trigger judgment: before the specified channel meets the condition, all data are judged as unqualified; after the trigger condition is met, all data of the judgment rule are judged as qualified. Note: even if the condition does not meet again, it is still judged as qualified because it has been triggered; in the rule, if the judgment is qualified, it does not affect other judgment rules, such as single channel judgment, multi-channel judgment, embedded algorithm judgment, and the like, if unqualified, the data is still unqualified.

[0134] In step S102, the optimization method is determined according to the type of low-quality data, and the low-quality data is optimized by the optimization method to determine the effective test data, wherein the effective test data includes the effective data and the optimized low-quality data.

[0135] Thus, through the test data optimization, the automatic optimization or replacement of the filtered low-quality or error key data in the data processing software is realized.

[0136] Optionally, with reference to Figure 4aAs shown, according to the type of low-quality data, the operation of determining the optimization method includes: determining the parameter type of the low-quality data, wherein the parameter type includes a calculation parameter and a measurement parameter; determining the low-quality data type of the parameter type, and determining the optimization method according to the low-quality data type, wherein the optimization method includes a pre-test zero position optimization method, a post-test zero position optimization method, a difference value optimization method, a linear optimization method, a post-test slope optimization method, an interpolation algorithm, data alignment, and a data supplementing strategy in a test.

[0137] Specifically, the device can realize the automatic optimization or replacement of the filtered low-quality or error critical data by the data processing device through the optimization of the filtered test data.

[0138] The data optimization method includes:

[0139] 1) Pre-test zero position optimization method: for inference and pressure data, optimization can be performed before the test, and the test zero position changes relatively stably under the condition;

[0140] 2) Post-test zero position optimization method: for inference and pressure data, optimization can be performed after the test, and the test zero position changes relatively stably under the condition;

[0141] 3) Difference value optimization method: only used for inlet pressure and pre-injection pressure, used for post-test optimization, and a fixed pressure difference exists under the optimization condition;

[0142] 4) Linear optimization method: for thrust and pressure data, suitable for post-test optimization, and the measurement zero position changes with the test time under the optimization condition;

[0143] 5) Post-test slope optimization method: for thrust, flow rate, and pressure data, used for post-test optimization, and the measurement sensor output is linearly distorted under the optimization condition, and seriously deviates from the normal value.

[0144] 6) Interpolation algorithm: according to the characteristics of the screened low-quality data, the interpolation algorithm is selected to obtain new data after interpolation.

[0145] a) Linear interpolation

[0146] b) Bilinear interpolation

[0147] 7) Data alignment: adjusting the data table structure to make it regular and convenient for further analysis.

[0148] 8) Truncation: truncating the required data table or data column to form new effective data.

[0149] 9) Data supplementing strategy in a test: supplementing strategy 1 and supplementing strategy 2. Reference Figure 4b As shown, wherein

[0150] Supplementing strategy 1

[0151] Pt1-2, Pt1-3, Pt1-4, Pt1-8, Pt1-7, Pt1-6 appear bad points:

[0152] 1) Replace the bad point with the symmetrical point. For example, Pt1-2-1 is damaged, and Pt1-8-1 is used to replace it.

[0153] 2) If the symmetrical point of the bad point is also damaged:

[0154] If the bad point is located in the 02-04 radial area, the arithmetic mean of the four adjacent points around the bad point is used to replace it. If one of the four adjacent points is also a bad point, the arithmetic mean of the two adjacent points in the circumferential or radial direction is used to replace it. If there are three consecutive bad points around the bad point, the measurement probe should be replaced.

[0155] If the bad point is located in the 01 or 05 radial area, the arithmetic mean of the two adjacent points in the circumferential direction is used to replace it. If one of the two adjacent points in the circumferential direction is also a bad point, the adjacent point on the same probe is used to replace it. If the replaced point is also a bad point, i.e. three consecutive bad points, the measurement probe should be replaced.

[0156] Supplementary point strategy 2

[0157] Pt1-1, Pt1-5 appear bad points:

[0158] 1) If the bad point is located in the 02-04 radial area, the arithmetic mean of the four adjacent points around the bad point is used to replace it. If one of the four adjacent points is also a bad point, the arithmetic mean of the two adjacent points in the circumferential or radial direction is used to replace it. If there are three consecutive bad points around the bad point, the measurement probe should be replaced.

[0159] 2) If the bad point is located in the 01 or 05 radial area, the arithmetic mean of the two adjacent points in the circumferential direction is used to replace it. If one of the two adjacent points in the circumferential direction is also a bad point, the adjacent point on the same probe is used to replace it. If the replaced point is also a bad point, i.e. three consecutive bad points, the measurement probe should be replaced.

[0160] 3) If the data of the bad point has been supplemented after using the above strategy, but the value is lower than the average static pressure, then the calculation of the steady-state distortion index is carried out according to the strategy of appearing backflow area (average static pressure is used to replace).

[0161] If there are no bad points in the AIP measurement section, only some points are lower than the average static pressure due to backflow area, then the average static pressure value is used to replace the point value for processing.

[0162] The data to be optimized or replaced. According to the low-quality data filtered by data quality intelligent analysis, low-quality data filtering and business logic screening, the user can select the corresponding optimization algorithm according to the low-quality data type (see 4.9.2 data quality analysis) and the business characteristics of the data (such as the same cross-section and state data group, the same group of sensor data, etc.), and optimize the low-quality data.

[0163] After interpreting the data quality, the device will group the low-quality data according to the parameter type of the calculation parameter and the measurement parameter, and then the user can select the corresponding optimization algorithm according to the low-quality data type, and select the replaceable optimization result. The correspondence between the optimization method and the low-quality data type is shown in Table 10.

[0164] Table 10

[0165]

[0166] In step S103, the valid test data and test data are transmitted to the terminal display device, and the terminal display device is used to display the evaluation analysis graph of the valid test data and test data.

[0167] Therefore, through the test data quality monitoring visualization, the overall situation of data quality, the summary situation of data problems and the visualization display of problem data are realized.

[0168] Optionally, the evaluation analysis graph includes a data quality summary evaluation graph, a data quality summary evaluation table and a data quality detailed evaluation graph.

[0169] Specifically, the data quality online monitoring and verification visualization is divided into two dimensions. The first dimension is the summary evaluation, which comprehensively and intuitively evaluates the data quality of the real-time collected data or the after-the-fact data. The user can clearly observe the data distribution graph of the selected data through the "summary evaluation graph", and according to the evaluation index of the data quality, the basic information is displayed, including the index name, the number of indexes, the percentage (the percentage of data meeting the index or not meeting the index in the total number) and the index evaluation result level (excellent, good, medium, poor). The user can also clearly understand the serial number, channel number, parameter type, parameter symbol, original value, physical unit, engineering value, engineering unit, expression (used to measure data quality) and data quality evaluation result (excellent, good, medium, poor) of the data in the selected data through the "summary evaluation table".

[0170] Note: During the test, only the data quality of the data collected within 1s to 10s is displayed, and the user can set the time range (within 10s) of the displayed incoming data. The data quality of all data can be displayed for the after-the-fact historical data.

[0171] The data quality summary evaluation graph is as follows:Figure 5 The left side is a data distribution diagram, and the right side shows basic information of selected data. A data quality summary evaluation table, such as Figure 6 .

[0172] The second dimension is a detailed evaluation, including a "summary evaluation diagram" and a "summary evaluation table" of the summary evaluation, and the data characteristics of the data selected by the user and a trend diagram can also be shown to the user. The user can select a corresponding visual graph according to the business requirements and the data characteristics.

[0173] A data quality detailed evaluation diagram, such as Figure 7 : The data trend (line chart) of the 0-0 section total pressure 1 (PT_01_1) is shown in the data line chart. The total number of data is 513, and the data characteristics of all data (including data that does not meet the data quality calculation judgment logic) are listed on the left. The user can select features such as average value, maximum value, minimum value, median, mode, standard deviation, and variance.

[0174] Another data quality detailed evaluation diagram, such as Figure 8 : The data trend (line chart) and data distribution (bar chart) of the 0-0 section total pressure 1 (PT_01_1) are shown in the data line chart. The total number of data is 426, and the data characteristics of only valid data (excluding data that does not meet the data quality calculation judgment logic) are listed on the left. The user can select features such as average value, maximum value, minimum value, median, mode, standard deviation, and variance.

[0175] Thus, the application realizes the screening of individuals with relatively poor data quality in the same group of sensors through intelligent analysis of data quality comparison, realizes the monitoring of individuals with relatively fast data quality decay rate in the same group of sensors through intelligent analysis of data quality mutation, realizes the automatic identification, filtering, and processing of low-quality or error data through test data filtering, realizes the automatic optimization or replacement of filtered low-quality or error key data in the data processing software through test data optimization, and realizes the visual display of the overall data quality, data problem summary, and problem data through test data quality monitoring visualization.

[0176] Exemplary apparatus

[0177] Figure 9 is a structural schematic diagram of a test data quality monitoring device provided by an exemplary embodiment of the application. As shown in Figure 9 , the device 900 includes:

[0178] A first determination module 910 is configured to determine the quality state data of the test data according to the analysis characteristic result and the business logic pre-set by the user, wherein the quality state data includes low-quality data and valid data.

[0179] The second determining module 920 is configured to determine an optimization method according to the type of the low-quality data, optimize the low-quality data by using the optimization method, and determine effective test data, wherein the effective test data includes the effective data and the optimized low-quality data.

[0180] The transmission module 930 is configured to transmit the effective test data and the test data to a terminal display device, and the terminal display device is configured to display an evaluation analysis graph of the effective test data and the test data. Optionally, the low-quality data includes missing data, abnormal data, mutation data and non-standard data. The apparatus 900 further includes a scoring module configured to score different types of quality state data and determine scores of the different types of quality state data, wherein the scores are used to indicate the levels of the quality state data, and the scoring module includes a first determining submodule configured to determine a first score of the effective data according to a proportion of the effective data in the test data, a second determining submodule configured to determine a second score of the missing data according to a proportion of the missing data in the test data, a third determining submodule configured to determine a third score of the abnormal data according to a proportion of the abnormal data in the test data, a fourth determining submodule configured to determine a fourth score of the mutation data according to a proportion of the mutation data in the test data, and a fifth determining submodule configured to determine a fifth score of the non-standard data according to a proportion of the non-standard data in the test data.

[0181] Optionally, the apparatus 900 further includes a third determining module configured to detect the test data by using a pre-set threshold test method / feature comparison algorithm, and determine a data amount of the abnormal data.

[0182] Optionally, the apparatus 900 further includes a fourth determining module configured to detect the test data by using a pre-set static mutation detection method / dynamic mutation detection method, and determine a data amount of the mutation data.

[0183] Optionally, the apparatus 900 further includes a fifth determining module configured to perform an alarm prompt on the low-quality data according to a pre-set judgment logic, and determine alarm prompt information, wherein the alarm prompt information includes a pre-alarm upper limit, a pre-alarm lower limit, an alarm upper limit, an alarm lower limit, the pre-alarm upper limit and the alarm upper limit, the pre-alarm lower limit and the alarm lower limit, the pre-alarm upper limit and the pre-alarm lower limit, the alarm upper limit and the alarm lower limit, the pre-alarm upper limit and the alarm upper limit and the pre-alarm lower limit and the alarm lower limit, the pre-alarm upper limit and the alarm upper limit and the pre-alarm lower limit and the alarm lower limit.

[0184] Optionally, the apparatus 900 further comprises a determining module configured to determine the test data by using a preset single-channel determining method / multi-channel determining method / embedded algorithm determining method / trigger determining method, and determine whether the data in the test data is low-quality data.

[0185] Optionally, the second determining module 920 comprises a sixth determining submodule configured to determine a parameter type of the low-quality data, wherein the parameter type comprises a calculation parameter and a measurement parameter; and a seventh determining submodule configured to determine a low-quality data type of the parameter type, and determine an optimization method according to the low-quality data type, wherein the optimization method comprises a pre-test zero position optimization method, a post-test zero position optimization method, a difference optimization method, a linear optimization method, a post-test slope optimization method, an interpolation algorithm, data alignment, and a test data supplement point.

[0186] Optionally, the evaluation analysis diagram comprises a data quality summary evaluation diagram, a data quality summary evaluation table, and a data quality detailed evaluation diagram.

[0187] Exemplary electronic device

[0188] Figure 10 is a structure of an electronic device provided by an exemplary embodiment of the present application. The electronic device can be either one or both of the first device and the second device, or a single device independent of them, which can communicate with the first device and the second device to receive the acquired input signals therefrom. Figure 10 Fig. 1 illustrates a block diagram of an electronic device according to an embodiment of the present application. As shown in Fig. 1, the electronic device 100 comprises one or more processors 101 and a memory 102. Figure 10

[0189] The processor 101 can be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction execution capabilities, and can control other components in the electronic device to perform desired functions.

[0190] ​The memory 102 can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program instructions can be stored on the computer-readable storage media, which the processor 101 can execute to implement the method of information mining on the history change record of the software program of the various embodiments of the present application and / or other desired functions described above. In one example, the electronic device can further include an input device 103 and an output device 104, which are interconnected through a bus system and / or other forms of connection mechanism (not shown).

[0191] In addition, the input device 103 can further include, for example, a keyboard, a mouse, and / or the like.

[0192] The output device 104 can output various information to the outside. The output device 104 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and / or the like.

[0193] Of course, in order to simplify, Figure 10 Only some of the components of the electronic device related to the present application are shown in FIG. 1, and components such as a bus, an input / output interface, and / or the like are omitted. In addition, the electronic device can further include any other appropriate components according to a specific application.

[0194] Exemplary computer program product and computer readable storage medium

[0195] In addition to the above-described method and device, an embodiment of the present application can be a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the method of information mining on the history change record according to various embodiments of the present application described in the above "Exemplary Method" section of the specification.

[0196] The computer program product can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, C++, and / or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote cloud device or server.

[0197] Furthermore, embodiments of the present invention may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the methods for information mining of historical change records according to various embodiments of the present invention as described in the "Exemplary Methods" section above.

[0198] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0199] The basic principles of the present invention have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in the present invention are merely examples and not limitations, and should not be considered as essential features of each embodiment of the present invention. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the present invention to the necessity of employing the aforementioned specific details.

[0200] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0201] The block diagrams of devices, systems, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, systems, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0202] The methods and systems of the present application can be implemented in numerous ways. For example, the methods and systems of the present application can be implemented using software, hardware, firmware, or any combination of software, hardware, and firmware. The above described order of steps for the methods is merely illustrative, and the steps of the methods of the present application are not limited to the order described above unless otherwise specifically stated. Furthermore, in some embodiments, the present application can also be implemented as a program recorded on a recording medium, which includes machine readable instructions for implementing the methods according to the present application. Thus, the present application also covers recording media storing programs for executing the methods according to the present application.

[0203] It is also to be noted that in the systems, devices, and methods of the present application, the various components and steps are as can be split into separate components and / or steps, or combined into a single component and / or step. Such splitting and / or combination is to be considered an equivalent scope within the scope of the present application. The above description of the disclosed aspects is given for illustrative purposes and is not intended to limit the scope of the application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0204] The above description has been presented for the purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although several example aspects and embodiments have been discussed, those skilled in the art will recognize certain variations, modifications, changes, additions, and sub-combinations thereof.

Claims

1. A method for monitoring the quality of experimental test data, characterized in that, The method comprises the following steps: determining quality state data of test data according to analysis characteristic results preset by a user and business logic, wherein the quality state data comprises low-quality data and valid data; determining an optimization method according to a type of the low-quality data, optimizing the low-quality data by the optimization method, and determining valid test data comprising the valid data and the optimized low-quality data; transmitting the valid test data and the test data to a terminal display device for displaying evaluation analysis graphs of the valid test data and the test data; the low-quality data comprises missing data, abnormal data, mutation data and non-standard data, and the method further comprises the following steps: scoring different types of the quality state data to determine scores of different types of the quality state data, wherein the scores are used to indicate levels of the quality state data, and the operation of scoring the different types of the quality state data to determine the scores of different types of the quality state data comprises the following steps: determining a first score of the valid data according to a proportion of the valid data in the test data; determining a second score of the missing data according to a proportion of the missing data in the test data; determining a third score of the abnormal data according to a proportion of the abnormal data in the test data; determining a fourth score of the mutation data according to a proportion of the mutation data in the test data; determining a fifth score of the non-standard data according to a proportion of the non-standard data in the test data; determining, by a preset single-channel judgment method, a multi-channel judgment method, an embedded algorithm judgment method or a trigger judgment method, whether data in the test data is the low-quality data; the operation of determining the optimization method according to the type of the low-quality data comprises the following steps: determining a parameter type of the low-quality data, wherein the parameter type comprises a calculation parameter and a measurement parameter; determining a low-quality data type of the parameter type, and determining the optimization method according to the low-quality data type, wherein the optimization method comprises a pre-test zero position optimization method, a post-test zero position optimization method, a difference value optimization method, a linear optimization method, a post-test slope optimization method, an interpolation algorithm, data alignment and test data point supplementing.

2. The method of claim 1, wherein, The method further comprises the following steps: detecting the test data by a preset threshold test method or a feature comparison algorithm to determine a data amount of the abnormal data.

3. The method of claim 1, wherein, The method further comprises the following steps: detecting the test data by a preset static mutation detection method or a dynamic mutation detection method to determine a data amount of the mutation data.

4. The method of claim 1, wherein, The method further comprises the following steps: According to a preset determination logic, the low-quality data is prompted for an alarm, and alarm prompt information is determined, wherein the alarm prompt information includes a pre-warning value and an alarm value, and the determination logic includes greater than a pre-warning upper limit, greater than a pre-warning lower limit, greater than an alarm upper limit, greater than an alarm lower limit, greater than the pre-warning upper limit and less than the alarm upper limit, greater than the pre-warning lower limit and less than the alarm lower limit, greater than the pre-warning upper limit and greater than the pre-warning lower limit, greater than the alarm upper limit and greater than the alarm lower limit, greater than the pre-warning upper limit and less than the alarm upper limit and greater than the pre-warning lower limit and less than the alarm lower limit, greater than the pre-warning upper limit and greater than the alarm upper limit and greater than the pre-warning lower limit and greater than the alarm lower limit.

5. The method of claim 1, wherein, The evaluation analysis diagram includes a data quality summary evaluation diagram, a data quality summary evaluation table, and a data quality detailed evaluation diagram.

6. A test data quality monitoring device, characterized by It includes: A first determination module is configured to determine quality state data of test data according to analysis characteristic results and business logic preset by a user, wherein the quality state data includes low-quality data and valid data; A second determination module is configured to determine an optimization method according to a type of the low-quality data, and to optimize the low-quality data by the optimization method to determine valid test data, wherein the valid test data includes the valid data and the low-quality data after optimization; A transmission module is configured to transmit the valid test data and the test data to a terminal display device, and the terminal display device is configured to display an evaluation analysis diagram of the valid test data and the test data; The low-quality data includes missing data, abnormal data, mutation data, and non-standard data, and the device further includes: A scoring module is configured to score different types of quality state data to determine scores of different types of quality state data, wherein the scores are used to indicate levels of the quality state data, and The scoring module includes: determining a first score of the valid data according to a proportion of the valid data in the test data; determining a second score of the missing data according to a proportion of the missing data in the test data; determining a third score of the abnormal data according to a proportion of the abnormal data in the test data; determining a fourth score of the mutation data according to a proportion of the mutation data in the test data; determining a fifth score of the non-standard data according to a proportion of the non-standard data in the test data; The device further includes a determination module configured to determine whether data in the test data is the low-quality data by a preset single-channel determination method, a multi-channel determination method, an embedded algorithm determination method, or a trigger determination method; The operation of determining an optimization method in the second determination module according to the type of the low-quality data includes: determining a parameter type of the low-quality data, wherein the parameter type includes a calculation parameter and a measurement parameter; determining a low-quality data type of the parameter type, and determining the optimization method according to the low-quality data type, wherein the optimization method comprises pre-test zero-position optimization method, post-test zero-position optimization method, difference optimization method, linear optimization method, post-test slope optimization method, interpolation algorithm, data alignment, and data point supplementing in a test.

7. A computer readable storage medium characterized in that, The storage medium stores a computer program, and the computer program is used for executing the method in any one of claims 1-5.

8. An electronic device, comprising: The electronic device comprises: a processor; a memory for storing executable instructions of the processor; the processor is used for reading the executable instructions from the memory and executing the instructions to implement the method in any one of claims 1-5.

Images