A data quality detection method, device, equipment and medium
By employing a hierarchical data quality inspection method, the configuration information of the target data inspection level is used to inspect the data of the cooling plant system. This method solves the problem of efficiently locating and resolving data quality issues in the cooling plant system, improving inspection efficiency and reducing workload.
Patent Information
- Application Number
- CN202210038120.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-13
- Publication Date
- 2026-02-13
- Estimated Expiration
- 2042-01-13
AI Technical Summary
Data quality issues in the cooling plant system lead to operational instability, and existing technologies are insufficient to efficiently locate and resolve these problems, resulting in equipment control issues and high costs.
A hierarchical data quality inspection method is adopted. The data to be inspected is inspected based on the configuration information of the target data inspection level. Only when the inspection result of the previous level is qualified can it proceed to the next level, and the inspection of unqualified data ends directly.
It improves the efficiency of data quality inspection, reduces workload and manpower costs, and ensures that data quality problems are located and resolved in a timely manner.
Smart Images

Figure CN114398357B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] Embodiments of the present application relate to computer technology, and particularly to a data quality detection method and device, equipment and medium. BACKGROUND
[0002] With the development of science and technology, in recent years, buildings have greatly improved in terms of scale and intelligence. Compared with before, people have higher requirements for the running time, load capacity, safety and energy saving level of air conditioning units. In this case, it is necessary to keep the stability of the cold station system.
[0003] The cold station system has large data volume and many data types, which leads to the complexity and inconsistency of the cold station system management, thereby causing the demand for automatic and intelligent control management and improving the requirement for data quality. Data quality problems can cause the cold station system to run unstably and cause a series of equipment operation control problems. Factors that cause data quality problems include sensor failure, collector problems, network communication problems, and collection program interruption, so it is difficult to locate the data quality problem and the cost of problem location is high. SUMMARY
[0004] Embodiments of the present application provide a data quality detection method, device, equipment and medium, which detects the quality of the data to be detected by hierarchical levels, can directly end the detection process when the quality detection is unqualified, improves the efficiency of data quality detection, and reduces the workload of data quality detection.
[0005] In a first aspect, the embodiments of the present application provide a data quality detection method, which comprises:
[0006] When it is detected that the data to be detected is input into a target data detection level, the quality of the data to be detected is detected according to target data detection configuration information associated with the target data detection level; the data to be detected is collected from at least two information points in a cold station system;
[0007] According to the quality detection result of the data to be detected, the data to be detected is transmitted to the next data detection level for quality detection.
[0008] In a second aspect, the embodiments of the present application also provide a data quality detection device, which comprises:
[0009] The data quality detection module is configured to detect the quality of the data to be detected according to the target data detection configuration information associated with the target data detection level when it is detected that the data to be detected is input into the target data detection level; the data to be detected is collected from at least two information points in a cold station system;
[0010] The data transmission module is configured to transmit the to-be-detected data to a next data detection level for quality detection according to the quality detection result of the to-be-detected data.
[0011] In a third aspect, an electronic device is provided, comprising:
[0012] one or more processors;
[0013] a memory configured to store one or more programs;
[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the data quality detection method provided by any of the embodiments of the present application.
[0015] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. When the program is executed by a processor, the data quality detection method provided by any of the embodiments of the present application is implemented.
[0016] The technical solution of the embodiments of the present application detects the to-be-detected data input into the target data detection level, performs quality detection on the to-be-detected data according to the target data detection configuration information associated with the target data detection level, and then transmits the to-be-detected data to a next data detection level for quality detection according to the quality detection result of the to-be-detected data. The quality detection of the to-be-detected data is performed in a hierarchical manner. Only when the detection result of the previous level meets the set condition, the to-be-detected data will be detected in the next level. The data quality detection efficiency is improved, and the workload of data quality detection is reduced. BRIEF DESCRIPTION OF DRAWINGS
[0017] Figure 1 is a flowchart of a data quality detection method in the first embodiment of the present application;
[0018] Figure 2 is a flowchart of a data quality detection method in the second embodiment of the present application;
[0019] Figure 3 is a flowchart of a data quality detection method in the third embodiment of the present application;
[0020] Figure 4 is a structural schematic diagram of a data quality detection device in the fourth embodiment of the present application;
[0021] Figure 5 is a structural schematic diagram of a device provided by the fifth embodiment of the present application. DETAILED DESCRIPTION
[0022] The application will be described in further detail below with reference to the drawings and embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. In addition, it should be noted that only the parts related to the present application are shown in the drawings for the convenience of description.
[0023] Embodiment one
[0024] Figure 1 For a flow chart of a data quality detection method in the embodiment one of the present application, the technical solution of the embodiment is applicable to the case of detecting the data quality of the cold station system in a hierarchical manner. The method can be executed by a data quality detection device, which can be realized by software and / or hardware and can be integrated in various general-purpose computer devices. The data quality detection method in the embodiment specifically includes the following steps:
[0025] In step 110, when it is detected that the to-be-detected data is input into the target data detection level, the quality of the to-be-detected data is detected according to the target data detection configuration information associated with the target data detection level. The to-be-detected data is obtained by at least two information points in the cold station system.
[0026] For the convenience of understanding, the cold station system is first explained. The cold station system includes multiple data branches that need to be detected, and each data branch includes multiple information points. Specifically, the cold station system includes five data branches that need to be detected, namely, a system branch, a cold machine branch, a refrigeration pump branch, a cooling pump branch, and a cooling tower branch.
[0027] An information point is used to represent a unique sensor, and the information point is a static description information of the sensor. For example, the information point is the ID number of the sensor. Each data branch includes multiple information points, that is, in each data branch, multiple information points for data acquisition are included. For example, the system branch includes five information points, and the data collected by the five information points are: chilled water supply temperature setting, cold station hourly cumulative cooling capacity, chilled water main supply temperature, and chilled water main return temperature. The cold machine branch includes seven information points, and the data collected by the seven information points are: last hour power consumption, chilled water outlet temperature setting, chilled water supply temperature, chilled water return temperature, cooling water outlet temperature, cooling water return temperature, and current percentage. The refrigeration pump branch includes two information points, and the data collected by the two information points are: frequency feedback of the frequency converter and point power. The cooling pump branch includes two information points, and the data collected by the two information points are: frequency feedback of the frequency converter and point power. The cooling tower branch includes two information points, and the data collected by the two information points are: frequency feedback of the frequency converter and point power.
[0028] In the embodiment, the data to be detected can be detected in a hierarchical manner according to a preset data quality detection sequence. Only when the detection result of the previous level is qualified, the data to be detected can be transmitted to the next level for quality detection. During the data quality detection process, the level at which the data to be detected is currently detected can be regarded as a target data detection level. When the data to be detected is detected to be input to the target data detection level, target data detection configuration information associated with the target data detection level is acquired, and then the data to be detected is subjected to target data detection of the current level according to the target data detection configuration information. The target data detection configuration information can be a data threshold or a calculation method of a data index for quality detection of the data to be detected, for example, a data threshold for dividing the data quality of a certain type of data, or a method for calculating the probability of fault data.
[0029] In a specific example, the target data detection level is an information point missing detection level, which is used to detect whether the data collected by a certain information point is missing in the data to be detected, that is, whether a type of data required to be collected in the cold station system is missing. When the data to be detected is detected to be input to the information point missing detection level, all information points contained in the cold station system can be read from the database. Further, the data to be detected is mapped to the corresponding information points. When one or more information points do not have corresponding data, it can be determined that the data to be detected has a problem of missing information points. In this example, the target data detection configuration information is the information points contained in the cold station system.
[0030] In step 120, according to the quality detection result of the data to be detected, the data to be detected is transmitted to the next data detection level for quality detection.
[0031] In the embodiment of the disclosure, after the data to be detected is subjected to quality detection at the target data detection level and the quality detection result corresponding to the level is obtained, the data to be detected can be transmitted to the next data detection level for quality detection according to the quality detection result of the data to be detected. Specifically, when the detection result obtained at the target data detection level is that the data is available, the data to be detected can be transmitted to the next data detection level for quality detection. When the detection result obtained at the target data detection level is that the data is unavailable, the data quality detection of the next level is not performed, and the data quality detection result corresponding to the target data detection level is directly output.
[0032] In a specific example, the target data detection level is an information point missing detection level, in which when it is determined that the to-be-detected data has an information point missing problem, the detection process is directly ended, and information of the information point missing is displayed, and the to-be-detected data is no longer transmitted to the next data detection level. On the contrary, if it is determined that the to-be-detected data does not have an information point missing problem, the to-be-detected data can be continuously transmitted to the next data detection level for data quality detection. For example, the next data detection level is a data completeness detection level.
[0033] The technical scheme of the embodiment of the present application, when detecting that the to-be-detected data is input into the target data detection level, performs quality detection on the to-be-detected data according to the target data detection configuration information associated with the target data detection level, and then transmits the to-be-detected data to the next data detection level for quality detection according to the quality detection result of the to-be-detected data, so as to realize quality detection on the to-be-detected data in a hierarchical manner. Only when the detection result of the previous level meets the set condition, the detection in the next level is performed, which improves the data quality detection efficiency and reduces the workload of data quality detection.
[0034] Embodiment Two
[0035] Figure 2 For the flowchart of the data quality detection method in the embodiment two of the present application, the embodiment further refines the above-mentioned embodiment, and provides specific steps of performing quality detection on the to-be-detected data according to the target data detection configuration information associated with the target data detection level, and specific steps of transmitting the to-be-detected data to the next data detection level for quality detection according to the quality detection result of the to-be-detected data. The following will be described in combination with Figure 2 The data quality detection method provided in the embodiment two of the present application is described, including the following steps:
[0036] In step 210, when detecting that the to-be-detected data is input into the target data detection level, quality detection is performed on the to-be-detected data according to the target data detection configuration information associated with the target data detection level; the to-be-detected data is obtained by at least two information points in a cold station system.
[0037] Optionally, the target data detection level is at least one of an information point missing detection level, a data completeness detection level, a single branch data detection level, a multi-branch relationship detection level, a card number judgment level, a cold quantity table data detection level, and a sample data screening level.
[0038] In the optional embodiment, according to the data quality detection requirement in the cold station system, the target data detection level can be at least one of the information point missing detection level, the data completeness detection level, the single branch data detection level, the multi-branch relationship detection level, the card number judgment level, the cold quantity table data detection level and the sample data screening level.
[0039] Optionally, according to the target data detection configuration information associated with the target data detection level, the quality detection of the to-be-detected data includes:
[0040] In the case where the target data detection level is the information point missing detection level, at least two information points contained in the cold station system are read from the database.
[0041] The data associated with each information point is determined in the to-be-detected data.
[0042] In the case where the data amount associated with at least one information point is 0, the quality detection result of the to-be-detected data is determined as data unavailable.
[0043] In the optional embodiment, a specific way of quality detection of to-be-detected data according to target data detection configuration information associated with the target data detection level is provided: in the case where the target data detection level is the information point missing detection level, it is needed to detect whether the data related to a certain information point is missing in the to-be-detected data. Specifically, first, the information points contained in the cold station system are read from the database, and then the to-be-detected data and each information point are matched to obtain the data corresponding to each information point in the to-be-detected data. When one or more information points do not have corresponding data, it can be determined that the instruction detection result of the to-be-detected data is data unavailable.
[0044] In a specific example, first, 3 information points contained in the cold station system are read from the database, and then the data corresponding to the above 3 information points is searched in the to-be-detected data, for example, the first information point has 100 pieces of data, the second information point has 150 pieces of data, and the third information point does not have corresponding data, then it is determined that the data of the third information point is missing. At this time, it can be determined that the instruction detection result of the to-be-detected data is data unavailable.
[0045] Optionally, according to the target data detection configuration information associated with the target data detection level, the quality detection of the to-be-detected data includes:
[0046] In the case where the target data quality detection level is the data completeness detection level, the proportion of complete data is calculated according to the collection time of the to-be-detected data.
[0047] In a case where the span of the collection time is greater than the first time threshold and the complete data proportion is greater than the set proportion threshold, the quality detection result of the to-be-detected data is determined as data available.
[0048] In the optional embodiment, another specific manner of quality detection of the to-be-detected data according to the target data detection configuration information associated with the target data detection level is provided: in a case where the target data quality detection level is the data completeness detection level, the data completeness of the data associated with each information point needs to be detected. Specifically, first, the complete data proportion of each information point corresponding data is calculated according to the collection time of the to-be-detected data, and the minimum value in the complete data proportions of the information points is taken as the complete data proportion of the system. Further, in a case where the span of the collection time is greater than the first time threshold and the complete data proportion of the system is greater than the set proportion threshold, the instruction detection result of the to-be-detected data can be determined as data available.
[0049] In an example of acquiring cold start data, first, the time range of the cooling season is limited to June, July, August and September, and the time range of the transition season is limited to April, May, October and November. For each information point, according to the collection time of the to-be-detected data, the length of time that the information point collects data and the total time span of collecting data of the information point are determined, and the complete data proportion of the information point is obtained by calculating the ratio between the length of time that the data is collected and the total time span of collecting data. For example, the data collection period of the information point is 15 minutes, and in theory, 8 groups of data will be collected in two hours, but actually, only 5 groups of data are included in the two hours of data of the information point, and 3 groups of data are missing, so the complete data proportion is 5 / 8. Similarly, the complete data proportions of the information points are calculated in turn, and the minimum value is taken as the complete data proportion of the cold station system. In a case where the total time span of collecting data in the cooling season or the transition season is greater than two months and the complete data proportion is greater than 90%, the quality detection result of the to-be-detected data is determined as data available. In a case where the above two conditions are not simultaneously satisfied, the quality detection result is determined as data unavailable, and the detection process is directly ended without being transmitted to the next detection level for quality detection.
[0050] Optionally, the quality detection of the to-be-detected data according to the target data detection configuration information associated with the target data detection level includes:
[0051] In a case where the target data quality detection level is the data completeness detection level, the complete data proportion is calculated according to the collection time of the to-be-detected data;
[0052] In a case where the complete data proportion is greater than the set proportion threshold, the quality detection result of the to-be-detected data is determined as data available.
[0053] In an example of acquiring cold station regulation data, the data input to the data integrity detection level is all data collected within 7 days. For each information point, the length of time for which data is collected for the information point is determined according to the collection time of the data associated with the information point, and the total time span for which data is collected for the information point is determined, and the proportion of complete data for the information point is obtained by calculating the ratio between the length of time for which data is collected and the total time span for which data is collected. The minimum value among the proportions of complete data corresponding to the information points can be selected as the proportion of complete data for the cold station system. If the proportion of complete data is greater than 90%, the quality detection result of the to-be-detected data is determined to be a high probability of data being available, if the proportion of complete data is greater than 50% and less than or equal to 90%, the quality detection result of the to-be-detected data is determined to be a low probability of data being available, and if the proportion of complete data is less than or equal to 50%, the detection process is directly ended and the to-be-detected data is not transmitted to the next detection level for quality detection.
[0054] Optionally, the quality of the to-be-detected data is detected according to the target data detection configuration information associated with the target data detection level, including:
[0055] In a case where the target data quality detection level is a single-branch data detection level, the fault data is determined in the to-be-detected data according to the data detection configuration information associated with the single-branch data detection level.
[0056] The total data amount corresponding to the information point and the fault data amount corresponding to the information point are determined, and the data fault probability of the information point is calculated according to the total data amount and the fault data amount.
[0057] The data fault probability of each data branch to which the information points belong is determined according to the data fault probability of each information point. A data branch includes at least one information point.
[0058] In a case where the data fault probability of at least one data branch is greater than or equal to a set probability threshold, the quality detection result of the to-be-detected data is determined to be data unavailable.
[0059] In the optional embodiment, another specific manner of quality detection of the to-be-detected data according to the target data detection configuration information associated with the target data detection level is provided: in the case that the target data quality detection level is a single-branch data detection level, the data quality in each branch of the cold station system needs to be detected. Specifically, first, the faulty data in the to-be-detected data is determined according to the data detection configuration information associated with the single-branch data detection level. For example, the data detection configuration information contains the quality judgment threshold of the data associated with each information point in each branch. According to the quality judgment threshold, the quality data of each information point can be divided. Further, for each information point, the total data quantity corresponding to the information point and the faulty data quantity corresponding to the information point are determined, and the data fault probability of the information point is obtained by calculating the ratio between the faulty data quantity and the total data quantity. Further, for each data branch, the data fault probability of each information point associated with the branch can be determined respectively, and the maximum value in the data fault probability is selected as the data fault probability corresponding to the branch. Finally, in the cold station system, if the data fault probability of one or more data branches is greater than or equal to 20%, it is determined that the quality detection result of the to-be-detected data is unusable. Otherwise, it is determined that the quality detection result of the to-be-detected data is usable. Specifically, when the data fault probability of the data branch is less than 5%, it is determined that the data quality of the branch is highly probable to be usable, and when the data fault probability of the data branch is greater than or equal to 5% and less than 20%, it is determined that the data quality of the branch is lowly probable to be usable.
[0060] In a specific example, the data detection configuration information associated with the single-branch data detection level is shown in Table 1. The faulty data in the data corresponding to each information point can be determined according to the judgment limit items in Table 1. Specifically, when the data satisfies the judgment limit condition of the corresponding information point, it is determined that the data is normal data, otherwise, it is determined that the data is faulty data.
[0061] Table 1
[0062]
[0063]
[0064] Optionally, the quality detection of the to-be-detected data according to the target data detection configuration information associated with the target data detection level comprises:
[0065] In the case that the target data quality detection level is a multi-branch relationship detection level, the data associated with the information points belonging to the first data branch in the to-be-detected data is compared with the data associated with the information points belonging to the second data branch; and / or, the data associated with different information points belonging to the second data branch in the to-be-detected data is compared;
[0066] According to the comparison result, determine the fault data volume associated with the information point to which the compared data belongs;
[0067] According to the fault data volume;
[0068] Calculate the data fault probability of the information point to which the compared data belongs;
[0069] In the case where the data fault probability of at least one information point is greater than or equal to the set probability threshold, determine that the quality detection result of the to-be-detected data is that the data is unavailable.
[0070] In this optional embodiment, another specific way of quality detection of to-be-detected data according to target data detection configuration information associated with the target data detection level is provided: in the case where the target data quality detection level is a multi-branch relationship detection level, the association relationship between data of different branches needs to be detected. Specifically, the data associated with the information point belonging to the first data branch in the to-be-detected data is compared with the data associated with the information point belonging to the second data branch, for example, the data associated with the information point belonging to the first data branch refers to the chilled water supply temperature setting in the system branch, and the data associated with the information point belonging to the second data branch refers to the chilled water outlet temperature setting of the chiller branch. It can also be that the data associated with different information points belonging to the second data branch is compared, for example, the data associated with different information points belonging to the second data branch can be the chilled water supply temperature and the chilled water return temperature belonging to the chiller branch, and can also be the cooling water outlet temperature and the cooling water return temperature.
[0071] After the comparison is completed, the fault data volume associated with the information point to which the compared data belongs can be determined according to the comparison result, and the data fault probability of the information point to which the compared data belongs can be calculated according to the fault data volume, for example, in the case where the comparison result meets the set condition, it is determined that the compared data is normal data, otherwise, the compared data is fault data. In the case where the data fault probability of one or more information points is greater than or equal to the set probability threshold, it is determined that the quality detection result of the to-be-detected data is that the data is unavailable.
[0072] In a specific example, the fault data is obtained according to the following judgment conditions:
[0073] a. Determine whether the hourly cumulative cooling capacity of the cold station in the system branch is less than the total rated cooling capacity of the started chiller * 1.2, if yes, determine that the current judged data is normal data, otherwise, determine that it is fault data;
[0074] b. Determine whether the chilled water supply temperature setting in the system branch is equal to the chilled water outlet temperature setting of the chiller branch, if yes, determine that the current judged chilled water supply temperature setting is normal data, otherwise, determine that it is fault data;
[0075] c. judging whether the chilled water supply temperature in the cold machine branch is less than the chilled water return temperature, if yes, determining that the current judged chilled water supply temperature is normal data, otherwise, determining as fault data;
[0076] d. judging whether the cooling water supply temperature in the cold machine branch is less than the cooling water return temperature, if yes, determining that the current judged cooling water supply temperature is normal data, otherwise, determining as fault data.
[0077] Through the above comparison process, after determining the fault data, the data fault probability corresponding to the information point of the accumulated cooling supply data of the cold station per hour, the data fault probability corresponding to the information point of the set chilled water supply temperature, the data fault probability corresponding to the information point of the collected chilled water supply temperature, and the data fault probability corresponding to the information point of the collected cooling water supply temperature are calculated. Among them, for each information point, the data fault probability calculation method is to calculate the ratio of the fault data amount associated with the information and the total data amount associated with the information point. Further, the maximum value of the data fault probability of the above four information points is taken as the final data fault probability. When the final data fault probability is greater than or equal to 20%, it is determined that the quality detection result of the to-be-detected data is unusable. Otherwise, it is determined that the quality detection result of the to-be-detected data is usable. Specifically, when the final data fault probability is less than 5%, it is determined that the quality of the to-be-detected data is highly probable to be usable, and when the final data fault probability is greater than or equal to 5% and less than 20%, it is determined that the data quality of the to-be-detected data is small probability to be usable.
[0078] Optionally, the quality of the to-be-detected data is detected according to the target data detection configuration information associated with the target data detection level, including:
[0079] In the case that the target data quality detection level is the card number judgment level, the maximum duration that the data associated with at least one information point in the to-be-detected data remains unchanged is determined.
[0080] If the maximum duration exceeds the second time threshold, it is determined that the quality detection result of the to-be-detected data is data unusable.
[0081] In this optional embodiment, another specific way of detecting the quality of the to-be-detected data according to the target data detection configuration information associated with the target data detection level is provided: in the case that the target data quality detection level is the card number judgment level, it is necessary to judge whether each information point in the to-be-detected data appears the card number phenomenon. Specifically, the maximum duration that the data associated with at least one information point in the to-be-detected data remains unchanged is determined, and if the maximum duration exceeds the pre-set second time threshold, it is determined that the quality detection result of the to-be-detected data is data unusable, otherwise, it is determined that the quality detection result of the to-be-detected data is data usable.
[0082] In one specific example, according to the collection time of the to-be-detected data, the maximum duration of data remaining unchanged in the data corresponding to each information point of the to-be-detected data is determined. Further, the maximum duration is compared with the second time threshold value corresponding to each information point, and if the maximum duration exceeds the second time threshold value, it is determined that the quality detection result of the to-be-detected data is that the data is unavailable. The second time threshold value corresponding to each information point is shown in Table 2.
[0083] Table 2
[0084]
[0085] Optionally, the quality of the to-be-detected data is detected according to the target data detection configuration information associated with the target data detection level, including:
[0086] In the case where the target data quality detection level is a cold meter data detection level, at least one target information point associated with the system branch is determined, and according to the data detection configuration information associated with the cold meter data detection level, the fault data in the data associated with the target information point is determined.
[0087] According to the fault data volume and the total data volume corresponding to the target information point, the data fault probability of each target information point is calculated.
[0088] In the case where the data fault probability of at least one target information point is greater than or equal to a set probability threshold, it is determined that the quality detection result of the to-be-detected data is that the data is unavailable.
[0089] In this optional embodiment, another specific way of detecting the quality of the to-be-detected data according to the target data detection configuration information associated with the target data detection level is provided: in the case where the target data quality detection level is a cold meter data detection level, the target information point associated with the system branch needs to be determined in the cold machine start state, and according to the data detection configuration information associated with the cold meter data detection level, the fault data in the data associated with the target information point is determined. The data detection configuration information can be threshold data for dividing data quality. Further, for each target information point, the data fault probability of the target information point can be calculated according to the fault data volume and the total data volume corresponding to the target information point. Finally, in the case where the data fault probability of one or more target information points is greater than or equal to a set probability threshold, it is determined that the quality detection result of the to-be-detected data is that the data is unavailable.
[0090] In one specific example, it is determined that there are four target information points associated with the system branch, and the data collected by the four target information points are chilled water main supply water temperature, chilled water main return water temperature, and chilled water main flow rate. According to the following determination conditions, the fault data is determined from the data collected by the four target information points:
[0091] a. The chilled water main supply water temperature is within ±1℃ of the lowest value of all chiller evaporator outlet water temperatures;
[0092] b. The chilled water main return water temperature is within ±1℃ of the lowest value of all chiller evaporator return water temperatures;
[0093] c. At the beginning of the chiller, the chilled water main flow rate is greater than 100 cubic meters / hour.
[0094] After obtaining the fault data corresponding to each target information point, the data fault probability corresponding to each target information point is calculated according to the fault data quantity corresponding to each target information point and the total data quantity. The maximum value among the data fault probabilities corresponding to the four target information points is selected as the final data fault probability of the level. If the final data fault probability is greater than or equal to 20%, it is determined that the quality detection result of the to-be-detected data is unusable. Otherwise, it is determined that the quality detection result of the to-be-detected data is usable. Specifically, when the final data fault probability is less than 5%, it is determined that the quality of the to-be-detected data is highly probable to be usable. When the final data fault probability is greater than or equal to 5% and less than 20%, it is determined that the data quality of the to-be-detected data is low probability to be usable.
[0095] Optionally, the quality of the to-be-detected data is detected according to the target data detection configuration information associated with the target data detection level, including:
[0096] In the case where the target data quality detection level is a sample data screening level, the number of data interruptions in the to-be-detected data is determined according to the data detection configuration information associated with the sample data screening level, and the number of interruptions is differentially filled to obtain the completed to-be-detected data;
[0097] In the completed to-be-detected data, the target data satisfying the chiller start condition is screened;
[0098] The target data is mapped to at least one time interval divided in advance. In the case where the data quantity of the target data associated with the set time interval is greater than a first quantity threshold, it is determined that the quality detection result of the to-be-detected data is data usable. The target data is used as a start model training sample.
[0099] In the optional embodiment, another specific manner of quality detection of the to-be-detected data according to the target data detection configuration information associated with the target data detection level is provided: in the case that the target data quality detection level is the sample data screening level, the missing data in the to-be-detected data is determined according to the data detection configuration information associated with the sample data screening level, and the missing data is differentially filled according to the data adjacent to the missing data, to obtain the completed to-be-detected data. In the completed to-be-detected data, the target data meeting the cold start condition is screened, and the target data is mapped into the plurality of time intervals divided in advance. Further, in the case that the data amount of the target data associated with the set time interval is greater than the first quantity threshold, it is determined that the quality detection result of the to-be-detected data is data available, otherwise, it is determined to be data unavailable. The target data can be used as a training sample of the cold start model.
[0100] In a specific example, the to-be-detected data includes indoor temperature, outdoor wet bulb temperature, chilled water main return temperature and cold station hourly cumulative cooling capacity. The data outside the range of [10, 35] in the indoor temperature data is determined as missing data, and the position of the missing data is differentially filled according to the data adjacent to the missing data, to obtain the completed indoor temperature data. The data outside the range of [10, 35] in the outdoor wet bulb temperature data is determined as missing data, and the position of the missing data is differentially filled according to the data adjacent to the missing data, to obtain the completed outdoor wet bulb temperature data. The data outside the range of [-50, 50] in the chilled water main return temperature is determined as missing data, and the position of the missing data is differentially filled according to the data adjacent to the missing data, to obtain the completed chilled water main return temperature. The data outside the range of [0, 99999] in the cold station hourly cumulative cooling capacity data is determined as missing data, and the position of the missing data is differentially filled according to the data adjacent to the missing data, to obtain the completed cold station hourly cumulative cooling capacity data. It is worth noting that when the data amount of the missing data exceeds the set threshold, the data of the day can be deleted.
[0101] In the completed to be detected data, the data of which the hourly cumulative cooling supply increases sharply (for example, the first difference value is greater than 100) is screened out, and the data is mapped to the pre-divided time interval. Specifically, the time interval is divided into [6, 9), [9, 10), [10, 11), [11, 12). If the number of data mapped to the first three time periods is greater than 30, it is determined that the quality detection result of the to-be-detected data is that the data is available, and the obtained data can be used to establish a cooling supply season model for predicting the start-up data. If the number of sample data mapped to the last three time periods is greater than 30, and the number of data in the first two time periods is greater than 20, it is determined that the quality detection result of the to-be-detected data is that the data is available, and the obtained data can be used to establish a transition season model for predicting the start-up data. If the above two conditions are not met, it is determined that the quality detection result is that the data is not available.
[0102] Optionally, the quality of the to-be-detected data is detected according to the target data detection configuration information associated with the target data detection level, including:
[0103] In the case where the target data quality detection level is a sample data screening level, the number of data breaks in the to-be-detected data is determined according to the data detection configuration information associated with the sample data screening level, and the number of data breaks is filled in difference, to obtain the completed to-be-detected data;
[0104] The data quantity of the completed to-be-detected data is determined.
[0105] In the case where the data quantity is greater than the second data threshold, it is determined that the quality detection result of the to-be-detected data is that the data is available; and the completed to-be-detected data is used as an adjustment model training sample.
[0106] In the embodiment, another specific way of detecting the quality of the to-be-detected data according to the target data detection configuration information associated with the target data detection level is provided: in the case where the target data quality detection level is a sample data screening level, first, the number of data breaks in the to-be-detected data is determined according to the data detection configuration information associated with the sample data screening level, and the number of data breaks is filled in according to the data adjacent to the number of data breaks, to obtain the completed to-be-detected data. Further, the data quantity of the completed to-be-detected data in a set time period (for example, within 5 days) is determined, and in the case where the data quantity is greater than the second data threshold, it is determined that the quality detection result of the to-be-detected data is that the data is available.
[0107] In a specific example, the to-be-detected data includes indoor temperature, outdoor wet-bulb temperature, chilled water main return temperature, and hourly cumulative cooling capacity of the cold station. The to-be-detected data is determined to be abnormal data, and the position of the abnormal data is filled in differentially. The manner of filling in the position of the abnormal data is the same as that of the previous optional embodiment, which will not be described herein again. If the to-be-detected data within 5 days is greater than 480, it can be determined that the quality detection result of the to-be-detected data is that the data is available. These to-be-detected data can be used to establish a regulation model for a cooling season or a transition season.
[0108] Step 220: In the case where the quality detection result of the to-be-detected data is that the data is available, the to-be-detected data is transmitted to the next data detection level for quality detection.
[0109] In the embodiment, in the case where the target data detection level detects that the quality detection result of the to-be-detected data is that the data is available, the to-be-detected data is transmitted to the next data detection level for quality detection, so as to realize hierarchical data quality detection.
[0110] For example, in the information point missing detection level, it is detected that the to-be-detected information does not have the problem of missing information point data. Then, the to-be-detected data can be transmitted to the data integrity detection level for data integrity detection.
[0111] Step 230: In the case where the quality detection result of the to-be-detected data is that the data is unavailable, the detection process is ended, and the quality detection result of the to-be-detected data is displayed.
[0112] In the embodiment, in the case where the target data detection level detects that the quality detection result of the to-be-detected data is that the data is unavailable, the to-be-detected data will not be continuously transmitted to the next data detection level for quality detection, but the detection process is directly ended, and the quality detection result of the to-be-detected data is displayed, so that the monitoring personnel can timely locate the data quality problem, and the human cost required for data quality problem detection is saved.
[0113] For example, in the information point missing detection level, it is detected that one or more information points have the problem of missing data. Then, the detection process is directly ended, the workload of data quality detection is reduced, and the missing information points are displayed, so that the monitoring personnel can timely locate the data quality problem.
[0114] The technical scheme of the embodiment of the present application, when detecting the target data detection level of the to-be-detected data, carries out quality detection on the to-be-detected data according to the target data detection configuration information associated with the target data detection level, in the case that the quality detection result of the to-be-detected data is that the data is available, transmits the to-be-detected data into the next data detection level for quality detection, in the case that the quality detection result of the to-be-detected data is that the data is not available, ends the detection process, and displays the quality detection result of the to-be-detected data. The quality detection is carried out on the to-be-detected data in a hierarchical manner, and in the case that the data quality is not qualified, an exception is directly thrown, and the quality detection of the next level is no longer carried out, thereby improving the data quality detection efficiency.
[0115] Embodiment three
[0116] Figure 3 The flowchart of the data quality detection method in the embodiment two of the present application is provided. The following will be described in combination with the flowchart. Figure 3 The data quality detection method provided in the embodiment three of the present application is described, including the following steps.
[0117] In step 310, the to-be-detected data is input into the information point missing detection level, and it is detected whether there is an information point missing problem. If yes, the detection process is ended, the detection of the next batch of data is restarted, and the information point missing problem is displayed. If no, step 320 is executed.
[0118] In step 320, the to-be-detected data is input into the data integrity detection level, and it is detected whether the data integrity is qualified. If yes, step 330 is executed. If no, the detection process is ended, the detection of the next batch of data is restarted, and the data integrity problem is displayed.
[0119] In step 330, the to-be-detected data is input into the single branch data detection level, and it is detected whether the data of each data branch is qualified. If yes, step 340 is executed. If no, the detection process is ended, and the data branch with unqualified data is displayed.
[0120] In step 340, the to-be-detected data is input into the multi-branch relationship detection level, and it is detected whether the data correlation relationship between the data branches is qualified. If yes, step 350 is executed. If no, the detection process is ended, the detection of the next batch of data is restarted, and the information that the data correlation relationship between the data branches is unqualified is displayed.
[0121] In step 350, the to-be-detected data is input into the card number judgment level, and it is detected whether there is a card number associated with each information point. If yes, the detection process is ended, the detection of the next batch of data is restarted, and the card number problem is displayed. If no, step 360 is executed.
[0122] Step 360, input the to-be-detected data into the cold quantity table data detection level, detect whether the data associated with the system branch is qualified, if yes, execute step 370 or 380, if not, end the detection process, start the detection of the next batch of data, and show the information that the data associated with the system branch is unqualified.
[0123] Step 370, according to the collection time of the to-be-detected data, determine the outage number data in the to-be-detected data, and perform differential filling on the outage number data, obtain target data satisfying the cold start condition in the completed data, and when the data amount of the target data is greater than the first quantity threshold, determine that the quality detection result of the to-be-detected data is data available.
[0124] Step 380, according to the collection time of the to-be-detected data, determine the outage number data in the to-be-detected data, and perform differential filling on the outage number data, and when the data amount of the to-be-detected data after completion is greater than the second quantity threshold, determine that the quality detection result of the to-be-detected data is data available.
[0125] It can be understood that step 370 and step 380 are executed according to actual scenes. In the scene of screening the training sample of the start model, step 370 is executed, and in the scene of screening the training sample of the adjustment model, step 380 is executed.
[0126] The technical scheme of the embodiment of the application performs hierarchical quality detection on to-be-detected data, only when the previous level detection is qualified, the to-be-detected data is transmitted to the next level for quality detection, which can reduce the workload of data quality detection and save the labor cost of data quality detection.
[0127] Embodiment four
[0128] Figure 4 A structure schematic diagram of a data quality detection device provided by the fourth embodiment of the application, the data quality detection device comprises a data quality detection module 410 and a data transmission module 420.
[0129] The data quality detection module 410 is configured to, when detecting that to-be-detected data is input into a target data detection level, perform quality detection on the to-be-detected data according to target data detection configuration information associated with the target data detection level; the to-be-detected data is collected by at least two information points in a cold station system.
[0130] The data transmission module 420 is configured to, according to the quality detection result of the to-be-detected data, transmit the to-be-detected data to a next data detection level for quality detection.
[0131] The technical scheme of the embodiment of the present application detects the target data detection level of the to-be-detected data, performs quality detection on the to-be-detected data according to the target data detection configuration information associated with the target data detection level, and then transmits the to-be-detected data to the next data detection level for quality detection according to the quality detection result of the to-be-detected data, so as to perform quality detection on the to-be-detected data in a hierarchical manner. Only when the detection result of the previous level meets the set condition, the to-be-detected data will enter the next level for detection, thereby improving the data quality detection efficiency and reducing the workload of data quality detection.
[0132] Optionally, the data transmission module 420 comprises:
[0133] The data transmission unit is configured to transmit the to-be-detected data to the next data detection level for quality detection when the quality detection result of the to-be-detected data is that the data is available.
[0134] The detection result display unit is configured to end the detection process and display the quality detection result of the to-be-detected data when the quality detection result of the to-be-detected data is that the data is unavailable.
[0135] Optionally, the target data detection level is at least one of an information point missing detection level, a data completeness detection level, a single branch data detection level, a multi-branch relationship detection level, a card number judgment level, a cold quantity table data detection level, and a sample data screening level.
[0136] Optionally, the data quality detection module 410 comprises:
[0137] The information point reading unit is configured to read at least two information points contained by the cold station system from a database when the target data detection level is the information point missing detection level.
[0138] The associated data acquisition unit is configured to determine the data associated with each information point in the to-be-detected data.
[0139] The first detection result determination unit is configured to determine that the quality detection result of the to-be-detected data is that the data is unavailable when the amount of data associated with at least one information point is 0.
[0140] Optionally, the data quality detection module 410 comprises:
[0141] The complete data proportion calculation unit is configured to calculate the complete data proportion according to the collection time of the to-be-detected data when the target data quality detection level is the data completeness detection level.
[0142] The second detection result determination unit is configured to determine that the quality detection result of the to-be-detected data is data available when the span of the collection time is greater than a first time threshold and the complete data proportion is greater than a set proportion threshold.
[0143] Optionally, the data quality detection module 410 comprises:
[0144] The first fault data determination unit is configured to determine fault data in the to-be-detected data according to data detection configuration information associated with the single-branch data detection level when the target data quality detection level is the single-branch data detection level.
[0145] The first information point fault probability calculation unit is configured to determine the total data amount corresponding to an information point and the fault data amount corresponding to the information point, and calculate the data fault probability of the information point according to the total data amount and the fault data amount.
[0146] The branch fault probability determination unit is configured to determine the data fault probability of a data branch to which each information point belongs according to the data fault probability of each information point; the data branch comprises at least one information point.
[0147] The third detection result determination unit is configured to determine that the quality detection result of the to-be-detected data is data unavailable when the data fault probability of at least one data branch is greater than or equal to a set probability threshold.
[0148] Optionally, the data quality detection module 410 comprises:
[0149] The data comparison unit is configured to compare data associated with information points belonging to a first data branch in the to-be-detected data with data associated with information points belonging to a second data branch when the target data quality detection level is the multi-branch relationship detection level; and / or compare data associated with different information points belonging to the second data branch in the to-be-detected data.
[0150] The fault data amount determination unit is configured to determine the fault data amount associated with information points of compared data according to a comparison result.
[0151] The second information point fault probability calculation unit is configured to calculate the data fault probability of information points of compared data according to the fault data amount.
[0152] The fourth detection result determination unit is configured to determine that the quality detection result of the to-be-detected data is data unavailable when the data fault probability of at least one information point is greater than or equal to a set probability threshold.
[0153] Optionally, the data quality detection module 410 comprises:
[0154] a duration time determination unit, configured to determine a maximum duration time during which data associated with at least one information point remains unchanged in the to-be-detected data, in a case where the target data quality detection level is a card number judgment level;
[0155] a fifth detection result determination unit, configured to determine that a quality detection result of the to-be-detected data is data unavailable, in a case where the maximum duration time exceeds a second time threshold.
[0156] Optionally, the data quality detection module 410 comprises:
[0157] a second fault data determination unit, configured to determine at least one target information point associated with a system branch, and determine fault data in data associated with the target information point according to data detection configuration information associated with a cold metering data detection level, in a case where the target data quality detection level is the cold metering data detection level.
[0158] a third information point fault probability calculation unit, configured to calculate a data fault probability of each target information point according to a fault data amount corresponding to the target information point and a total data amount.
[0159] a sixth detection result determination unit, configured to determine that the quality detection result of the to-be-detected data is data unavailable, in a case where the data fault probability of at least one target information point is greater than or equal to a set probability threshold.
[0160] Optionally, the data quality detection module 410 comprises:
[0161] a first data complement unit, configured to determine a discontinuity data in the to-be-detected data according to a collection time of the to-be-detected data, and perform difference filling on the discontinuity data to obtain complemented to-be-detected data, in a case where the target data quality detection level is a sample data screening level.
[0162] a target data screening unit, configured to screen target data satisfying a cold start condition from the complemented to-be-detected data.
[0163] a seventh detection result determination unit, configured to map the target data to at least one time interval divided in advance, and determine that a quality detection result of the to-be-detected data is data available in a case where a data amount of target data associated with a set time interval is greater than a first quantity threshold; the target data is used as a cold start model training sample.
[0164] Optionally, the data quality detection module 410 comprises:
[0165] The second data completion unit is configured to, when the target data quality detection level is a sample data screening level, determine the number of discontinuous data in the to-be-detected data according to the collection time of the to-be-detected data, and perform differential filling on the number of discontinuous data to obtain completed to-be-detected data.
[0166] The data amount determination unit is configured to determine the data amount of the completed to-be-detected data.
[0167] The eighth detection result determination unit is configured to, when the data amount is greater than the second data threshold, determine that the quality detection result of the to-be-detected data is data available.
[0168] The data quality detection device provided by the embodiment of the present application can execute the data quality detection method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.
[0169] Embodiment five
[0170] Figure 5 A structural schematic diagram of an electronic device provided by embodiment five of the present application is shown in FIG. 5. Figure 5 As shown in FIG. 5, the electronic device includes a processor 50, a memory 51, an input device 52, and an output device 53; the number of processors 50 in the device can be one or more, Figure 5 and the processor 50 in the device is taken as an example; the processor 50, the memory 51, the input device 52, and the output device 53 in the device can be connected through a bus or other means, Figure 5 and the connection through the bus is taken as an example.
[0171] The memory 51 is a kind of computer readable storage medium, which can be used to store software programs, computer executable programs, and modules, such as program instructions / modules (for example, the data quality detection module 410 and the data transmission module 420 in the data quality detection device) corresponding to the data quality detection method in the embodiment of the present application. The processor 50 executes the software programs, instructions, and modules stored in the memory 51, thereby performing various functional applications and data processing of the device, that is, implementing the data quality detection method described above, including:
[0172] When detecting that to-be-detected data is input into a target data detection level, performing quality detection on the to-be-detected data according to target data detection configuration information associated with the target data detection level; the to-be-detected data is collected from at least two information points in a cold station system;
[0173] According to the quality detection result of the to-be-detected data, transmitting the to-be-detected data to a next data detection level for quality detection.
[0174] The memory 51 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; and the data storage area can store data created according to the use of the terminal and the like. In addition, the memory 51 can include a high-speed random access memory, and can also include a non-volatile memory such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state memory device. In some examples, the memory 51 can further include a memory disposed remotely with respect to the processor 50, which can be connected to the device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.
[0175] Embodiment six
[0176] The embodiment six of the present application also provides a computer readable storage medium having a computer program stored thereon, which, when executed by a computer processor, is used to execute a data quality detection method, the method comprising:
[0177] When a target data detection level to be detected is detected, quality detection is performed on the to-be-detected data according to target data detection configuration information associated with the target data detection level; the to-be-detected data is obtained by at least two information points in a cold station system.
[0178] According to the quality detection result of the to-be-detected data, the to-be-detected data is transmitted to a next data detection level for quality detection.
[0179] Of course, the storage medium provided by the embodiment of the present application and containing computer executable instructions is not limited to the method operations as described above, and can also perform related operations in the data quality detection method provided by any embodiment of the present application.
[0180] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary universal hardware, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH memory, a hard disk, or an optical disc, and includes a number of instructions for causing a computer device (which can be a personal computer, an application server, or a network device, etc.) to execute the methods described in the various embodiments of the present application.
[0181] It is worth noting that the above-mentioned embodiment of the data quality detection device comprises various units and modules only according to the logical division of functions, but is not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific name of each functional unit is only for the convenience of mutual differentiation, and does not limit the protection scope of the present application.
[0182] It is noted that the above are only preferred embodiments of the present application and the technical principles applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and those skilled in the art can make various obvious changes, readjustments and substitutions without departing from the scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.
Claims
1. A data quality inspection method, characterized in that, include: When the target data detection level is detected, the data to be detected is subjected to quality detection based on the target data detection configuration information associated with the target data detection level; the data to be detected is collected from at least two information points in the cooling station system; the target data detection level is at least one of the following: information point missing detection level, data integrity detection level, single branch data detection level, multi-branch relationship detection level, card number judgment level, cooling volume meter data detection level, and sample data screening level; Based on the quality inspection results of the data to be inspected, the data to be inspected is passed to the next data inspection level for quality inspection; Based on the quality inspection results of the data to be inspected, the data to be inspected is passed to the next data inspection level for quality inspection, including: If the quality inspection result of the data to be inspected indicates that the data is usable, the data to be inspected is passed to the next data inspection level for quality inspection; the data inspection levels are arranged according to a pre-set data quality inspection order. If the quality inspection result of the data to be inspected is that the data is unavailable, the inspection process ends and the quality inspection result of the data to be inspected is displayed. Based on the target data detection configuration information associated with the target data detection level, quality detection is performed on the data to be detected, including: When the target data quality detection level is the single-branch data detection level, fault data is determined from the data to be detected based on the data detection configuration information associated with the single-branch data detection level. Determine the total amount of data corresponding to the information point and the amount of fault data corresponding to the information point, and calculate the data fault probability of the information point based on the total amount of data and the amount of fault data. Based on the data failure probability of each information point, the data failure probability of the data branch to which each information point belongs is determined; the data branch includes at least one information point. If the probability of data failure in at least one data branch is greater than or equal to a set probability threshold, the quality detection result of the data to be detected is determined to be unusable. Based on the target data detection configuration information associated with the target data detection level, quality detection is performed on the data to be detected, including: When the target data quality detection level is the multi-branch relationship detection level, the data associated with the information points belonging to the first data branch in the data to be detected is compared with the data associated with the information points belonging to the second data branch; and / or, the data associated with different information points belonging to the second data branch in the data to be detected is compared. Based on the comparison results, determine the amount of fault data associated with the information point to which the data being compared belongs; Based on the amount of faulty data, calculate the probability of data failure of the information point to which the data being compared belongs; If the probability of data failure at at least one information point is greater than or equal to a set probability threshold, the quality inspection result of the data to be tested is determined to be unusable.
2. The method according to claim 1, characterized in that, Based on the target data detection configuration information associated with the target data detection level, quality detection is performed on the data to be detected, including: When the target data detection level is the missing information point detection level, at least two information points contained in the cold station system are read from the database; Determine the data associated with each information point from the data to be detected; If the amount of data associated with at least one information point is 0, the quality inspection result of the data to be detected is determined to be unusable.
3. The method according to claim 1, characterized in that, Based on the target data detection configuration information associated with the target data detection level, quality detection is performed on the data to be detected, including: When the target data quality detection level is the data integrity detection level, the percentage of complete data is calculated based on the collection time of the data to be detected. If the span of the collection time is greater than a first time threshold and the proportion of complete data is greater than a set proportion threshold, the quality detection result of the data to be detected is determined to be usable.
4. The method according to claim 1, characterized in that, Based on the target data detection configuration information associated with the target data detection level, quality detection is performed on the data to be detected, including: When the target data quality detection level is the card number judgment level, determine the maximum duration during which the data associated with at least one information point in the data to be detected remains unchanged. If the maximum duration exceeds the second time threshold, the quality detection result of the data to be detected is determined to be unusable.
5. The method according to claim 1, characterized in that, Based on the target data detection configuration information associated with the target data detection level, quality detection is performed on the data to be detected, including: When the target data quality detection level is the cold meter data detection level, at least one target information point associated with the system branch is determined, and fault data is determined from the data associated with the target information point based on the data detection configuration information associated with the cold meter data detection level. Calculate the data failure probability of each target information point based on the amount of fault data corresponding to the target information point and the total amount of data. If the probability of data failure at at least one target information point is greater than or equal to a set probability threshold, the quality inspection result of the data to be detected is determined to be unusable.
6. The method according to claim 1, characterized in that, Based on the target data detection configuration information associated with the target data detection level, quality detection is performed on the data to be detected, including: When the target data quality detection level is the sample data screening level, the data detection configuration information associated with the sample data screening level is used to determine the number of interrupted data in the data to be detected, and the interrupted data is differentially filled to obtain the completed data to be detected. From the completed data to be tested, target data that meets the cold start conditions are selected; The target data is mapped to at least one pre-divided time interval. If the amount of target data associated with a set time interval is greater than a first quantity threshold, the quality detection result of the data to be detected is determined to be usable. The target data is used as training samples for the startup model.
7. The method according to claim 1, characterized in that, Based on the target data detection configuration information associated with the target data detection level, quality detection is performed on the data to be detected, including: When the target data quality detection level is the sample data screening level, the number of interrupted data to be detected is determined based on the data detection configuration information associated with the sample data screening level, and the interrupted data is differentially filled to obtain the completed data to be detected. Determine the amount of data in the completed data to be detected; If the amount of data is greater than the second data threshold, the quality detection result of the data to be detected is determined to be usable; the completed data to be detected is used as a training sample for the adjustment model.
8. A data quality detection device, characterized in that, include: The data quality detection module is used to perform quality detection on the data to be detected based on the target data detection configuration information associated with the target data detection level when the data to be detected is input into the target data detection level. The data to be detected is collected from at least two information points in the cooling station system. The target data detection level is at least one of the following: information point missing detection level, data integrity detection level, single branch data detection level, multi-branch relationship detection level, card number judgment level, cooling volume meter data detection level, and sample data screening level. The data input module is used to input the data to be tested into the next data detection level for quality detection based on the quality detection result of the data to be tested; The data input module includes: The data input unit is used to input the data to be tested into the next data detection level for quality detection when the quality detection result of the data to be tested is that the data is usable; the data detection levels are arranged according to a pre-set data quality detection order. The test result display unit is used to end the test process and display the quality test result of the data to be tested when the quality test result of the data to be tested is that the data is unavailable. The data quality inspection module includes: The first fault data determination unit is used to determine fault data in the data to be detected based on the data detection configuration information associated with the single branch data detection level when the target data quality detection level is the single branch data detection level. The first information point failure probability calculation unit is used to determine the total amount of data corresponding to the information point and the amount of faulty data corresponding to the information point, and to calculate the data failure probability of the information point based on the total amount of data and the amount of faulty data. A branch fault probability determination unit is used to determine the data fault probability of the data branch to which each information point belongs based on the data fault probability of each information point; the data branch includes at least one information point. The third detection result determination unit is used to determine the quality detection result of the data to be detected as unusable when the data failure probability of at least one data branch is greater than or equal to a set probability threshold. The data quality inspection module includes: The data comparison unit is used to compare the data associated with the information points belonging to the first data branch in the data to be detected with the data associated with the information points belonging to the second data branch when the target data quality detection level is the multi-branch relationship detection level; and / or, to compare the data associated with different information points belonging to the second data branch in the data to be detected. The fault data volume determination unit is used to determine the amount of fault data associated with the information point to which the data being compared belongs, based on the comparison results. The second information point failure probability calculation unit is used to calculate the data failure probability of the information point to which the data being compared belongs based on the amount of failure data. The fourth detection result determination unit is used to determine that the quality detection result of the data to be detected is unusable when the data failure probability of at least one information point is greater than or equal to a set probability threshold.
9. An electronic device, characterized in that, The device includes: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data quality detection method as described in any one of claims 1-7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the data quality detection method as described in any one of claims 1-7.
Citation Information
Patent Citations
A method for detecting data quality and a device for detecting data quality
CN109491990A
Multi-dimensional transportation data fusion and data quality detection method
CN113742330A