A sewage monitoring data quality control method, device and equipment

Abstract

The present invention provides a method, device and equipment for quality control of sewage monitoring data, which relates to the field of computer information processing technology. The method comprises: obtaining sewage monitoring data; preprocessing the sewage monitoring data to obtain a data set; matching target data quality control rule conditions in a quality control rule library according to the data set; performing quality control inspection on the sewage monitoring data according to target quality control rules matching the target data quality control rule conditions, obtaining inspection results, and outputting them. The scheme of the present invention can realize fast and accurate rule matching and reasoning for sewage monitoring, and improve the effectiveness and accuracy of sewage monitoring data quality control.

Description

Technical Field

[0001] The present invention relates to the technical field of computer information processing, and in particular to a method, device and equipment for quality control of sewage monitoring data. Background Art

[0002] In the water conservancy and water services business, the data information contained in sewage monitoring data is extremely rich, so the quality control rules are also numerous and complex. Conventional quality rule verification technology may not cover all business scenarios and data types, resulting in incomplete or inaccurate sewage monitoring data verification results. For a large amount of real-time sewage monitoring data, conventional quality rule verification technology may not be able to process and feedback verification results in a timely manner, affecting the efficiency of business processing. Summary of the invention

[0003] The present invention provides a sewage monitoring data quality control method, device and equipment to solve the problem of low effectiveness and accuracy of sewage monitoring data quality control in the prior art.

[0004] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0005] A sewage monitoring data quality control method, comprising:

[0006] Obtaining sewage monitoring data;

[0007] Preprocessing the sewage monitoring data to obtain a data set;

[0008] According to the data set, matching target data quality control rule conditions in a quality control rule library;

[0009] According to the target quality control rules that match the target data quality control rule conditions, the sewage monitoring data is subjected to quality control inspection, and the inspection results are obtained and output.

[0010] Optionally, the sewage monitoring data is preprocessed to obtain a data set, including:

[0011] Performing data cleaning on the sewage monitoring data, processing missing values, correcting erroneous data and eliminating duplicate records;

[0012] Integrate the cleaned data into a unified preset format;

[0013] The data set is obtained by converting the data in a unified preset format according to preset data rules.

[0014] Optionally, matching target data quality control rule conditions in a quality control rule library according to the data set includes:

[0015] The attributes and results of the data set are matched with the nodes of the network structure of the quality control rule base, and the nodes of the network structure include a first node and a second node.

[0016] Optionally, matching the attributes and results of the data set with nodes of the network structure of the quality control rule base includes:

[0017] Mapping the attributes and results of the data set to the nodes of the network structure, wherein the first node includes one quality control rule condition and the second node includes multiple quality control rule conditions;

[0018] After the data set is matched with the first node, it is then matched with the second node.

[0019] Optionally, according to a target quality control rule that matches the target data quality control rule, a quality control test is performed on the sewage monitoring data to obtain a test result and output it, including:

[0020] After the data set is matched with the first node, it is matched with the second node to obtain target quality control data;

[0021] According to the target quality control data, determine whether the sewage monitoring data meets the target quality control rules, obtain the inspection results, and output them.

[0022] Optionally, judging whether the sewage monitoring data meets the target quality control rules according to the target quality control rules, obtaining the test results, and outputting them, including:

[0023] If the sewage monitoring data meets the target quality control rules, the sewage monitoring data passes the quality control, and the sewage monitoring data that passes the quality control is output;

[0024] If the sewage monitoring data does not meet the target quality control rules, it will not pass the quality control, and the data items and reasons that do not meet the target quality control rules will be output.

[0025] Optionally, the sewage monitoring data quality control method further includes:

[0026] When the sewage monitoring data does not meet the target quality control rules, the quality control rule library is updated.

[0027] The present invention also provides a sewage monitoring data quality control device, which is characterized by comprising:

[0028] An acquisition module, used to acquire sewage monitoring data;

[0029] The processing module is used to pre-process the sewage monitoring data to obtain a data set; match the target data quality control rule conditions in the quality control rule library according to the data set; perform quality control inspection on the sewage monitoring data according to the target quality control rules that match the target data quality control rule conditions, obtain the inspection results, and output them.

[0030] The present invention also provides a computing device, comprising: a processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method described above is executed.

[0031] The present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the method described above.

[0032] The above solution of the present invention includes at least the following beneficial effects:

[0033] The above scheme of the present invention obtains sewage monitoring data; pre-processes the sewage monitoring data to obtain a data set; matches the target data quality control rule conditions in the quality control rule library according to the data set; performs quality control inspection on the sewage monitoring data according to the target quality control rules that match the target data quality control rule conditions, obtains the inspection results, and outputs them. The scheme of the present invention can integrate the business data of sewage monitoring into the process, perform automated process management, realize fast and accurate rule matching and reasoning, and improve the effectiveness and accuracy of sewage monitoring data quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flow chart of a sewage monitoring data quality control method provided by an embodiment of the present invention;

[0035] Figure 2 It is a working logic diagram of the sewage monitoring data quality control method provided by an embodiment of the present invention;

[0036] Figure 3 is a schematic diagram of the network structure of a rule engine provided by an embodiment of the present invention;

[0037] Figure 4 It is a module schematic diagram of a sewage monitoring data quality control device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0039] like Figure 1 As shown, an embodiment of the present invention provides a sewage monitoring data quality control method, comprising:

[0040] Step 11, obtaining sewage monitoring data;

[0041] Step 12, preprocessing the sewage monitoring data to obtain a data set;

[0042] Step 13, matching the target data quality control rule conditions in the quality control rule library according to the data set;

[0043] Step 14: Perform quality control inspection on the sewage monitoring data according to the target quality control rules that match the target data quality control rule conditions, obtain the inspection results, and output them.

[0044] In this embodiment, the sewage monitoring data involves multiple aspects, including physical indicators, chemical indicators, biological indicators, and radioactive indicators, among which the physical indicators mainly include odor, temperature, turbidity, transparency, color, etc. These data can be obtained by direct observation or instrument measurement.

[0045] Chemical indicators are an important part of sewage monitoring, including non-specific indicators (such as conductivity, pH value, hardness, alkalinity, inorganic acidity, etc.), inorganic indicators (such as toxic metals, nitrates, nitrites, phosphates, etc.), non-specific organic indicators (such as total oxygen demand, chemical oxygen demand, biochemical oxygen demand, total organic carbon, potassium permanganate index, phenols, etc.), dissolved gases (such as oxygen, carbon dioxide, etc.), ammonia nitrogen, residual chlorine, conductivity, etc.

[0046] Radioactive indicators include total alpha rays, total beta rays, uranium, radium, thorium, etc. These indicators are mainly used to assess the radioactive contamination in sewage, which poses a potential threat to the environment and human health.

[0047] The acquired sewage monitoring data is converted into a data set in a unified preset format. The data set is matched with the rule base as a data quality control target. When the data set meets the quality control rules, it is determined that the current data meets the quality control and meets the business requirements, and the sewage monitoring data of the data set is output and stored.

[0048] This embodiment obtains sewage monitoring data, pre-processes the sewage monitoring data, obtains a data set, matches the target data quality control rule conditions in the quality control rule library according to the data set, performs quality control inspection on the sewage monitoring data according to the target quality control rule matching the target data quality control rule conditions, obtains the inspection results, and outputs them. The business data of sewage monitoring can be integrated into the process, and automated process management can be performed to achieve fast and accurate rule matching and reasoning, thereby improving the effectiveness and accuracy of sewage monitoring data quality control processing.

[0049] In an optional embodiment of the present invention, step 12 may include:

[0050] Step 121, performing data cleaning on the sewage monitoring data, processing missing values, correcting erroneous data and eliminating duplicate records;

[0051] Step 122, integrating the cleaned data into a unified preset format;

[0052] Step 123, converting the data in a unified preset format according to preset data rules to obtain the data set.

[0053] In this embodiment, after obtaining the sewage monitoring data, the sewage monitoring data needs to be preprocessed before subsequent data quality control can be performed. Before applying the quality control rules, it is important to preprocess the data, including data cleaning (such as removing outliers and filling missing values), data conversion (such as date formatting and data type conversion), and data standardization (such as unifying units).

[0054] In the preprocessing of this embodiment, it is necessary to determine the integrity of the data and check whether the data is missing key fields. Determine the accuracy of the data: Check whether the data is within a reasonable range, such as water consumption cannot be a negative number. Determine the consistency of the data: Check whether there are logical contradictions between the data, such as whether the date format is consistent, and whether there is a reasonable association between two related fields. Determine the timeliness of the data: Check whether the data is up to date or within the specified time range.

[0055] Specifically, data cleaning is first performed, including processing missing values, correcting erroneous data, and eliminating duplicate records. Next, data integration is performed to unify the data format of multiple data sources and integrate them. Then, data transformation, such as standardization and normalization, is performed to make the data comparable. Data reduction is also required to reduce the data dimension through feature selection or extraction. Finally, the preprocessed data is verified and evaluated to ensure data quality and lay a solid foundation for subsequent analysis and modeling.

[0056] In an optional embodiment of the present invention, step 13 may include:

[0057] Step 131, matching the attributes and results of the data set with the nodes of the network structure of the quality control rule base, wherein the nodes of the network structure include a first node and a second node.

[0058] In this embodiment, a quality control rule library is established based on the drainage household portrait table, and the drainage household portrait table is shown in Table 1 below:

[0059] Table 1, Drainage household portrait table:

[0060] Serial number Drainage household type Monitoring parameters 1 Catering Mud level, water level, ammonia nitrogen 2 Construction Site Suspended Matter 3 Government agencies Mud level, water level, ammonia nitrogen 4 Residents Water Level 5 Auto Repair Machine Washing Suspended matter, oil in water 6 Washing PH, conductivity 7 Garbage Collection Mud level, ammonia nitrogen 8 Industrial Water level, PH, conductivity, ammonia nitrogen 9 Farmers Market Mud level, ammonia nitrogen 10 Livestock and poultry farming Mud level, ammonia nitrogen 11 Comprehensive business Mud level, water level 12 Medical Residual Chlorine

[0061] Each type of drainage household has corresponding monitoring parameters that need to be monitored. Each type of drainage household is used as a rule file to ensure that each rule file is configured with three parts at the same time, namely attributes, conditions and results;

[0062] The properties define the execution attribute information of the current rule, such as the priority of the rule in the rule base and during execution, the effective date and expiration date of the rule, etc. The condition is the triggering condition when executing the rule. The result indicates the action triggered when the current rule meets the condition.

[0063] Each rule file is also configured with a clear name and description. The drainage user type can be used as the name and the monitoring parameter as the description, such as the name: catering, and the description: mud level, water level, ammonia nitrogen.

[0064] Deploy all configured rule files to the network structure of the rule engine to obtain a quality control rule library, where the attributes in the data set correspond to the names of the rule files and the results correspond to the descriptions of the rule files. The rule files are converted into one or more nodes in the network structure in the rule engine. The nodes include rule condition information that matches the data set, such as naming and description.

[0065] Then, the data set is matched with the target data quality control rule conditions in the quality control rule library through the nodes of the network structure.

[0066] The network structure is a network composed of a starting node, a first node, a second node, and a terminal node, wherein the starting node is used to receive all data entering the system and pass it to the subsequent first node and second node, and the first node and the second node are used to match the quality control rule conditions on the data. When all the rule conditions are matched successfully, the data is input to the terminal node.

[0067] In an optional embodiment of the present invention, step 131 may include:

[0068] Step 1311, mapping the attributes and results of the data set to the nodes of the network structure, the first node includes a quality control rule condition, and the second node includes multiple quality control rule conditions; when the data set matches the first node, it is matched with the second node.

[0069] In this embodiment, Figure 3 As shown, the first node is responsible for matching a single quality control rule condition, and the second node is responsible for combined matching between multiple quality control rule conditions.

[0070] When a piece of data is input into the system, its attributes and results will be passed to the first node network of the network structure to match the quality control rule conditions of the first node. If the match is successful, the data will be stored in the first node network and continue to propagate to the second node network of the network structure.

[0071] The second node network finds all data combinations that meet the entire rule condition by combining the single condition matching results from the first node network and the combined condition matching results from other second nodes.

[0072] In a specific embodiment, the matching relationship between the data set and the network structure is expressed by the following formula:

[0073] [text{matching relationship} = \text{Rete network structure} \times \text{dataset} ];

[0074] Among them, "Rete network structure" represents the network composed of the first node and the second node, and "data set" represents the data set in the business. This formula represents the process of matching the fact set through the network structure to obtain the matching relationship.

[0075] Create rule file parsing and nodes:

[0076] First, parse the configured rule file into a format recognizable by the rule engine. Each rule file contains a name (such as a rule ID), a description (specific instructions for the rule), and a rule condition (i.e., the conditions that must be met to trigger the rule). In the RETE network, each rule condition can be converted into one or more nodes. These nodes represent the conditional part of the rule, for example, the value of a data field must be greater than a certain threshold.

[0077] Set the node properties:

[0078] Each node in the RETE network has a unique identifier (such as a node ID) and stores all information related to the rule conditions, including naming (which may come from the naming of the rule file), description (description of the rule file or specific description of the condition), and conditional logic (such as comparison operators, values, etc.). Nodes may also contain links to other nodes (i.e., edges in the network), which represent the logical relationship between rule conditions (such as AND, OR).

[0079] Dataset Mapping:

[0080] A data set usually contains multiple data records, each record contains multiple attributes (fields) and corresponding results (or values). During the mapping process, each attribute or result of the data set is considered a "fact" and these facts are input into the RETE network. The rule engine traverses the nodes in the network and matches each fact with the conditions in the node. If the fact meets the conditions of the node, the node is activated and may trigger further rule evaluation or execute corresponding actions.

[0081] Matching and result processing:

[0082] When facts in a data set match nodes in a RETE network, the rule engine performs corresponding actions based on the logical relationships and conditions of the nodes. If all conditions of a rule are met (i.e., all relevant nodes of the rule are activated), the rule is considered triggered and subsequent actions defined in the rule can be performed, such as data verification, data conversion, or report generation.

[0083] Optimization and performance considerations:

[0084] The RETE network optimizes performance through its efficient node sharing and incremental update mechanism. When new facts are added to the network, only the affected nodes are re-evaluated, not the entire network. The rule engine may also adopt other optimization techniques such as rule sorting, indexing, etc. to further improve matching efficiency.

[0085] Through the above steps, the attributes and results of the data set are effectively mapped to the nodes of the RETE network structure, and the matching with the target data quality control rule conditions in the quality control rule library is achieved through the rule engine.

[0086] In matching, the temporal redundancy and structural similarity between rules are also utilized to share the same first node. This means that if multiple rules contain the same condition, they will share a first node, which can improve matching efficiency.

[0087] When a piece of data matches multiple rules, a conflict occurs. To resolve the conflict, a priority value is set for each rule. In case of a conflict, the rule with a higher priority value will be executed first. If the priority values ​​are the same, the rules are executed in the order in which they are in the rule set.

[0088] In an optional embodiment of the present invention, step 14 includes:

[0089] Step 141, after the data set is matched with the first node, it is matched with the second node to obtain target quality control data;

[0090] Step 142, based on the target quality control data, determine whether the sewage monitoring data meets the target quality control rules, obtain the test results, and output them.

[0091] In this embodiment, when the data in the data set is input into the system, its attributes and results will be transmitted to the first node network of the network structure to match the quality control rule conditions of the first node. If the match is successful, the node is activated, the data will be stored in the first node network, and according to the logical relationship between the nodes (AND, OR, etc.), it will continue to propagate to the second node network of the network structure.

[0092] The second node network combines the single condition matching results from the first node network and the combined condition matching results from other second nodes to find all data combinations that meet the entire rule conditions. When all conditions of a rule are met (that is, all relevant nodes of the rule are activated), the rule is considered triggered and the target quality control data is obtained.

[0093] The target quality control data includes information on sewage monitoring data. The target quality control data is compared and verified with the target quality control rules. For example, if the target quality control data is catering sewage detection data, and its data information includes monitoring data of mud level, water level, and ammonia nitrogen, then the data is considered to comply with the quality control rules.

[0094] This embodiment avoids unnecessary traversal by being able to quickly find the corresponding successor node; data will continue to propagate downward along the network only when it meets the requirements of the current node, reducing invalid matching; and a large amount of repeated calculations is avoided by saving matching status and sharing nodes.

[0095] In an optional embodiment of the present invention, step 142 may include:

[0096] Step 1421, if the sewage monitoring data meets the target quality control rule, the sewage monitoring data passes the quality control, and the sewage monitoring data that passes the quality control is output;

[0097] Step 1422: If the sewage monitoring data does not meet the target quality control rules, it will not pass the quality control, and the data items and reasons that do not meet the target quality control rules will be output.

[0098] In this embodiment, if all data meet the quality control rules, the system should generate a report or message that it has passed the quality control, and may continue with the subsequent data processing or analysis process;

[0099] If the data does not comply with the rules, the system generates a detailed report listing all the data points that violate the rules and their specific quality control rules. The report analyzes the illegal data to determine the cause of the violation, which may include data entry errors, data transmission errors, data processing logic errors, etc., so that the problem can be identified and appropriate corrective measures can be taken.

[0100] In an optional embodiment of the present invention, the sewage monitoring data quality control method further includes:

[0101] Step 15: When the sewage monitoring data does not meet the target quality control rules, the quality control rule library is updated.

[0102] In this embodiment, when the data does not match the quality control rules, the system updates the database.

[0103] First, accurately identify which data items do not match the quality control rules. This can be determined through the output of the rule engine, which details all data items that do not meet the rules and the reasons why.

[0104] Based on the unmatched data items and quality control rules, determine the update strategy. The update strategy may include:

[0105] Direct Correction: If the error is obvious and the correct value can be determined directly, then correct the data directly in the database.

[0106] Mark as Error: In some cases, the data cannot be corrected directly, but a flag or status field can be set in the database to indicate that there is a problem with these data items.

[0107] Delete: If data items are invalid or cannot be repaired, delete these data items from the database.

[0108] Keep original data and add comments: To preserve data integrity, you may need to keep the original data and add comments or logs in the database to record the reasons for the mismatch and possible corrective actions.

[0109] Based on the determined update strategy, write a database update script or program that can connect to the database, perform the necessary update operations, and record any changes.

[0110] Execute the update script or program in the test to ensure that it works as expected and does not corrupt other data in the database. Once the test passes, the update can be executed in the business environment.

[0111] After the update is complete, rerun the quality control process on the data to verify that the problem has been resolved. Ensure that all updated data meets the quality control rules.

[0112] Record the history of all update operations, including the time, content, and reason of the update, to facilitate future data auditing and error tracking.

[0113] The sewage monitoring data quality control method described in the above embodiment of the present invention can give full play to its advantages in various application scenarios, perform complex pattern matching and conditional judgment, support multiple data types, integrate the business data of sewage monitoring into the process, perform automated process management, and achieve fast and accurate rule matching and reasoning, thereby improving the effectiveness and accuracy of sewage monitoring data processing.

[0114] It is possible to separate big data quality control rules from business systems and achieve a high degree of decoupling, which helps improve the maintainability and scalability of applications. When updating big data quality control rules, there is no need to update the entire application, only the quality control rule library needs to be updated. If big data quality control rules need to be maintained and modified in the later stage, it can save manpower and material resources and improve the maintainability and reusability of complex codes.

[0115] The embodiment of the present invention further provides a sewage monitoring data quality control device 40, comprising:

[0116] An acquisition module 41 is used to acquire sewage monitoring data;

[0117] The processing module 42 is used to pre-process the sewage monitoring data to obtain a data set; match the target data quality control rule conditions in the quality control rule library according to the data set; perform quality control inspection on the sewage monitoring data according to the target quality control rules that match the target data quality control rule conditions, obtain the inspection results, and output them.

[0118] Optionally, the sewage monitoring data is preprocessed to obtain a data set, including:

[0119] Performing data cleaning on the sewage monitoring data, processing missing values, correcting erroneous data and eliminating duplicate records;

[0120] Integrate the cleaned data into a unified preset format;

[0121] The data set is obtained by converting the data in a unified preset format according to preset data rules.

[0122] Optionally, matching target data quality control rule conditions in a quality control rule library according to the data set includes:

[0123] The attributes and results of the data set are matched with the nodes of the network structure of the quality control rule base, and the nodes of the network structure include a first node and a second node.

[0124] Optionally, matching the attributes and results of the data set with nodes of the network structure of the quality control rule base includes:

[0125] Mapping the attributes and results of the data set to the nodes of the network structure, wherein the first node includes one quality control rule condition and the second node includes multiple quality control rule conditions;

[0126] After the data set is matched with the first node, it is then matched with the second node.

[0127] Optionally, according to a target quality control rule that matches the target data quality control rule, a quality control test is performed on the sewage monitoring data to obtain a test result and output it, including:

[0128] After the data set is matched with the first node, it is matched with the second node to obtain target quality control data;

[0129] According to the target quality control data, determine whether the sewage monitoring data meets the target quality control rules, obtain the test results, and output them.

[0130] Optionally, judging whether the sewage monitoring data meets the target quality control rules according to the target quality control data, obtaining the inspection results, and outputting them, including:

[0131] If the sewage monitoring data meets the target quality control rules, the sewage monitoring data passes the quality control, and the sewage monitoring data that passes the quality control is output;

[0132] If the sewage monitoring data does not meet the target quality control rules, it will not pass the quality control, and the data items and reasons that do not meet the target quality control rules will be output.

[0133] Optionally, the sewage monitoring data quality control method further includes:

[0134] When the sewage monitoring data does not meet the target quality control rules, the quality control rule library is updated.

[0135] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0136] The embodiment of the present invention further provides a computing device, including: a processor, a memory storing a computer program, and when the computer program is executed by the processor, the method described in the above embodiment is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0137] In an embodiment of the present invention, a computer-readable storage medium is further provided, which stores instructions, and when the instructions are executed on a computer, the computer executes the method described in the above embodiment. All implementations in the above method embodiment are applicable to this embodiment, and can also achieve the same technical effect.

[0138] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0139] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0140] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0141] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0143] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0144] In addition, it should be pointed out that in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. Moreover, the steps of performing the above series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order, and some steps can be performed in parallel or independently of each other. For those of ordinary skill in the art, it is understandable that all or any steps or components of the method and apparatus of the present invention can be implemented in hardware, firmware, software or a combination thereof in any computing device (including processors, storage media, etc.) or a network of computing devices, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0145] Therefore, the purpose of the present invention can also be achieved by running a program or a group of programs on any computing device. The computing device can be a well-known general device. Therefore, the purpose of the present invention can also be achieved by simply providing a program product containing a program code that implements the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be pointed out that in the device and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. In addition, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order. Some steps can be performed in parallel or independently of each other.

[0146] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A sewage monitoring data quality control method, characterized in that: include: Obtaining sewage monitoring data; Preprocessing the sewage monitoring data to obtain a data set; According to the data set, matching target data quality control rule conditions in a quality control rule library; According to the target quality control rules that match the target data quality control rule conditions, the sewage monitoring data is subjected to quality control inspection, and the inspection results are obtained and output; Wherein, according to the data set, matching the target data quality control rule conditions in the quality control rule library includes: Matching the attributes and results of the data set with nodes of a network structure of a quality control rule base, wherein the nodes of the network structure include a first node and a second node; The method of matching the attributes and results of the data set with the nodes of the network structure of the quality control rule base includes: Mapping the attributes and results of the data set to the nodes of the network structure, wherein the first node includes one quality control rule condition and the second node includes multiple quality control rule conditions; After the data set is matched with the first node, it is then matched with the second node; Among them, according to the target quality control rules that match the target data quality control rules, the sewage monitoring data is quality controlled and tested, and the test results are obtained and output, including: After the data set is matched with the first node, it is matched with the second node to obtain target quality control data; According to the target quality control data, determine whether the sewage monitoring data meets the target quality control rules, obtain the test results, and output them; Among them, according to the drainage household portrait table, a quality control rule library is established, and each type of drainage household is used as a rule file. Each rule file is configured with attributes, conditions and results at the same time; when the data is input into the quality control rule library, its attributes and results will be transmitted to the first node network of the network structure to match the quality control rule conditions of the first node; if the match is successful, the data will be stored in the first node network and continue to propagate to the second node network of the network structure; the second node network performs combined condition matching on the matching results of multiple first node networks to find all data groups that meet all rule conditions; among them, the first node is responsible for matching a single quality control rule condition, and the second node is responsible for the combined matching between multiple quality control rule conditions, and the node includes the rule condition information that is named and described to match the data set.

2. The sewage monitoring data quality control method according to claim 1, characterized in that: The sewage monitoring data is preprocessed to obtain a data set, including: Performing data cleaning on the sewage monitoring data, processing missing values, correcting erroneous data and eliminating duplicate records; Integrate the cleaned data into a unified preset format; The data set is obtained by converting the data in a unified preset format according to preset data rules.

3. The sewage monitoring data quality control method according to claim 1, characterized in that: According to the target quality control data, determine whether the sewage monitoring data meets the target quality control rules, obtain the inspection results, and output them, including: If the sewage monitoring data meets the target quality control rules, the sewage monitoring data passes the quality control, and the sewage monitoring data that passes the quality control is output; If the sewage monitoring data does not meet the target quality control rules, it will not pass the quality control, and the data items and reasons that do not meet the target quality control rules will be output.

4. The sewage monitoring data quality control method according to claim 3 is characterized in that: Also includes: When the sewage monitoring data does not meet the target quality control rules, the quality control rule library is updated.

5. A sewage monitoring data quality control device, characterized in that: include: An acquisition module, used to acquire sewage monitoring data; A processing module, used for preprocessing the sewage monitoring data to obtain a data set; According to the data set, matching target data quality control rule conditions in a quality control rule library; According to the target quality control rules that match the target data quality control rule conditions, the sewage monitoring data is subjected to quality control inspection, and the inspection results are obtained and output; Wherein, according to the data set, matching the target data quality control rule conditions in the quality control rule library includes: Matching the attributes and results of the data set with nodes of a network structure of a quality control rule base, wherein the nodes of the network structure include a first node and a second node; The method of matching the attributes and results of the data set with the nodes of the network structure of the quality control rule base includes: Mapping the attributes and results of the data set to the nodes of the network structure, wherein the first node includes one quality control rule condition and the second node includes multiple quality control rule conditions; After the data set is matched with the first node, it is then matched with the second node; Among them, according to the target quality control rules that match the target data quality control rules, the sewage monitoring data is quality controlled and tested, and the test results are obtained and output, including: After the data set is matched with the first node, it is matched with the second node to obtain target quality control data; According to the target quality control data, determine whether the sewage monitoring data meets the target quality control rules, obtain the test results, and output them; Among them, according to the drainage household portrait table, a quality control rule library is established, and each type of drainage household is used as a rule file. Each rule file is configured with attributes, conditions and results at the same time; when the data is input into the quality control rule library, its attributes and results will be transmitted to the first node network of the network structure to match the quality control rule conditions of the first node; if the match is successful, the data will be stored in the first node network and continue to propagate to the second node network of the network structure; the second node network finds all data groups that meet the entire rule conditions by combining the single condition matching results from the first node network and the combined condition matching results from other second nodes; among them, the first node is responsible for the matching of a single quality control rule condition, and the second node is responsible for the combined matching between multiple quality control rule conditions. The node includes the named and described rule condition information that matches the data set.

6. A computing device, characterized in that include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 4 is performed.

7. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 4.

Images