Intelligent preprocessing method and system for data acquisition of concentrator

Abstract

The invention provides an intelligent preprocessing method and system for concentrator data acquisition, and relates to the technical field of data preprocessing, and the method comprises the steps: building a data acquisition network through presetting data acquisition nodes, obtaining a data transmission path, obtaining independent acquisition data packets, obtaining preprocessing nodes, and calculating node processing abnormal coefficients. Node preprocessing judgment and node processing labeling are carried out, abnormal labeling is adjusted into normal labeling, a comprehensive processing abnormal coefficient is calculated, and comprehensive preprocessing judgment is carried out on the independently collected data packet; the method comprises the steps of calculating a level anomaly coefficient, carrying out level preprocessing judgment, obtaining level preprocessing judgment information of a preset acquisition level, carrying out preprocessing anomaly positioning on a path area network, and carrying out data preprocessing adjustment through a concentrator. And preprocessing and centralized updating are carried out through the concentrator, so that the stability of a data acquisition network and the accuracy of data are ensured.

Description

Technical Field

[0001] The invention proposes a concentrator data acquisition intelligent preprocessing method and system, which relate to the technical field of data preprocessing, and in particular to the technical field of concentrator data acquisition preprocessing. Background Art

[0002] In the current field of data collection and processing, with the rapid development of the Internet of Things, big data and cloud computing technologies, the scale and complexity of data collection are increasing. Traditional data collection and processing methods often rely on manual configuration and monitoring, which is not only inefficient, but also difficult to cope with large-scale, high-frequency data collection needs. Due to the complexity of the data collection environment, the instability of the data transmission path, and the differences in the performance of data collection nodes, the existing data collection system often contains anomalies or errors in the collected data. It is difficult for existing technologies to automatically and intelligently perform data preprocessing and process anomaly identification. Summary of the invention

[0003] The present invention provides a concentrator data collection intelligent preprocessing method and system to solve the above problems: The present invention proposes a concentrator data collection intelligent preprocessing method and system, the preprocessing method comprising: S1. Build a data collection network through preset data collection nodes, obtain a collection area network through preset collection levels, generate a data transmission path to obtain a path area network, connect a concentrator, and obtain two-way communication data; S2. Obtain independent collection data packets according to the data transmission path, obtain preprocessing nodes, and then obtain node-level processing information, calculate node processing abnormality coefficients, perform node preprocessing determination, and obtain node preprocessing determination information; S3, perform node processing marking according to the node preprocessing determination information, adjust the abnormal marking to a normal marking, calculate the comprehensive processing abnormality coefficient, perform comprehensive preprocessing determination on the independent collection data packet, and obtain comprehensive preprocessing determination information; S4. Calculate the level anomaly coefficient, perform level preprocessing judgment, obtain the level preprocessing judgment information of the preset collection level, locate the preprocessing anomaly of the path area network, and perform data preprocessing adjustment through the concentrator.

[0004] Further, the S1 includes: Acquire a preset data collection node, wherein the preset data collection node performs data collection of preset collection data to obtain node collection data; According to the node positions of the preset data acquisition nodes, communication connections are performed on each preset data acquisition node to obtain a data acquisition network; Obtaining a preset collection level, dividing the data collection network horizontally into collection areas of the same level according to the preset collection level, obtaining collection areas of different collection levels, updating the data collection network, and obtaining a collection area network; Generate a data transmission path in the collection area network according to preset collection data, update the collection area network, and obtain a path area network; The path area network is connected to the concentrator for bidirectional communication to obtain bidirectional communication data.

[0005] Further, the S2 includes: Packing the node collection data of each transmission on each data transmission path to obtain independent collection data packets; Obtain a preset preprocessing process, and obtain multiple preprocessing nodes of the preset preprocessing process according to the process information; According to the preprocessing node, corresponding preprocessing operations are performed on the data of each preset collection level of each independent collection data packet to obtain node level processing information of each independent collection data packet; Calculate the node processing anomaly coefficient of each preset collection level of each independent collection data packet according to the node level processing information; Compare the node processing abnormality coefficient with a preset node abnormality threshold to obtain node abnormality comparison information; A node preprocessing determination is performed on the preset collection level of the independent collection data packet according to the node abnormality comparison information to obtain node preprocessing determination information of the preset collection level of the independent collection data packet.

[0006] Further, the S3 includes: According to the node preprocessing determination information, the preset collection level corresponding to the independent collection data packet is marked as abnormal node processing and normal node processing; When the node preprocessing determination information is a node processing abnormality mark, the corresponding preset collection level data is reprocessed according to the node processing abnormality mark until the node processing abnormality mark is converted to the node processing normal mark; Obtain node processing anomaly coefficients of all preset collection levels of independent collection data packets, and calculate the comprehensive processing anomaly coefficient of the independent collection data packets; Comparing the comprehensive processing abnormality coefficient with a preset comprehensive abnormality threshold to obtain comprehensive abnormality comparison information; A comprehensive preprocessing judgment is performed on the independently collected data packet according to the comprehensive abnormality comparison information to obtain comprehensive preprocessing judgment information of the independently collected data packet.

[0007] Further, the S4 includes: Obtain all node processing anomaly coefficients for each preset collection level, and calculate the level anomaly coefficient of the preset collection level; Compare the level abnormality coefficient with a preset level abnormality threshold to obtain a level comparison result; Performing level preprocessing determination on the preset acquisition level according to the level abnormality comparison result to obtain level preprocessing determination information of the preset acquisition level; Preprocessing anomaly location is performed on the path area network through node preprocessing determination information, comprehensive preprocessing determination information and level preprocessing determination information to obtain network preprocessing anomaly location information, and the network preprocessing anomaly location information is preprocessed and centrally updated through the concentrator to obtain update information.

[0008] Furthermore, the pretreatment system comprises: A network construction module is used to construct a data collection network through preset data collection nodes, obtain a collection area network through preset collection levels, generate a data transmission path to obtain a path area network, connect a concentrator, and obtain two-way communication data; A node preprocessing module is used to obtain independent collection data packets according to the data transmission path, obtain preprocessing nodes, and then obtain node-level processing information, calculate node processing abnormality coefficients, perform node preprocessing judgments, and obtain node preprocessing judgment information; A comprehensive preprocessing module is used to perform node processing marking according to node preprocessing judgment information, adjust abnormal marking to normal marking, calculate comprehensive processing abnormality coefficient, perform comprehensive preprocessing judgment on independent collection data packets, and obtain comprehensive preprocessing judgment information; The calculation and positioning module is used to calculate the level anomaly coefficient, perform level preprocessing judgment, obtain the level preprocessing judgment information of the preset collection level, perform preprocessing anomaly positioning on the path area network, and perform data preprocessing adjustment through the concentrator.

[0009] Furthermore, the network construction module includes: A data collection module is used to obtain a preset data collection node, wherein the preset data collection node performs data collection of preset collection data to obtain node collection data; A data collection network construction module, used to establish communication connections between various preset data collection nodes according to the node positions of the preset data collection nodes to obtain a data collection network; A regional network construction module is used to obtain a preset collection level, divide the data collection network into collection areas of the same level according to the preset collection level, obtain collection areas of different collection levels, update the data collection network, and obtain a collection area network; A path network construction module, used to generate a data transmission path in the collection area network according to preset collection data, update the collection area network, and obtain a path area network; The communication connection module is used to establish a two-way communication connection between the path area network and the concentrator to obtain two-way communication data.

[0010] Furthermore, the node preprocessing module includes: A data packaging module is used to package the node collection data of each transmission of each data transmission path to obtain an independent collection data packet; A process node acquisition module is used to acquire a preset preprocessing process and acquire multiple preprocessing nodes of the preset preprocessing process according to the process information; A data packet level processing module, used to perform corresponding preprocessing operations on data of each preset collection level of each independent collection data packet according to the preprocessing node, and obtain node level processing information of each independent collection data packet; A data packet level anomaly analysis module, used for calculating a node processing anomaly coefficient of each preset collection level of each independent collection data packet according to the node level processing information; Compare the node processing abnormality coefficient with a preset node abnormality threshold to obtain node abnormality comparison information; The data packet level abnormality determination module is used to perform node preprocessing determination on the preset collection level of the independent collection data packet according to the node abnormality comparison information, and obtain node preprocessing determination information of the preset collection level of the independent collection data packet.

[0011] Furthermore, the comprehensive preprocessing module includes: A node marking module is used to mark abnormal node processing and normal node processing for a preset collection level corresponding to an independent collection data packet according to node pre-processing determination information; When the node preprocessing determination information is a node processing abnormality mark, the corresponding preset collection level data is reprocessed according to the node processing abnormality mark until the node processing abnormality mark is converted to the node processing normal mark; A comprehensive analysis module, used to obtain node processing anomaly coefficients of all preset collection levels of independent collection data packets, and calculate the comprehensive processing anomaly coefficients of the independent collection data packets; A comprehensive determination module, used for comparing the comprehensive processing abnormality coefficient with a preset comprehensive abnormality threshold to obtain comprehensive abnormality comparison information; A comprehensive preprocessing judgment is performed on the independently collected data packet according to the comprehensive abnormality comparison information to obtain comprehensive preprocessing judgment information of the independently collected data packet.

[0012] Furthermore, the calculation and positioning module includes: A level anomaly analysis module is used to obtain all node processing anomaly coefficients of each preset collection level and calculate the level anomaly coefficient of the preset collection level; Compare the level abnormality coefficient with a preset level abnormality threshold to obtain a level comparison result; A level abnormality determination module, used for performing level preprocessing determination on the preset acquisition level according to the level abnormality comparison result, and obtaining level preprocessing determination information of the preset acquisition level; The abnormality location update module is used to perform preprocessing abnormality location on the path area network through node preprocessing judgment information, comprehensive preprocessing judgment information and level preprocessing judgment information, obtain network preprocessing abnormality location information, and perform preprocessing centralized update on the network preprocessing abnormality location information through the concentrator to obtain update information.

[0013] Beneficial effects of the present invention: Through intelligent preprocessing, anomalies and errors in data can be discovered and corrected in time, thereby improving the accuracy and reliability of data. Through separate data processing anomaly monitoring of nodes, independent data packets and levels, it is possible to process anomalies when they are still tiny, avoiding a large amount of complex data preprocessing in the later stage. Through classification methods such as nodes, independent data packets and levels, the concentrator can realize flexible data processing in terms of classification at nodes, independent data packets and levels, and accurately locate and modify data anomalies; the preprocessing process can reduce the burden of subsequent data processing and analysis, and improve the operating efficiency of the overall system. Through anomaly location and data preprocessing adjustment, potential problems in the system can be discovered and resolved in time, enhancing the stability and reliability of the system. Providing high-quality data support makes decisions based on these data more accurate and reliable, which helps to improve business efficiency and competitiveness. Through the automated preprocessing process, the need for manual intervention is reduced, thereby reducing the maintenance cost of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A schematic diagram of an intelligent preprocessing method for concentrator data collection; Figure 2 This is a communication connection diagram. DETAILED DESCRIPTION

[0015] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0016] In one embodiment of the present invention, a concentrator data collection intelligent preprocessing method and system is proposed by the present invention, and the preprocessing method includes: S1. Build a data collection network through preset data collection nodes, obtain a collection area network through preset collection levels, generate a data transmission path to obtain a path area network, connect a concentrator, and obtain two-way communication data; S2. Obtain independent collection data packets according to the data transmission path, obtain preprocessing nodes, and then obtain node-level processing information, calculate node processing abnormality coefficients, perform node preprocessing determination, and obtain node preprocessing determination information; S3, perform node processing marking according to the node preprocessing determination information, adjust the abnormal marking to a normal marking, calculate the comprehensive processing abnormality coefficient, perform comprehensive preprocessing determination on the independent collection data packet, and obtain comprehensive preprocessing determination information; S4, calculate the level anomaly coefficient, perform level preprocessing judgment, obtain the level preprocessing judgment information of the preset collection level, perform preprocessing anomaly positioning on the path area network, and perform data preprocessing adjustment through the concentrator, such as Figure 1 shown.

[0017] The working principle of the above technical solution is as follows: a basic data collection network is constructed through preset data collection nodes. According to the preset collection level (such as by geographical area, device type or data importance, etc.), a specific collection area network is obtained. Data transmission paths are generated, which define how data is obtained and transmitted in the network, thereby forming a path area network. The concentrator is connected. As a data transfer station, the concentrator is responsible for collecting data from various collection nodes, allowing two-way communication, and performing unified data preprocessing operations; extracting independent data packets from the data transmission path, all data collected according to independent collection requirements (including data corresponding to independent collection requirements in each node), each data packet contains data collected from a specific node. Identify the preprocessing nodes corresponding to each data packet, which are responsible for performing preliminary data processing tasks. Obtain node-level processing information, which includes information on data quality, integrity, timeliness, etc. Calculate the node processing anomaly coefficient to evaluate the degree of abnormality of each node data. Perform node preprocessing judgment based on the anomaly coefficient and mark the result as normal or abnormal. Based on the node preprocessing judgment information, anomaly data is marked and tried to be adjusted to a normal state (such as through data interpolation, smoothing, etc.). Calculate the comprehensive processing anomaly coefficient, which is a comprehensive assessment of the degree of anomaly of the entire data packet. Perform comprehensive preprocessing judgments to determine whether the data packet needs further cleaning or correction. Calculate the level anomaly coefficient, which is an assessment of the degree of anomaly for a preset collection level (such as geographic area or device type). Perform level preprocessing judgments to determine whether special processing is required for data at that level. Perform preprocessing anomaly location on the path area network to identify the main source or transmission path of abnormal data. Perform data preprocessing adjustments through the concentrator, such as adjusting the frequency of data collection, optimizing the data transmission path, or applying more advanced data cleaning.

[0018] The technical effect of the above technical solution is: through intelligent preprocessing, anomalies and errors in the data can be discovered and corrected in time, thereby improving the accuracy and reliability of the data. Through the data processing anomaly monitoring of nodes, independent data packets and levels, it is possible to process the anomalies when they are still microscopic, avoiding a large amount of complex data preprocessing in the later stage. Through classification methods such as nodes, independent data packets and levels, the concentrator can realize flexible data processing in the classification of nodes, independent data packets and levels, and accurately locate and modify data anomalies; the preprocessing process can reduce the burden of subsequent data processing and analysis, and improve the operating efficiency of the overall system. Through anomaly positioning and data preprocessing adjustment, potential problems in the system can be discovered and solved in time, and the stability and reliability of the system can be enhanced. Providing high-quality data support makes decisions based on these data more accurate and reliable, which helps to improve business efficiency and competitiveness. Through the automated preprocessing process, the need for manual intervention is reduced, thereby reducing the maintenance cost of the system.

[0019] In one embodiment of the present invention, the S1 includes: Acquire a preset data collection node, wherein the preset data collection node performs data collection of preset collection data to obtain node collection data; According to the node positions of the preset data acquisition nodes, communication connections are performed on each preset data acquisition node to obtain a data acquisition network; Obtaining a preset collection level, dividing the data collection network horizontally into collection areas of the same level according to the preset collection level, obtaining collection areas of different collection levels, updating the data collection network, and obtaining a collection area network; Generate a data transmission path in the collection area network according to preset collection data, update the collection area network, and obtain a path area network; The path area network is connected to the concentrator for bidirectional communication to obtain bidirectional communication data, such as Figure 2 shown.

[0020] The working principle of the above technical solution is: according to the preset configuration information or rules, identify and obtain all preset data collection nodes. These nodes are deployed at physical locations where data needs to be monitored or collected, such as sensors, meters, cameras, etc. The preset data collection nodes collect data according to the preset collection rules (such as collection frequency, data type, etc.), and store the collected data (i.e., node collection data) locally. According to the node location of the preset data collection nodes (such as geographical location, network topology, etc.), the system connects these nodes to form a data collection network by wired or wireless means. This network allows data to be transmitted and shared between nodes. The system obtains preset collection levels, which include different collection weights or collection types, etc. According to the preset collection levels, the system divides the data collection network horizontally into the same level collection areas, that is, divides the network into different collection areas. Each collection area contains a group of data collection nodes with the same collection weight, type or function. The data collection network is updated to reflect the division of these collection areas, thereby obtaining a collection area network. In the collection area network, the system generates a data transmission path according to the transmission requirements of the preset collection data (data requirement information, etc.). These paths define how data is transmitted between different collection areas to ensure that the data can reach the target location accurately and efficiently. The collection area network is updated to include these data transmission paths to obtain the path area network. The path area network is connected to the concentrator for two-way communication. The concentrator is a central processing unit that is responsible for receiving data from each collection node and processing, storing or forwarding it. Through the two-way communication connection, the concentrator can obtain two-way communication data from the path area network, that is, uplink data from the collection node to the concentrator and downlink data from the concentrator to the collection node (such as control instructions, preprocessing, etc.).

[0021] The technical effect of the above technical solution is: by presetting data collection nodes and collection rules, the system can efficiently and accurately collect the required preset data according to the preset nodes and perform data transmission and processing according to the preset path, thereby reducing data redundancy and errors. Network division and path generation are performed according to node locations and collection levels, so that the data collection network has a high degree of flexibility and scalability. By generating data transmission paths and establishing two-way communication connections, the system can ensure the reliable transmission and sharing of data in the collection area network. As the core of data management, the concentrator can uniformly receive, process and store data from various collection nodes. Through automated data collection, transmission and management processes, the system can reduce operation and maintenance costs and improve the operating efficiency and reliability of the overall system.

[0022] In one embodiment of the present invention, S2 includes: Packing the node collection data of each transmission on each data transmission path to obtain independent collection data packets; Obtain a preset preprocessing process, and obtain multiple preprocessing nodes of the preset preprocessing process according to the process information; According to the preprocessing node, corresponding preprocessing operations are performed on the data of each preset collection level of each independent collection data packet to obtain node level processing information of each independent collection data packet; Calculate the node processing anomaly coefficient of each preset collection level of each independent collection data packet according to the node level processing information; The calculation formula of the node processing abnormality coefficient is:

[0023] Among them, JDY is the node processing abnormality coefficient, E is the total number of preprocessing nodes, b is the number of preprocessing indicators of preprocessing nodes, S id is the actual processing data of the ith preprocessing index of the dth preprocessing node, y id is the preset processing data of the ith preprocessing index of the dth preprocessing node, q d is the preset weight data of the dth preprocessing node, q i is the preset weight data of the i-th preprocessing indicator; the actual processing data used in the formula are all node data.

[0024] Compare the node processing abnormality coefficient with a preset node abnormality threshold to obtain node abnormality comparison information; A node preprocessing determination is performed on the preset collection level of the independent collection data packet according to the node abnormality comparison information to obtain node preprocessing determination information of the preset collection level of the independent collection data packet.

[0025] The working principle of the above technical solution is as follows: on each data transmission path, the system will package and process the node collection data for each transmission. Data collected from the same collection node, at the same collection level, and at the same time or similar time will be integrated into an independent data packet, that is, an independent collection data packet. This can simplify the data processing process and improve the efficiency of data processing. The system will obtain preprocessing process information according to the preset configuration or rules. This preprocessing process usually includes multiple preprocessing steps to perform preprocessing node information tasks. These preprocessing nodes are designed to provide standards for performing specific data processing tasks, such as data cleaning, format conversion, data compression, etc. Based on the preprocessing process information, the system will identify the preprocessing node tasks that each independent collection data packet needs to go through, and perform corresponding preprocessing operations on the data in the data packet. These operations will vary depending on the preset collection level, because different levels of data require different processing strategies. After completing the preprocessing operation, the system will calculate the node processing anomaly coefficient of each preset collection level for each independent collection data packet based on the information collected during the processing (such as processing time, resource consumption, data changes, etc.). This coefficient is a quantitative indicator used to evaluate whether there are anomalies or errors in the data during processing. The system will compare the calculated node processing anomaly coefficient with the preset node anomaly threshold. If the anomaly coefficient of a certain collection level exceeds the threshold, it is considered that the data at that level has an anomaly in the processing process. Based on the comparison results, the system will generate node anomaly comparison information and make node preprocessing judgments on the preset collection levels of independent collection data packets. This judgment result will indicate which levels of data need further inspection or processing.

[0026] The technical effect of the above technical solution is: through the automated execution of data packaging and preprocessing processes, the system can efficiently process large amounts of data and reduce manual intervention and waiting time. Preprocessing operations can identify and correct errors and anomalies in the data, improving the accuracy and reliability of the data. The system can apply different preprocessing strategies according to the preset collection levels to meet diverse data processing needs. By calculating the node processing anomaly coefficient and comparing it with the threshold, the system can automatically detect and determine anomalies in the data processing process, improving the stability and reliability of the system. Automated data processing and anomaly detection processes can reduce the need for manual monitoring and intervention, reduce operation and maintenance costs, and improve the operating efficiency of the overall system.

[0027] In one embodiment of the present invention, S3 includes: According to the node preprocessing judgment information, the preset collection level corresponding to the independent collection data packet is marked with node processing abnormalities and node processing normality; when the node processing abnormality coefficient is greater than the preset node abnormality threshold, the node processing abnormality is marked, otherwise the node processing is marked normally.

[0028] When the node preprocessing determination information is a node processing abnormality mark, the corresponding preset collection level data is reprocessed according to the node processing abnormality mark until the node processing abnormality mark is converted to the node processing normal mark; Obtain node processing anomaly coefficients of all preset collection levels of independent collection data packets, and calculate the comprehensive processing anomaly coefficient of the independent collection data packets; The calculation formula of the comprehensive processing abnormality coefficient is:

[0029] Among them, ZCL is the comprehensive processing anomaly coefficient, r is the number of preset collection levels, t is the number of data collection nodes occupied by all information stored in independent collection data packets in all preset collection levels, JDY yo The node processing anomaly coefficient, Q, of the oth data collection node with the information of the independent collection data packet at the yth preset collection level A Preset weight data for independent data collection data packets; Comparing the comprehensive processing abnormality coefficient with a preset comprehensive abnormality threshold to obtain comprehensive abnormality comparison information; The independent data packet is subjected to comprehensive preprocessing judgment according to the comprehensive abnormality comparison information to obtain comprehensive preprocessing judgment information of the independent data packet. When the comprehensive processing abnormality coefficient is greater than the preset comprehensive abnormality threshold, the comprehensive preprocessing of the independent data packet is judged to be abnormal, otherwise it is judged to be normal.

[0030] The working principle of the above technical solution is as follows: according to the node preprocessing judgment information, the system will mark the data of each preset collection level in the independent collection data packet. If the node processing abnormality coefficient is greater than the preset node abnormality threshold, the node processing abnormality marking is performed; otherwise, the node processing is normal. The marking enables the system to quickly identify which data levels have problems. When the data of a preset collection level is marked as node processing abnormality, the system triggers the reprocessing mechanism. The system applies new preprocessing to the data at this level, or adjusts the parameters of the existing preprocessing steps to try to correct the abnormality. The reprocessing process will continue until the data at this level is re-marked as node processing normal. After completing the data marking and reprocessing of all preset collection levels, the system will obtain the node processing abnormality coefficient of each level and calculate the comprehensive processing abnormality coefficient of the independent collection data packet. This coefficient is a comprehensive indicator used to evaluate the overall abnormality of the entire data packet during the processing process. The system will compare the calculated comprehensive processing abnormality coefficient with the preset comprehensive abnormality threshold. If the comprehensive processing abnormality coefficient is greater than the preset comprehensive abnormality threshold, the comprehensive preprocessing of the independent collection data packet is judged to be abnormal; otherwise, it is judged to be normal.

[0031] The technical effect of the above technical solution is: through the node processing exception and normal marking and reprocessing mechanism, the system can more accurately identify and process anomalies in the data, thereby improving the accuracy of data processing. The calculation of the comprehensive processing anomaly coefficient and the comprehensive preprocessing judgment provide the system with an additional error detection and processing mechanism, enhancing the robustness and stability of the system. And judge whether there are anomalies in the preprocessing of the collected overall data packet, and make preprocessing adjustments accordingly. Through the automated marking, reprocessing and judgment process, the system can optimize the data processing process, reduce manual intervention and waiting time, and improve the overall processing efficiency. The system can output high-quality data that has been strictly processed and verified. The automated exception handling process can reduce the need for manual monitoring and intervention, reduce operation and maintenance costs, and improve the operating efficiency of the overall system. At the same time, by discovering and resolving anomalies in advance, the adverse effects of potential data quality problems on subsequent business can also be avoided.

[0032] In one embodiment of the present invention, the S4 includes: Obtain all node processing anomaly coefficients for each preset collection level, and calculate the level anomaly coefficient of the preset collection level; The calculation formula of the level abnormality coefficient is:

[0033] Among them, JBX is the level abnormality coefficient, h is the total number of data collection nodes of the preset collection level, n is the total number of independent collection data packets of the preset collection level, JDY kl The sum of abnormal coefficients of the node processing the lth independent data packet of the kth preset collection level, Q B It is the preset weight data of the preset collection level; Compare the level abnormality coefficient with a preset level abnormality threshold to obtain a level comparison result; Performing level preprocessing determination on the preset acquisition level according to the level abnormality comparison result to obtain level preprocessing determination information of the preset acquisition level; when the level abnormality coefficient is greater than the preset level abnormality threshold, determining that the preset acquisition level has abnormal preprocessing, otherwise, determining that it is normal preprocessing; Preprocessing anomaly location is performed on the path area network through node preprocessing determination information, comprehensive preprocessing determination information and level preprocessing determination information to obtain network preprocessing anomaly location information, and the network preprocessing anomaly location information is preprocessed and centrally updated through the concentrator to obtain update information.

[0034] The working principle of the above technical solution is as follows: for each preset collection level, the system collects the processing anomaly coefficients of all nodes under the level. These anomaly coefficients reflect the degree of anomaly of the node when processing data. Subsequently, the system calculates a comprehensive level anomaly coefficient based on these node anomaly coefficients to evaluate the overall degree of anomaly of the preset collection level. The calculated level anomaly coefficient is compared with the preset level anomaly threshold. If the level anomaly coefficient is higher than the threshold, the preset collection level is determined to have abnormal preprocessing; if it is lower than or equal to the threshold, it is determined to be normal preprocessing. This step helps the system quickly locate the collection level with problems. The system combines node preprocessing judgment information (abnormal judgment for a single node), comprehensive preprocessing judgment information (abnormal judgment for the entire data packet) and level preprocessing judgment information (abnormal judgment for a specific collection level) to locate preprocessing anomalies in the path area network. This process aims to accurately identify which parts or levels of data preprocessing in the network have abnormalities. Once the network preprocessing anomaly location information is determined, the system will centrally update these anomaly information through the concentrator. This includes updating the status of abnormal nodes, adjusting preprocessing strategies, recording abnormal logs, etc. Centralized updates ensure consistency and accuracy in processing data across the entire network.

[0035] The technical effect of the above technical solution is: by combining the pre-processing judgment information at the node, comprehensive and level levels, the system can more accurately locate the pre-processing anomalies in the network, thereby improving the accuracy and efficiency of exception handling. Timely anomaly positioning and centralized updating can enable the system to quickly respond to and handle anomalies in the network, reduce service interruptions or data quality problems caused by anomalies, and thus enhance the robustness and stability of the network. Accurate anomaly positioning enables the system to allocate resources more specifically to handle anomalies and avoid unnecessary waste of resources. For example, the system can give priority to abnormal levels or nodes that have a greater impact on the business. Through timely exception handling and updating, the system can ensure that the data in the network maintains high quality after pre-processing. The automated anomaly positioning and centralized updating mechanism reduces the need for manual intervention and reduces operation and maintenance costs. At the same time, by discovering and resolving anomalies in advance, the adverse effects of potential network failures on the business can also be avoided.

[0036] In one embodiment of the present invention, the pretreatment system comprises: A network construction module is used to construct a data collection network through preset data collection nodes, obtain a collection area network through preset collection levels, generate a data transmission path to obtain a path area network, connect a concentrator, and obtain two-way communication data; A node preprocessing module is used to obtain independent collection data packets according to the data transmission path, obtain preprocessing nodes, and then obtain node-level processing information, calculate node processing abnormality coefficients, perform node preprocessing judgments, and obtain node preprocessing judgment information; A comprehensive preprocessing module is used to perform node processing marking according to node preprocessing judgment information, adjust abnormal marking to normal marking, calculate comprehensive processing abnormality coefficient, perform comprehensive preprocessing judgment on independent collection data packets, and obtain comprehensive preprocessing judgment information; The calculation and positioning module is used to calculate the level anomaly coefficient, perform level preprocessing judgment, obtain the level preprocessing judgment information of the preset collection level, perform preprocessing anomaly positioning on the path area network, and perform data preprocessing adjustment through the concentrator.

[0037] The working principle of the above technical solution is as follows: a basic data collection network is constructed through preset data collection nodes. According to the preset collection level (such as by geographical area, device type or data importance, etc.), a specific collection area network is obtained. Data transmission paths are generated, which define how data is obtained and transmitted in the network, thereby forming a path area network. The concentrator is connected. As a data transfer station, the concentrator is responsible for collecting data from various collection nodes, allowing two-way communication, and performing unified data preprocessing operations; extracting independent data packets from the data transmission path, all data collected according to independent collection requirements (including data corresponding to independent collection requirements in each node), each data packet contains data collected from a specific node. Identify the preprocessing nodes corresponding to each data packet, which are responsible for performing preliminary data processing tasks. Obtain node-level processing information, which includes information on data quality, integrity, timeliness, etc. Calculate the node processing anomaly coefficient to evaluate the degree of abnormality of each node data. Perform node preprocessing judgment based on the anomaly coefficient and mark the result as normal or abnormal. Based on the node preprocessing judgment information, anomaly data is marked and tried to be adjusted to a normal state (such as through data interpolation, smoothing, etc.). Calculate the comprehensive processing anomaly coefficient, which is a comprehensive assessment of the degree of anomaly of the entire data packet. Perform comprehensive preprocessing judgments to determine whether the data packet needs further cleaning or correction. Calculate the level anomaly coefficient, which is an assessment of the degree of anomaly for a preset collection level (such as geographic area or device type). Perform level preprocessing judgments to determine whether special processing is required for data at that level. Perform preprocessing anomaly location on the path area network to identify the main source or transmission path of abnormal data. Perform data preprocessing adjustments through the concentrator, such as adjusting the frequency of data collection, optimizing the data transmission path, or applying more advanced data cleaning.

[0038] The technical effect of the above technical solution is: through intelligent preprocessing, anomalies and errors in the data can be discovered and corrected in time, thereby improving the accuracy and reliability of the data. Through the data processing anomaly monitoring of nodes, independent data packets and levels, it is possible to process the anomalies when they are still microscopic, avoiding a large amount of complex data preprocessing in the later stage. Through classification methods such as nodes, independent data packets and levels, the concentrator can realize flexible data processing in the classification of nodes, independent data packets and levels, and accurately locate and modify data anomalies; the preprocessing process can reduce the burden of subsequent data processing and analysis, and improve the operating efficiency of the overall system. Through anomaly positioning and data preprocessing adjustment, potential problems in the system can be discovered and solved in time, and the stability and reliability of the system can be enhanced. Providing high-quality data support makes decisions based on these data more accurate and reliable, which helps to improve business efficiency and competitiveness. Through the automated preprocessing process, the need for manual intervention is reduced, thereby reducing the maintenance cost of the system.

[0039] In one embodiment of the present invention, the network construction module includes: A data collection module is used to obtain a preset data collection node, wherein the preset data collection node performs data collection of preset collection data to obtain node collection data; A data collection network construction module, used to establish communication connections between various preset data collection nodes according to the node positions of the preset data collection nodes to obtain a data collection network; A regional network construction module is used to obtain a preset collection level, divide the data collection network into collection areas of the same level according to the preset collection level, obtain collection areas of different collection levels, update the data collection network, and obtain a collection area network; A path network construction module, used to generate a data transmission path in the collection area network according to preset collection data, update the collection area network, and obtain a path area network; The communication connection module is used to establish a two-way communication connection between the path area network and the concentrator to obtain two-way communication data.

[0040] The working principle of the above technical solution is: according to the preset configuration information or rules, identify and obtain all preset data collection nodes. These nodes are deployed at physical locations where data needs to be monitored or collected, such as sensors, meters, cameras, etc. The preset data collection nodes collect data according to the preset collection rules (such as collection frequency, data type, etc.), and store the collected data (i.e., node collection data) locally. According to the node location of the preset data collection nodes (such as geographical location, network topology, etc.), the system connects these nodes to form a data collection network by wired or wireless means. This network allows data to be transmitted and shared between nodes. The system obtains preset collection levels, which include different collection weights or collection types, etc. According to the preset collection levels, the system divides the data collection network horizontally into the same level collection areas, that is, divides the network into different collection areas. Each collection area contains a group of data collection nodes with the same collection weight, type or function. The data collection network is updated to reflect the division of these collection areas, thereby obtaining a collection area network. In the collection area network, the system generates a data transmission path according to the transmission requirements of the preset collection data (data requirement information, etc.). These paths define how data is transmitted between different collection areas to ensure that the data can reach the target location accurately and efficiently. The collection area network is updated to include these data transmission paths to obtain the path area network. The path area network is connected to the concentrator for two-way communication. The concentrator is a central processing unit that is responsible for receiving data from each collection node and processing, storing or forwarding it. Through the two-way communication connection, the concentrator can obtain two-way communication data from the path area network, that is, uplink data from the collection node to the concentrator and downlink data from the concentrator to the collection node (such as control instructions, preprocessing, etc.).

[0041] The technical effect of the above technical solution is: by presetting data collection nodes and collection rules, the system can efficiently and accurately collect the required preset data according to the preset nodes and perform data transmission and processing according to the preset path, thereby reducing data redundancy and errors. Network division and path generation are performed according to node locations and collection levels, so that the data collection network has a high degree of flexibility and scalability. By generating data transmission paths and establishing two-way communication connections, the system can ensure the reliable transmission and sharing of data in the collection area network. As the core of data management, the concentrator can uniformly receive, process and store data from various collection nodes. Through automated data collection, transmission and management processes, the system can reduce operation and maintenance costs and improve the operating efficiency and reliability of the overall system.

[0042] In one embodiment of the present invention, the node preprocessing module includes: A data packaging module is used to package the node collection data of each transmission of each data transmission path to obtain an independent collection data packet; A process node acquisition module is used to acquire a preset preprocessing process and acquire multiple preprocessing nodes of the preset preprocessing process according to the process information; A data packet level processing module, used to perform corresponding preprocessing operations on data of each preset collection level of each independent collection data packet according to the preprocessing node, and obtain node level processing information of each independent collection data packet; A data packet level anomaly analysis module, used for calculating a node processing anomaly coefficient of each preset collection level of each independent collection data packet according to the node level processing information; The calculation formula of the node processing abnormality coefficient is:

[0043] Among them, JDY is the node processing abnormality coefficient, E is the total number of preprocessing nodes, b is the number of preprocessing indicators of preprocessing nodes, S id is the actual processing data of the ith preprocessing index of the dth preprocessing node, y id is the preset processing data of the ith preprocessing index of the dth preprocessing node, q d is the preset weight data of the dth preprocessing node, q i is the preset weight data of the i-th preprocessing indicator; the actual processing data used in the formula are all node data.

[0044] Compare the node processing abnormality coefficient with a preset node abnormality threshold to obtain node abnormality comparison information; The data packet level abnormality determination module is used to perform node preprocessing determination on the preset collection level of the independent collection data packet according to the node abnormality comparison information, and obtain node preprocessing determination information of the preset collection level of the independent collection data packet.

[0045] The working principle of the above technical solution is as follows: on each data transmission path, the system will package and process the node collection data for each transmission. Data collected from the same collection node, at the same collection level, and at the same time or similar time will be integrated into an independent data packet, that is, an independent collection data packet. This can simplify the data processing process and improve the efficiency of data processing. The system will obtain preprocessing process information according to the preset configuration or rules. This preprocessing process usually includes multiple preprocessing steps to perform preprocessing node information tasks. These preprocessing nodes are designed to provide standards for performing specific data processing tasks, such as data cleaning, format conversion, data compression, etc. Based on the preprocessing process information, the system will identify the preprocessing node tasks that each independent collection data packet needs to go through, and perform corresponding preprocessing operations on the data in the data packet. These operations will vary depending on the preset collection level, because different levels of data require different processing strategies. After completing the preprocessing operation, the system will calculate the node processing anomaly coefficient of each preset collection level for each independent collection data packet based on the information collected during the processing (such as processing time, resource consumption, data changes, etc.). This coefficient is a quantitative indicator used to evaluate whether there are anomalies or errors in the data during processing. The system will compare the calculated node processing anomaly coefficient with the preset node anomaly threshold. If the anomaly coefficient of a certain collection level exceeds the threshold, it is considered that the data at that level has an anomaly in the processing process. Based on the comparison results, the system will generate node anomaly comparison information and make node preprocessing judgments on the preset collection levels of independent collection data packets. This judgment result will indicate which levels of data need further inspection or processing.

[0046] The technical effect of the above technical solution is: through the automated execution of data packaging and preprocessing processes, the system can efficiently process large amounts of data and reduce manual intervention and waiting time. Preprocessing operations can identify and correct errors and anomalies in the data, improving the accuracy and reliability of the data. The system can apply different preprocessing strategies according to the preset collection levels to meet diverse data processing needs. By calculating the node processing anomaly coefficient and comparing it with the threshold, the system can automatically detect and determine anomalies in the data processing process, improving the stability and reliability of the system. Automated data processing and anomaly detection processes can reduce the need for manual monitoring and intervention, reduce operation and maintenance costs, and improve the operating efficiency of the overall system.

[0047] In one embodiment of the present invention, the comprehensive preprocessing module comprises: The node marking module is used to mark the node processing abnormality and the node processing normality for the preset collection level corresponding to the independent collection data packet according to the node preprocessing judgment information; when the node processing abnormality coefficient is greater than the preset node abnormality threshold, the node processing abnormality is marked, otherwise the node processing is marked normally.

[0048] When the node preprocessing determination information is a node processing abnormality mark, the corresponding preset collection level data is reprocessed according to the node processing abnormality mark until the node processing abnormality mark is converted to the node processing normal mark; A comprehensive analysis module, used to obtain node processing anomaly coefficients of all preset collection levels of independent collection data packets, and calculate the comprehensive processing anomaly coefficients of the independent collection data packets; The calculation formula of the comprehensive processing abnormality coefficient is:

[0049] Among them, ZCL is the comprehensive processing anomaly coefficient, r is the number of preset collection levels, t is the number of data collection nodes occupied by all information stored in independent collection data packets in all preset collection levels, JDY yo The node processing anomaly coefficient, Q, of the oth data collection node with the information of the independent collection data packet at the yth preset collection level A Preset weight data for independent data collection data packets; A comprehensive determination module, used for comparing the comprehensive processing abnormality coefficient with a preset comprehensive abnormality threshold to obtain comprehensive abnormality comparison information; The independent data packet is subjected to comprehensive preprocessing judgment according to the comprehensive abnormality comparison information to obtain comprehensive preprocessing judgment information of the independent data packet. When the comprehensive processing abnormality coefficient is greater than the preset comprehensive abnormality threshold, the comprehensive preprocessing of the independent data packet is judged to be abnormal, otherwise it is judged to be normal.

[0050] The working principle of the above technical solution is as follows: according to the node preprocessing judgment information, the system will mark the data of each preset collection level in the independent collection data packet. If the node processing abnormality coefficient is greater than the preset node abnormality threshold, the node processing abnormality marking is performed; otherwise, the node processing is normal. The marking enables the system to quickly identify which data levels have problems. When the data of a preset collection level is marked as node processing abnormality, the system triggers the reprocessing mechanism. The system applies new preprocessing to the data at this level, or adjusts the parameters of the existing preprocessing steps to try to correct the abnormality. The reprocessing process will continue until the data at this level is re-marked as node processing normal. After completing the data marking and reprocessing of all preset collection levels, the system will obtain the node processing abnormality coefficient of each level and calculate the comprehensive processing abnormality coefficient of the independent collection data packet. This coefficient is a comprehensive indicator used to evaluate the overall abnormality of the entire data packet during the processing process. The system will compare the calculated comprehensive processing abnormality coefficient with the preset comprehensive abnormality threshold. If the comprehensive processing abnormality coefficient is greater than the preset comprehensive abnormality threshold, the comprehensive preprocessing of the independent collection data packet is judged to be abnormal; otherwise, it is judged to be normal.

[0051] The technical effect of the above technical solution is: through the node processing exception and normal marking and reprocessing mechanism, the system can more accurately identify and process anomalies in the data, thereby improving the accuracy of data processing. The calculation of the comprehensive processing anomaly coefficient and the comprehensive preprocessing judgment provide the system with an additional error detection and processing mechanism, enhancing the robustness and stability of the system. And judge whether there are anomalies in the preprocessing of the collected overall data packet, and make preprocessing adjustments accordingly. Through the automated marking, reprocessing and judgment process, the system can optimize the data processing process, reduce manual intervention and waiting time, and improve the overall processing efficiency. The system can output high-quality data that has been strictly processed and verified. The automated exception handling process can reduce the need for manual monitoring and intervention, reduce operation and maintenance costs, and improve the operating efficiency of the overall system. At the same time, by discovering and resolving anomalies in advance, the adverse effects of potential data quality problems on subsequent business can also be avoided.

[0052] In one embodiment of the present invention, the calculation and positioning module includes: A level anomaly analysis module is used to obtain all node processing anomaly coefficients of each preset collection level and calculate the level anomaly coefficient of the preset collection level; The calculation formula of the level abnormality coefficient is:

[0053] Among them, JBX is the level abnormality coefficient, h is the total number of data collection nodes of the preset collection level, n is the total number of independent collection data packets of the preset collection level, JDY kl The sum of abnormal coefficients of the node processing the lth independent data packet of the kth preset collection level, Q B It is the preset weight data of the preset collection level; Compare the level abnormality coefficient with a preset level abnormality threshold to obtain a level comparison result; A level abnormality determination module is used to perform level preprocessing determination on the preset acquisition level according to the level abnormality comparison result, and obtain level preprocessing determination information of the preset acquisition level; when the level abnormality coefficient is greater than the preset level abnormality threshold, the preset acquisition level is determined to have abnormal preprocessing, otherwise, it is determined to be normal preprocessing; The abnormality location update module is used to perform preprocessing abnormality location on the path area network through node preprocessing judgment information, comprehensive preprocessing judgment information and level preprocessing judgment information, obtain network preprocessing abnormality location information, and perform preprocessing centralized update on the network preprocessing abnormality location information through the concentrator to obtain update information.

[0054] The working principle of the above technical solution is as follows: for each preset collection level, the system collects the processing anomaly coefficients of all nodes under the level. These anomaly coefficients reflect the degree of anomaly of the node when processing data. Subsequently, the system calculates a comprehensive level anomaly coefficient based on these node anomaly coefficients to evaluate the overall degree of anomaly of the preset collection level. The calculated level anomaly coefficient is compared with the preset level anomaly threshold. If the level anomaly coefficient is higher than the threshold, the preset collection level is determined to have abnormal preprocessing; if it is lower than or equal to the threshold, it is determined to be normal preprocessing. This step helps the system quickly locate the collection level with problems. The system combines node preprocessing judgment information (abnormal judgment for a single node), comprehensive preprocessing judgment information (abnormal judgment for the entire data packet) and level preprocessing judgment information (abnormal judgment for a specific collection level) to locate preprocessing anomalies in the path area network. This process aims to accurately identify which parts or levels of data preprocessing in the network have abnormalities. Once the network preprocessing anomaly location information is determined, the system will centrally update these anomaly information through the concentrator. This includes updating the status of abnormal nodes, adjusting preprocessing strategies, recording abnormal logs, etc. Centralized updates ensure consistency and accuracy in processing data across the entire network.

[0055] The technical effect of the above technical solution is: by combining the pre-processing judgment information at the node, comprehensive and level levels, the system can more accurately locate the pre-processing anomalies in the network, thereby improving the accuracy and efficiency of exception handling. Timely anomaly positioning and centralized updating can enable the system to quickly respond to and handle anomalies in the network, reduce service interruptions or data quality problems caused by anomalies, and thus enhance the robustness and stability of the network. Accurate anomaly positioning enables the system to allocate resources more specifically to handle anomalies and avoid unnecessary waste of resources. For example, the system can give priority to abnormal levels or nodes that have a greater impact on the business. Through timely exception handling and updating, the system can ensure that the data in the network maintains high quality after pre-processing. The automated anomaly positioning and centralized updating mechanism reduces the need for manual intervention and reduces operation and maintenance costs. At the same time, by discovering and resolving anomalies in advance, the adverse effects of potential network failures on the business can also be avoided.

[0056] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A concentrator data collection intelligent preprocessing method, characterized in that: The pretreatment method comprises: S1. Build a data collection network through preset data collection nodes, obtain a collection area network through preset collection levels, generate a data transmission path to obtain a path area network, connect a concentrator, and obtain two-way communication data; S2. Obtain independent collection data packets according to the data transmission path, obtain preprocessing nodes, and then obtain node-level processing information, calculate node processing abnormality coefficients, perform node preprocessing determination, and obtain node preprocessing determination information; S3, perform node processing marking according to the node preprocessing determination information, adjust the abnormal marking to a normal marking, calculate the comprehensive processing abnormality coefficient, perform comprehensive preprocessing determination on the independent collection data packet, and obtain comprehensive preprocessing determination information; S4. Calculate the level anomaly coefficient, perform level preprocessing judgment, obtain the level preprocessing judgment information of the preset collection level, locate the preprocessing anomaly of the path area network, and perform data preprocessing adjustment through the concentrator.

2. According to claim 1, a concentrator data collection intelligent preprocessing method is characterized in that: The S1 includes: Acquire a preset data collection node, wherein the preset data collection node performs data collection of preset collection data to obtain node collection data; According to the node positions of the preset data acquisition nodes, communication connections are performed on each preset data acquisition node to obtain a data acquisition network; Obtaining a preset collection level, dividing the data collection network horizontally into collection areas of the same level according to the preset collection level, obtaining collection areas of different collection levels, updating the data collection network, and obtaining a collection area network; Generate a data transmission path in the collection area network according to preset collection data, update the collection area network, and obtain a path area network; The path area network is connected to the concentrator for bidirectional communication to obtain bidirectional communication data.

3. According to claim 1, a concentrator data acquisition intelligent preprocessing method is characterized in that: The S2 includes: Packing the node collection data of each transmission on each data transmission path to obtain independent collection data packets; Obtain a preset preprocessing process, and obtain multiple preprocessing nodes of the preset preprocessing process according to the process information; According to the preprocessing node, corresponding preprocessing operations are performed on the data of each preset collection level of each independent collection data packet to obtain node level processing information of each independent collection data packet; Calculating a node processing anomaly coefficient of each preset collection level for each independent collection data packet according to the node level processing information; Compare the node processing abnormality coefficient with a preset node abnormality threshold to obtain node abnormality comparison information; A node preprocessing determination is performed on the preset collection level of the independent collection data packet according to the node abnormality comparison information to obtain node preprocessing determination information of the preset collection level of the independent collection data packet.

4. According to claim 1, a concentrator data acquisition intelligent preprocessing method is characterized in that: The S3 includes: According to the node preprocessing determination information, the preset collection level corresponding to the independent collection data packet is marked as abnormal node processing and normal node processing; When the node preprocessing determination information is a node processing abnormality mark, the corresponding preset collection level data is reprocessed according to the node processing abnormality mark until the node processing abnormality mark is converted to the node processing normal mark; Obtain node processing anomaly coefficients of all preset collection levels of independent collection data packets, and calculate the comprehensive processing anomaly coefficient of the independent collection data packets; Comparing the comprehensive processing abnormality coefficient with a preset comprehensive abnormality threshold to obtain comprehensive abnormality comparison information; A comprehensive preprocessing judgment is performed on the independently collected data packet according to the comprehensive abnormality comparison information to obtain comprehensive preprocessing judgment information of the independently collected data packet.

5. According to claim 1, a concentrator data collection intelligent preprocessing method is characterized in that: The S4 includes: Obtain all node processing anomaly coefficients for each preset collection level, and calculate the level anomaly coefficient of the preset collection level; Compare the level abnormality coefficient with a preset level abnormality threshold to obtain a level comparison result; Performing level preprocessing determination on the preset acquisition level according to the level abnormality comparison result to obtain level preprocessing determination information of the preset acquisition level; Preprocessing anomaly location is performed on the path area network through node preprocessing determination information, comprehensive preprocessing determination information and level preprocessing determination information to obtain network preprocessing anomaly location information, and the network preprocessing anomaly location information is preprocessed and centrally updated through the concentrator to obtain update information.

6. A concentrator data acquisition intelligent preprocessing system, characterized in that: The pretreatment system comprises: A network construction module is used to construct a data collection network through preset data collection nodes, obtain a collection area network through preset collection levels, generate a data transmission path to obtain a path area network, connect a concentrator, and obtain two-way communication data; A node preprocessing module is used to obtain independent collection data packets according to the data transmission path, obtain preprocessing nodes, and then obtain node-level processing information, calculate node processing abnormality coefficients, perform node preprocessing judgments, and obtain node preprocessing judgment information; A comprehensive preprocessing module is used to perform node processing marking according to node preprocessing judgment information, adjust abnormal marking to normal marking, calculate comprehensive processing abnormality coefficient, perform comprehensive preprocessing judgment on independent collection data packets, and obtain comprehensive preprocessing judgment information; The calculation and positioning module is used to calculate the level anomaly coefficient, perform level preprocessing judgment, obtain the level preprocessing judgment information of the preset collection level, perform preprocessing anomaly positioning on the path area network, and perform data preprocessing adjustment through the concentrator.

7. The concentrator data acquisition intelligent preprocessing system according to claim 6, characterized in that: The network building module includes: A data collection module is used to obtain a preset data collection node, wherein the preset data collection node performs data collection of preset collection data to obtain node collection data; A data collection network construction module, used to establish communication connections between various preset data collection nodes according to the node positions of the preset data collection nodes to obtain a data collection network; A regional network construction module is used to obtain a preset collection level, divide the data collection network into collection areas of the same level according to the preset collection level, obtain collection areas of different collection levels, update the data collection network, and obtain a collection area network; A path network construction module, used to generate a data transmission path in the collection area network according to preset collection data, update the collection area network, and obtain a path area network; The communication connection module is used to establish a two-way communication connection between the path area network and the concentrator to obtain two-way communication data.

8. The concentrator data acquisition intelligent preprocessing system according to claim 6, characterized in that: The node preprocessing module comprises: A data packaging module is used to package the node collection data of each transmission of each data transmission path to obtain an independent collection data packet; A process node acquisition module is used to acquire a preset preprocessing process and acquire multiple preprocessing nodes of the preset preprocessing process according to the process information; A data packet level processing module, used to perform corresponding preprocessing operations on data of each preset collection level of each independent collection data packet according to the preprocessing node, and obtain node level processing information of each independent collection data packet; A data packet level anomaly analysis module, used for calculating a node processing anomaly coefficient of each preset collection level of each independent collection data packet according to the node level processing information; Compare the node processing abnormality coefficient with a preset node abnormality threshold to obtain node abnormality comparison information; The data packet level abnormality determination module is used to perform node preprocessing determination on the preset collection level of the independent collection data packet according to the node abnormality comparison information, and obtain node preprocessing determination information of the preset collection level of the independent collection data packet.

9. The concentrator data acquisition intelligent preprocessing system according to claim 6, characterized in that: The comprehensive preprocessing module comprises: A node marking module is used to mark abnormal node processing and normal node processing for a preset collection level corresponding to an independent collection data packet according to node pre-processing determination information; When the node preprocessing determination information is a node processing abnormality mark, the corresponding preset collection level data is reprocessed according to the node processing abnormality mark until the node processing abnormality mark is converted to the node processing normal mark; A comprehensive analysis module, used to obtain node processing anomaly coefficients of all preset collection levels of independent collection data packets, and calculate the comprehensive processing anomaly coefficients of the independent collection data packets; A comprehensive determination module, used for comparing the comprehensive processing abnormality coefficient with a preset comprehensive abnormality threshold to obtain comprehensive abnormality comparison information; A comprehensive preprocessing judgment is performed on the independently collected data packet according to the comprehensive abnormality comparison information to obtain comprehensive preprocessing judgment information of the independently collected data packet.

10. The concentrator data acquisition intelligent preprocessing system according to claim 6, characterized in that: The calculation and positioning module includes: A level anomaly analysis module is used to obtain all node processing anomaly coefficients of each preset collection level and calculate the level anomaly coefficient of the preset collection level; Compare the level abnormality coefficient with a preset level abnormality threshold to obtain a level comparison result; A level abnormality determination module, used for performing level preprocessing determination on the preset acquisition level according to the level abnormality comparison result, and obtaining level preprocessing determination information of the preset acquisition level; The abnormality location update module is used to perform preprocessing abnormality location on the path area network through node preprocessing judgment information, comprehensive preprocessing judgment information and level preprocessing judgment information, obtain network preprocessing abnormality location information, and perform preprocessing centralized update on the network preprocessing abnormality location information through the concentrator to obtain update information.

Images