Data processing method and system based on Internet of Things communication

Abstract

The invention discloses a data processing method and system based on Internet of Things communication, and relates to the technical field of communication data processing, and the method comprises the steps: analyzing a communication topological structure to determine communication line information between a data collection end and a data processing end, so as to evaluate the influence or interference condition of each communication line; and a basis is provided for subsequent communication transmission quality evaluation. The communication transmission quality and the multi-source data quality are evaluated according to the communication line information and the matching relationship between the multi-source data and the communication data, and a basis is provided for subsequent data fusion, so that the reliability of data processing is improved. The combination weight of each type of multi-source data is distributed through the communication transmission quality of the multi-source data and the quality of the multi-source data, multi-source data fusion is carried out according to the multi-source data association diagram and the combination weight of the multi-source data, a fusion information set is generated, value information is indirectly reflected through the fusion information set, and subsequent value information analysis is assisted. The data redundancy is reduced, and the safe and stable operation of the Internet of Things is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of communication data processing, and in particular to a data processing method and system based on Internet of Things communication. Background Art

[0002] Data processing in IoT communications involves the widespread deployment of IoT devices and the real-time generation and transmission of data. With the rapid development of IoT technology, a wide range of intelligent devices, such as sensors and RFID tags, have been deployed in large numbers, capable of collecting and transmitting data in real time. This data is characterized by its time-series nature, structured nature, sparsity, massive volume, diversity, and real-time nature, placing high demands on data processing systems.

[0003] The goal of IoT communication data processing is to efficiently and accurately process and analyze this data to extract valuable information and knowledge. Through technologies such as data cleaning, data storage, data fusion, and data mining, IoT data can be pre-processed, stored, integrated, and analyzed. This supports enterprise decision-making, optimizes operational efficiency, creates new business models, and drives the in-depth development of digital transformation. Furthermore, data processing can help achieve remote equipment monitoring, fault prediction, and efficient resource utilization, thereby enhancing enterprise competitiveness.

[0004] In the existing technology, in order to extract valuable information from multi-source data, it may be necessary to refer to multi-source data from multiple data acquisition terminals for joint processing and analysis. The large number of data categories and quantities makes the data redundant, and there is no data that can directly represent the relationship between multi-source data, resulting in poor data processing effects and low adaptability and reliability of processing.

[0005] Therefore, how to improve the processing effect, adaptability and reliability is a technical problem that needs to be solved. Summary of the Invention

[0006] The purpose of the present invention is to solve the problems in the prior art of poor data processing effect, low adaptability and reliability of processing under Internet of Things communication, and to propose a data processing method based on Internet of Things communication, which includes: Obtaining a communication network of the target Internet of Things, determining a data acquisition end and a data processing end in the communication topology of the target Internet of Things, establishing a communication topology based on the data acquisition end and the data processing end, and analyzing the communication topology to determine communication line information between the data acquisition end and the data processing end; Multi-source data is acquired through the data acquisition end, which transmits the multi-source data to the data processing end. At the same time, network monitoring tools are used to capture communication data during the data transmission process, and the multi-source data is matched with the communication data to obtain the matching relationship between the multi-source data and the communication data. Evaluate the communication transmission quality and multi-source data quality based on the communication line information and the matching relationship between multi-source data and communication data, mine the association rules between data collection terminals, match the multi-source data according to the association rules, and generate a multi-source data association graph; The combination weight of each type of multi-source data is assigned based on the communication transmission quality and multi-source data quality of the multi-source data. The multi-source data is fused according to the multi-source data association graph and the combination weight of the multi-source data to generate a fusion information set to help further data analysis.

[0007] In some embodiments of the present application, analyzing the communication topology to determine the communication line information between the data acquisition end and the data processing end includes: The communication topology is converted into a circuit diagram in graph theory. A type of node in the circuit diagram represents the data acquisition end or the data processing end. The edges in the circuit diagram represent the communication link between the data acquisition end and the data processing end. The length of the edge is quantified based on the communication link status of wired transmission and wireless transmission. Collect information about interference sources in the actual environment corresponding to the communication topology, mark the interference sources at the corresponding positions in the diagram, and determine the interference intensity of the interference sources based on the interference source information, the distance between the interference source and the data acquisition terminal, and the distance between the interference source and the communication link. Treat the interference sources as Class II nodes in the circuit diagram and mark the interference intensity of each Class II node. The influence amount of each edge in the line graph is determined according to the length of the edge in the line graph, the communication link condition and the interference strength of the second type of nodes, and the influence amount and the communication link condition are used as the communication line information of the edge.

[0008] In some embodiments of the present application, the amount of influence on each edge in the circuit diagram is determined based on the length of the edge in the circuit diagram, the communication link status, and the interference strength of the second type of nodes, including: The affected range of the edge is determined by the length of the edge in the circuit diagram and the communication link status. The second-class nodes within the affected range are counted and the interference strength of each second-class node is distributed to obtain the interference strength of each second-class node for the edge. The impact amount of each edge is determined by combining the length of the edge in the circuit diagram. ; in, In the circuit diagram The amount of influence on the edge, In the circuit diagram The length of the side, is the initial impact, Indicated by the circuit diagram The initial influence amount obtained by mapping the length of the edge, In the circuit diagram The number of type II nodes within the affected range of the edge, For the The influence weight of the second-class nodes, For the circuit diagram The first edge The interference intensity of the second-class nodes, In the circuit diagram The preset constant corresponding to the edge.

[0009] In some embodiments of the present application, multi-source data is matched with communication data to obtain a matching relationship between multi-source data and communication data, including: A time window is set according to the length of the edges and communication links in the circuit diagram. The corresponding communication data is searched in this time window according to the timestamps and other matching features of the multi-source data to obtain the matching relationship between the multi-source data and the communication data. Other matching features include one or more of the packet size features, IP address features, protocol type features and data content features.

[0010] In some embodiments of the present application, the communication transmission quality and the multi-source data quality are evaluated based on the communication line information and the matching relationship between the multi-source data and the communication data, including: Confirm the attribute information of the multi-source data, and set a reasonable interval of communication data that matches the multi-source data based on the influence amount, communication link status, and the attribute information of the multi-source data; Compare the actual value and reasonable interval of the same type of communication data. When the actual value is within the reasonable interval, the deviation value of this type of communication data is 0. Otherwise, the deviation value of this type of communication data is the difference between the actual value and the reasonable interval. The endpoint of the reasonable interval close to the actual value is regarded as the near endpoint, and the difference between the actual value and the reasonable interval is the difference between the actual value and the near endpoint; The deviation values of each type of communication data are integrated, and the integrated deviation values are used to describe the communication transmission quality; Conduct multi-dimensional analysis and inspection on multi-source data to determine the quality of multi-source data.

[0011] In some embodiments of the present application, mining association rules between data collection terminals includes: Acquire the multi-source data previously generated by the data collection end, perform data preprocessing on the multi-source data, analyze the scale characteristics of the multi-source data of each data collection end to determine the liquidity of the multi-source data of each data collection end, set the parameters of the mining algorithm based on the liquidity of the multi-source data of the data collection end, and perform association rule mining on the preprocessed multi-source data to obtain association rules.

[0012] In some embodiments of the present application, the combination weight of each type of multi-source data is allocated based on the communication transmission quality and multi-source data quality of the multi-source data, including: Evaluate the encryption security of each type of multi-source data at the data collection end and during transmission, and define the proofreading value of each type of multi-source data based on encryption security, communication transmission quality of multi-source data, and multi-source data quality; ; in, For the The calibration value of multi-source data, 、 Respectively The conversion coefficient corresponding to the communication transmission quality of multi-source data and the conversion coefficient corresponding to the quality of multi-source data, 、 Respectively Communication transmission quality and multi-source data quality of similar multi-source data, For the Encryption security of multi-source data, For the Preset constants corresponding to multi-source data; The calibration values of all multi-source data are normalized, and then the combination weight of each type of multi-source data is assigned according to the calibration values of the multi-source data.

[0013] In some embodiments of the present application, multi-source data is fused according to the multi-source data association graph and the combination weight of the multi-source data to generate a fusion information set, including: According to the multi-source data association graph, the associated multi-source data are weightedly summed according to their respective combination weights to obtain a representative value. The fusion information set includes the associated multi-source data and the representative value.

[0014] Correspondingly, the present application also provides a data processing system based on Internet of Things communication, comprising: A line module is used to obtain the communication network of the target Internet of Things, determine the data acquisition end and the data processing end in the communication topology of the target Internet of Things, establish a communication topology based on the data acquisition end and the data processing end, and analyze the communication topology to determine the communication line information between the data acquisition end and the data processing end; The matching module is used to obtain multi-source data through the data acquisition terminal, which transmits the multi-source data to the data processing terminal. At the same time, a network monitoring tool is used to capture the communication data during the data transmission process, and the multi-source data is matched with the communication data to obtain a matching relationship between the multi-source data and the communication data. The association module is used to evaluate the communication transmission quality and multi-source data quality based on the communication line information and the matching relationship between multi-source data and communication data, to mine the association rules between data collection terminals, to match multi-source data according to the association rules, and to generate a multi-source data association graph; The analysis module is used to assign the combination weight of each type of multi-source data based on the communication transmission quality and multi-source data quality of the multi-source data, fuse the multi-source data according to the multi-source data association graph and the combination weight of the multi-source data, and generate a fusion information set to help subsequent further data analysis.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. Analyze the communication topology to determine the communication line information between the data acquisition end and the data processing end, so as to evaluate the impact or interference of each communication line and provide a basis for subsequent communication transmission quality evaluation.

[0016] 2. Evaluate communication transmission quality and multi-source data quality based on the matching relationship between communication line information and multi-source data and communication data, providing a basis for subsequent data fusion and improving data processing reliability. The combination weights for each type of multi-source data are assigned based on the communication transmission quality and multi-source data quality. Multi-source data fusion is performed based on the multi-source data association graph and the combination weights. The multi-source data is adaptively combined according to the combination weights to generate a fused information set. This fused information set indirectly reflects valuable information and assists in subsequent value information analysis. This reduces data redundancy, improves data processing efficiency, and ensures the secure and stable operation of the Internet of Things. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a data processing method based on Internet of Things communication proposed by the present invention; Figure 2 This is a structural diagram of a data processing system based on Internet of Things communication proposed by the present invention. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0019] Reference Figure 1 , a data processing method based on Internet of Things communication, comprising the following steps: Step S101, obtain the communication network of the target Internet of Things, determine the data acquisition end and the data processing end in the communication topology structure of the target Internet of Things, establish a communication topology structure based on the data acquisition end and the data processing end, and analyze the communication topology structure to determine the communication line information between the data acquisition end and the data processing end.

[0020] In this embodiment, all data acquisition terminals (such as sensors and smart devices) and data processing terminals (such as data centers and cloud servers) are identified. Based on the obtained network topology, a communication topology model is established using graph theory, where nodes represent data acquisition terminals and data processing terminals, and edges represent communication links.

[0021] In some embodiments of the present application, analyzing the communication topology to determine the communication line information between the data acquisition end and the data processing end includes: The communication topology is converted into a circuit diagram in graph theory. A type of node in the circuit diagram represents the data acquisition end or the data processing end. The edges in the circuit diagram represent the communication link between the data acquisition end and the data processing end. The length of the edge is quantified based on the communication link status of wired transmission and wireless transmission. Collect information about interference sources in the actual environment corresponding to the communication topology, mark the interference sources at the corresponding positions in the diagram, and determine the interference intensity of the interference sources based on the interference source information, the distance between the interference source and the data acquisition terminal, and the distance between the interference source and the communication link. Treat the interference sources as Class II nodes in the circuit diagram and mark the interference intensity of each Class II node. The influence amount of each edge in the line graph is determined according to the length of the edge in the line graph, the communication link condition and the interference strength of the second type of nodes, and the influence amount and the communication link condition are used as the communication line information of the edge.

[0022] In this embodiment, the communication link status includes whether it is wired or wireless, as well as the specific type of line (such as optical fiber, network cable, Wi-Fi, Bluetooth, etc.). For edge quantification, for wired lines, the actual length is recorded; for wireless lines, the coverage range or signal strength can be recorded.

[0023] In this embodiment, interference source information refers to specific information (such as power and frequency) about electromagnetic devices that can affect communication signals. The interference intensity is determined based on a combination of this information, the distance between the interference source and the data acquisition terminal, and the distance between the interference source and the communication link. Generally speaking, the closer the interference source is to the line, the greater the impact of the electromagnetic field it generates on the line, potentially resulting in greater interference. This is because the intensity of the electromagnetic field decreases with distance, so closer distances generally indicate higher interference levels. The acquisition terminal is responsible for capturing and recording changes in line status. If the interference source is closer to the acquisition terminal, the collected signal may contain more interference components, which may affect the accuracy and reliability of the data.

[0024] It can be understood that the impact amount on each edge refers to the degree of impact of interference on the communication line.

[0025] In some embodiments of the present application, the amount of influence on each edge in the circuit diagram is determined based on the length of the edge in the circuit diagram, the communication link status, and the interference strength of the second type of nodes, including: The affected range of the edge is determined by the length of the edge in the circuit diagram and the communication link status. The second-class nodes within the affected range are counted and the interference strength of each second-class node is distributed to obtain the interference strength of each second-class node for the edge. The impact amount of each edge is determined by combining the length of the edge in the circuit diagram. ; in, In the circuit diagram The amount of influence on the edge, In the circuit diagram The length of the side, is the initial impact, Indicated by the circuit diagram The initial influence amount obtained by mapping the length of the edge, In the circuit diagram The number of type II nodes within the affected range of the edge, For the The influence weight of the second-class nodes, For the circuit diagram The first edge The interference intensity of the second-class nodes, In the circuit diagram The preset constant corresponding to the edge.

[0026] In this embodiment, a preliminary impact range is determined for each edge based on its length (which can reflect effects such as signal attenuation), communication link type (such as optical fiber, copper cable, and wireless communication), and protocol (such as TCP / IP, HTTP, and Wi-Fi), which have different transmission characteristics and anti-interference capabilities). The interference strength of each Class II node is distributed to obtain the interference strength of each Class II node with respect to the edge. This specifies the interference strength of the interference source, that is, the specific interference strength of the interference source on a particular edge (communication transmission route).

[0027] In this embodiment, the lengths of different sides represent different degrees of signal attenuation, so the length corresponds to an initial impact amount. Indicates the correction of the initial impact of the interference source on the edge.

[0028] In step S102, multi-source data is acquired through the data acquisition terminal, and the data acquisition terminal transmits the multi-source data to the data processing terminal. At the same time, a network monitoring tool is used to capture the communication data during the data transmission process, and the multi-source data is matched with the communication data to obtain a matching relationship between the multi-source data and the communication data.

[0029] In this embodiment, a data acquisition program is deployed on the data acquisition terminal to collect multi-source data according to a preset sampling frequency and data format. A network monitoring tool (such as Wireshark or tcpdump) is deployed on the communication network to capture communication data packets during data transmission. The captured communication data packets are timestamped and matched with the multi-source data to ensure that each data packet is mapped to a specific data source.

[0030] It can be understood that the matching relationship between multi-source data and communication data mentioned here, for example, data acquisition terminal 1 collects multi-source data 1, and the communication parameters generated by multi-source data 1 during the transmission process are communication data 1, and multi-source data 1 corresponds to communication data 1.

[0031] In some embodiments of the present application, multi-source data is matched with communication data to obtain a matching relationship between multi-source data and communication data, including: A time window is set according to the length of the edges and communication links in the circuit diagram. The corresponding communication data is searched in this time window according to the timestamps and other matching features of the multi-source data to obtain the matching relationship between the multi-source data and the communication data. Other matching features include one or more of the packet size features, IP address features, protocol type features and data content features.

[0032] In this embodiment, in order to improve the accuracy of the matching, since there may be a slight time difference between the collection of communication data and multi-source data (for example, due to factors such as network delays and data processing delays), it is not possible to rely solely on precise timestamp matching. Instead, a reasonable time window should be set to allow timestamps to fluctuate within a certain range. For example, a time window of ±1 second or ±500 milliseconds can be set, and data records within this time window are considered relevant. In addition to timestamps, other feature information can also be used to further confirm the matching relationship between communication data and multi-source data. This feature information may include packet size, source IP address, destination IP address, protocol type, data content, etc.

[0033] Packet size matching: If both the communication data and the multi-source data contain packet size information, packet size can be used as a matching condition. However, it should be noted that packet size can be affected by various factors (such as data compression and encryption), so matching cannot be determined solely based on packet size.

[0034] IP address matching: If both the communication data and the multi-source data contain source or destination IP address information, the IP address can be used as a matching condition. This can significantly improve matching accuracy, especially when the multi-source data contains metadata related to a specific IP address (such as device ID, user ID, etc.).

[0035] Protocol type matching: Different communication protocols may have different packet formats and transmission characteristics. Therefore, if both communication data and multi-source data contain information about the protocol type, the protocol type can be used as a matching condition. This helps exclude data records that are not related to a specific protocol.

[0036] Step S103, evaluating the communication transmission quality and the multi-source data quality based on the communication line information and the matching relationship between the multi-source data and the communication data, mining the association rules between the data collection terminals, matching the multi-source data according to the association rules, and generating a multi-source data association graph.

[0037] In this embodiment, quality assessment is performed on multi-source data and communication transmission conditions to facilitate subsequent data fusion, and associated data is associated to generate a multi-source data association graph.

[0038] In some embodiments of the present application, the communication transmission quality and the multi-source data quality are evaluated based on the communication line information and the matching relationship between the multi-source data and the communication data, including: Confirm the attribute information of the multi-source data, and set a reasonable interval of communication data that matches the multi-source data based on the influence amount, communication link status, and the attribute information of the multi-source data; Compare the actual value and reasonable interval of the same type of communication data. When the actual value is within the reasonable interval, the deviation value of this type of communication data is 0. Otherwise, the deviation value of this type of communication data is the difference between the actual value and the reasonable interval. The endpoint of the reasonable interval close to the actual value is regarded as the near endpoint, and the difference between the actual value and the reasonable interval is the difference between the actual value and the near endpoint; The deviation values of each type of communication data are integrated, and the integrated deviation values are used to describe the communication transmission quality; Conduct multi-dimensional analysis and inspection on multi-source data to determine the quality of multi-source data.

[0039] In this embodiment, the communication data includes transmission delay, packet loss rate, bandwidth utilization, etc. The attribute information of multi-source data refers to the attributes that can affect the communication data, such as data volume, format, type, etc. These will affect the setting of the reasonable interval of the communication data. Multi-dimensional analysis (integrity, accuracy, timeliness, etc.) and inspection are performed on the multi-source data to determine the quality of the multi-source data. Data integrity assessment: Check whether the data is lost or damaged during transmission. Verify whether the data is received in the expected format and structure. Data accuracy assessment: Compare the data at the data source and the receiving end to ensure that the data has not been tampered with during transmission. Use methods such as checksums or hash values to verify data integrity. Data timeliness assessment: Analyze the time difference from data generation to reception to ensure that the data arrives within a reasonable time. Evaluate whether the data meets real-time requirements.

[0040] In some embodiments of the present application, mining association rules between data collection terminals includes: Acquire the multi-source data previously generated by the data collection end, perform data preprocessing on the multi-source data, analyze the scale characteristics of the multi-source data of each data collection end to determine the liquidity of the multi-source data of each data collection end, set the parameters of the mining algorithm based on the liquidity of the multi-source data of the data collection end, and perform association rule mining on the preprocessed multi-source data to obtain association rules.

[0041] In this embodiment, scale characteristics include data volume, data update frequency, etc., and liquidity (generally, average data change) is determined. The parameters (minimum support, minimum confidence, etc.) of the mining algorithm (common algorithms include Apriori algorithm, FP-Growth algorithm, etc.) are set based on the liquidity.

[0042] It is understandable that the specific goal of the association rules here is the goal of further subsequent analysis, such as device status detection and prediction under the Internet of Things, resource optimization and scheduling of the Internet of Things, anomaly detection, etc.

[0043] In step S104, a combination weight of each type of multi-source data is assigned based on the communication transmission quality and the multi-source data quality of the multi-source data, and multi-source data is fused according to the multi-source data association graph and the combination weight of the multi-source data to generate a fusion information set to assist in subsequent further data analysis.

[0044] In this embodiment, multi-source data is fused by weighting the data quality and communication transmission quality to facilitate subsequent data analysis (such as equipment status monitoring and prediction, resource optimization and scheduling, and anomaly detection and identification). The fused information set is used as input for subsequent data analysis, mining, and decision support.

[0045] In some embodiments of the present application, the combination weight of each type of multi-source data is allocated based on the communication transmission quality and multi-source data quality of the multi-source data, including: Evaluate the encryption security of each type of multi-source data at the data collection end and during transmission, and define the proofreading value of each type of multi-source data based on encryption security, communication transmission quality of multi-source data, and multi-source data quality; ; in, For the The calibration value of multi-source data, 、 Respectively The conversion coefficient corresponding to the communication transmission quality of multi-source data and the conversion coefficient corresponding to the quality of multi-source data, 、 Respectively Communication transmission quality and multi-source data quality of similar multi-source data, For the Encryption security of multi-source data, For the Preset constants corresponding to multi-source data; The calibration values of all multi-source data are normalized, and then the combination weight of each type of multi-source data is assigned according to the calibration values of the multi-source data.

[0046] In this embodiment, different proofreading values correspond to different combination weights. The encryption of data collection and transmission is also a way to describe the quality. It represents the correction of the combination of communication transmission quality and multi-source data quality through encryption security.

[0047] In some embodiments of the present application, multi-source data is fused according to the multi-source data association graph and the combination weight of the multi-source data to generate a fusion information set, including: According to the multi-source data association graph, the associated multi-source data are weightedly summed according to their respective combination weights to obtain a representative value. The fusion information set includes the associated multi-source data and the representative value.

[0048] In this embodiment, the weighted sum can be the features of each multi-source data after refinement, and the representative value represents the contribution of the fused information set to subsequent analysis. The fused information set includes associated multi-source data (which can be the data obtained by combining the features extracted from each multi-source data) and the representative value.

[0049] Compared with the prior art, the present invention has the following beneficial effects: 1. Analyze the communication topology to determine the communication line information between the data acquisition end and the data processing end, so as to evaluate the impact or interference of each communication line and provide a basis for subsequent communication transmission quality evaluation.

[0050] 2. Evaluate communication transmission quality and multi-source data quality based on the matching relationship between communication line information and multi-source data and communication data, providing a basis for subsequent data fusion and improving data processing reliability. The combination weights for each type of multi-source data are assigned based on the communication transmission quality and multi-source data quality. Multi-source data fusion is performed based on the multi-source data association graph and the combination weights. The multi-source data is adaptively combined according to the combination weights to generate a fused information set. This fused information set indirectly reflects valuable information and assists in subsequent value information analysis. This reduces data redundancy, improves data processing efficiency, and ensures the secure and stable operation of the Internet of Things.

[0051] Improved Data Accuracy: By integrating data from multiple sources and processing it through a fusion algorithm, the fused information set can eliminate redundancy and errors in the data, thereby improving data accuracy. This improved accuracy helps subsequent data analysis draw more reliable conclusions.

[0052] Enhanced data integrity: Because the fused information set contains comprehensive information from all multi-source data, it provides a more comprehensive view of the data. This completeness enables data analysts to gain a more comprehensive understanding of the data and discover more potential patterns and trends.

[0053] Improved decision support capabilities: The high-quality data from the fused information set provides a solid foundation for decision-making. Whether it's business or technical decisions, deeper analysis and predictions based on the fused information set enable more informed decisions.

[0054] Through the above description of the embodiments, those skilled in the art will clearly understand that the present invention can be implemented via hardware or via software combined with a necessary general-purpose hardware platform. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product. This software product can be stored on a non-volatile storage medium (such as a CD-ROM, USB flash drive, or external hard drive) and includes instructions for enabling a computer device (such as a personal computer, server, or network device) to execute the methods described in various implementation scenarios of the present invention.

[0055] Correspondingly, this application also provides a data processing system based on Internet of Things communication, such as Figure 2 Shown, including, A line module is used to obtain the communication network of the target Internet of Things, determine the data acquisition end and the data processing end in the communication topology of the target Internet of Things, establish a communication topology based on the data acquisition end and the data processing end, and analyze the communication topology to determine the communication line information between the data acquisition end and the data processing end; The matching module is used to obtain multi-source data through the data acquisition terminal, which transmits the multi-source data to the data processing terminal. At the same time, a network monitoring tool is used to capture the communication data during the data transmission process, and the multi-source data is matched with the communication data to obtain a matching relationship between the multi-source data and the communication data. The association module is used to evaluate the communication transmission quality and multi-source data quality based on the communication line information and the matching relationship between multi-source data and communication data, to mine the association rules between data collection terminals, to match multi-source data according to the association rules, and to generate a multi-source data association graph; The analysis module is used to assign the combination weight of each type of multi-source data based on the communication transmission quality and multi-source data quality of the multi-source data, fuse the multi-source data according to the multi-source data association graph and the combination weight of the multi-source data, and generate a fusion information set to help subsequent further data analysis.

[0056] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0057] Those skilled in the art will appreciate that the modules in the system of the implementation scenario can be distributed in the system of the implementation scenario according to the implementation scenario description, or can be modified accordingly and located in one or more systems different from the implementation scenario. The modules of the above implementation scenario can be combined into one module or further divided into multiple submodules.

[0058] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A data processing method based on Internet of Things communication, characterized in that: include, Obtaining a communication network of the target Internet of Things, determining a data acquisition end and a data processing end in the communication topology of the target Internet of Things, establishing a communication topology based on the data acquisition end and the data processing end, and analyzing the communication topology to determine communication line information between the data acquisition end and the data processing end; Multi-source data is acquired through the data acquisition end, which transmits the multi-source data to the data processing end. At the same time, network monitoring tools are used to capture communication data during the data transmission process, and the multi-source data is matched with the communication data to obtain the matching relationship between the multi-source data and the communication data. Evaluate the communication transmission quality and multi-source data quality based on the communication line information and the matching relationship between multi-source data and communication data, mine the association rules between data collection terminals, match the multi-source data according to the association rules, and generate a multi-source data association graph; The combination weight of each type of multi-source data is assigned based on the communication transmission quality and multi-source data quality of the multi-source data. The multi-source data is fused according to the multi-source data association graph and the combination weight of the multi-source data to generate a fusion information set to help further data analysis.

2. The data processing method based on Internet of Things communication according to claim 1, characterized in that: Analyze the communication topology to determine the communication line information between the data acquisition end and the data processing end, including, The communication topology is converted into a circuit diagram in graph theory. A type of node in the circuit diagram represents the data acquisition end or the data processing end. The edges in the circuit diagram represent the communication link between the data acquisition end and the data processing end. The length of the edge is quantified based on the communication link status of wired transmission and wireless transmission. Collect information about interference sources in the actual environment corresponding to the communication topology, mark the interference sources at the corresponding positions in the diagram, and determine the interference intensity of the interference sources based on the interference source information, the distance between the interference source and the data acquisition terminal, and the distance between the interference source and the communication link. Treat the interference sources as Class II nodes in the circuit diagram and mark the interference intensity of each Class II node. The influence amount of each edge in the line graph is determined according to the length of the edge in the line graph, the communication link condition and the interference strength of the second type of nodes, and the influence amount and the communication link condition are used as the communication line information of the edge.

3. The data processing method based on Internet of Things communication according to claim 2, characterized in that: The influence amount of each edge in the circuit diagram is determined according to the length of the edge in the circuit diagram, the communication link status and the interference strength of the second type of nodes, including: The affected range of the edge is determined by the length of the edge in the circuit diagram and the communication link status. The second-class nodes within the affected range are counted and the interference strength of each second-class node is distributed to obtain the interference strength of each second-class node for the edge. The impact amount of each edge is determined by combining the length of the edge in the circuit diagram. ； in, In the circuit diagram The amount of influence on the edge, In the circuit diagram The length of the side, is the initial impact, Indicated by the circuit diagram The initial influence amount obtained by mapping the length of the edge, In the circuit diagram The number of type II nodes within the affected range of the edge, For the The influence weight of the second-class nodes, For the circuit diagram The first edge The interference intensity of the second-class nodes, In the circuit diagram The preset constant corresponding to the edge.

4. The data processing method based on Internet of Things communication according to claim 2, characterized in that: Match the multi-source data with the communication data to obtain the matching relationship between the multi-source data and the communication data, including: A time window is set according to the length of the edges and communication links in the circuit diagram. The corresponding communication data is searched in this time window according to the timestamps and other matching features of the multi-source data to obtain the matching relationship between the multi-source data and the communication data. Other matching features include one or more of the packet size features, IP address features, protocol type features and data content features.

5. The data processing method based on Internet of Things communication according to claim 2, characterized in that: Evaluate the communication transmission quality and multi-source data quality based on the communication line information and the matching relationship between multi-source data and communication data, including: Confirm the attribute information of the multi-source data, and set a reasonable interval of communication data that matches the multi-source data based on the influence amount, communication link status, and the attribute information of the multi-source data; Compare the actual value and reasonable interval of the same type of communication data. When the actual value is within the reasonable interval, the deviation value of this type of communication data is 0. Otherwise, the deviation value of this type of communication data is the difference between the actual value and the reasonable interval. The endpoint of the reasonable interval close to the actual value is regarded as the near endpoint, and the difference between the actual value and the reasonable interval is the difference between the actual value and the near endpoint; The deviation values of each type of communication data are integrated, and the integrated deviation values are used to describe the communication transmission quality; Conduct multi-dimensional analysis and inspection on multi-source data to determine the quality of multi-source data.

6. The data processing method based on Internet of Things communication according to claim 1, characterized in that: Mining the association rules between data collection terminals, including: Acquire the multi-source data previously generated by the data collection end, perform data preprocessing on the multi-source data, analyze the scale characteristics of the multi-source data of each data collection end to determine the liquidity of the multi-source data of each data collection end, set the parameters of the mining algorithm based on the liquidity of the multi-source data of the data collection end, and perform association rule mining on the preprocessed multi-source data to obtain association rules.

7. The data processing method based on Internet of Things communication according to claim 5, characterized in that: The combination weight of each type of multi-source data is assigned by the communication transmission quality and multi-source data quality of the multi-source data. include, Evaluate the encryption security of each type of multi-source data at the data collection end and during transmission, and define the proofreading value of each type of multi-source data based on encryption security, communication transmission quality of multi-source data, and multi-source data quality; ； in, For the The calibration value of multi-source data, 、 Respectively The conversion coefficient corresponding to the communication transmission quality of multi-source data and the conversion coefficient corresponding to the quality of multi-source data, 、 Respectively Communication transmission quality and multi-source data quality of similar multi-source data, For the Encryption security of multi-source data, For the Preset constants corresponding to multi-source data; The calibration values of all multi-source data are normalized, and then the combination weight of each type of multi-source data is assigned according to the calibration values of the multi-source data.

8. The data processing method based on Internet of Things communication according to claim 7, characterized in that: Multi-source data is fused according to the multi-source data association graph and the combination weight of multi-source data to generate a fusion information set, including: According to the multi-source data association graph, the associated multi-source data are weightedly summed according to their respective combination weights to obtain a representative value. The fusion information set includes the associated multi-source data and the representative value.

9. A data processing system based on Internet of Things communication, characterized in that: include, A line module is used to obtain the communication network of the target Internet of Things, determine the data acquisition end and the data processing end in the communication topology of the target Internet of Things, establish a communication topology based on the data acquisition end and the data processing end, and analyze the communication topology to determine the communication line information between the data acquisition end and the data processing end; The matching module is used to obtain multi-source data through the data acquisition terminal, which transmits the multi-source data to the data processing terminal. At the same time, a network monitoring tool is used to capture the communication data during the data transmission process, and the multi-source data is matched with the communication data to obtain a matching relationship between the multi-source data and the communication data. The association module is used to evaluate the communication transmission quality and multi-source data quality based on the communication line information and the matching relationship between multi-source data and communication data, to mine the association rules between data collection terminals, to match multi-source data according to the association rules, and to generate a multi-source data association graph; The analysis module is used to assign the combination weight of each type of multi-source data based on the communication transmission quality and multi-source data quality of the multi-source data, fuse the multi-source data according to the multi-source data association graph and the combination weight of the multi-source data, and generate a fusion information set to help subsequent further data analysis.

Images