Data processing system for multi-protocol data acquisition and data write-back

Abstract

The invention discloses a data processing system for multi-protocol data acquisition and data write-back, and belongs to the technical field of automation. Comprising a data acquisition unit, a communication unit and a database unit, the data acquisition unit comprises an interface module, a driving module, a protocol conversion module and a protocol library; the protocol conversion module obtains the head, the effective load, the tail and the selectable field of various protocol data, performs feature extraction on the content in the effective load, and maps a source address, a timestamp and features into a JSON format; and the data write-back module converts the JSON format into a data vector. The data information is subjected to protocol conversion processing, features in the data information are extracted, the data information is stored in combination with the write-back module of the database unit, protocol unification of the data information is achieved, the processing and storage efficiency of the data information is improved, and the write-back efficiency of the data information is improved.

Description

Technical Field

[0001] The present invention belongs to the field of automation technology, and more specifically, particularly relates to a data processing system for multi-protocol data acquisition and data write-back. At the same time, the present invention also relates to a 111. Background Art

[0002] Industrial control computers can summarize, analyze and organize various information collected by sensors and local area networks to achieve the integration of information management and automatic control, and ensure information security through authority authentication.

[0003] After a long period of continuous development, the industrial production process automation system has made great progress, especially on the basis of making full use of computer technology. It has played an important role in the production process and has become an indispensable tool for the safe, stable and automated operation of the production process.

[0004] Industrial automation is the control of various parameters in industrial production to achieve various process controls. In the entire industrial production, human operations are minimized and energy other than animals and various information are fully utilized to carry out production work. This is called industrial automated production.

[0005] Industrial automatic control systems integrate information management and automatic control by summarizing, analyzing and organizing various information collected by sensors and local area networks through industrial control computers, and ensure information security through authority authentication.

[0006] In today's industrial field, efficient operation and management of factory equipment is crucial. Realizing data collection of common equipment in the factory is the basis of factory automation control, and data collection can bring many benefits to the factory. First of all, by obtaining data such as the operating status and production parameters of the equipment in real time, equipment failures and abnormal conditions can be discovered in time, maintenance and repairs can be carried out in advance, downtime can be reduced, and production efficiency can be improved. Secondly, data collection provides a basis for optimizing the production process. Through the analysis of data, bottlenecks in production can be found, targeted improvements can be made, and product quality and production efficiency can be improved. In addition, data collection also provides support for the intelligent management of enterprises, realizing remote monitoring and management of equipment, and improving management efficiency and scientific decision-making.

[0007] However, there are some problems with the existing technology: there are many devices in the existing factories, and the protocols of each device are different, which causes great troubles for data collection and makes it impossible to unify data information, resulting in too high a delay in data collection and inability to achieve fast automated control. It also causes great troubles for data storage and indexing. Therefore, we propose a data processing system for multi-protocol data collection and data writeback. Summary of the invention

[0008] In view of the problems existing in the prior art, the purpose of the present invention is to provide a data processing system for multi-protocol data acquisition and data writeback, which performs protocol conversion processing on data information through a data acquisition unit, extracts features in the data information, and collects and stores the data information in combination with the write-back module of the database unit, thereby achieving protocol unification of data information, improving the processing and storage efficiency of data information, and improving the write-back efficiency of data information.

[0009] To achieve the above-mentioned object, the present invention provides the following technical solutions: a data processing system for multi-protocol data acquisition and data write-back, comprising a data acquisition unit, a communication unit and a database unit; The data acquisition unit is used to connect to sensors or controllers on various devices in the factory to collect data information from various devices and to remotely control and adjust the devices. The data acquisition unit includes an interface module, which is responsible for communicating with sensors or controllers on various devices and is provided with a variety of digital input and output interfaces and analog input and output interfaces. The data acquisition unit includes a driver module, which is used to drive the corresponding running program when receiving a variety of protocol data to complete the acquisition of a variety of protocol data. The data acquisition unit includes a protocol conversion module, which is used to convert a variety of protocol data into the same protocol format. The data acquisition unit includes a protocol library, which contains parsing and encapsulation functions of various protocols for parsing and encapsulating a variety of protocol data; After parsing the data information of multiple protocols, the protocol conversion module obtains the header, payload, tail and optional fields of the multiple protocol data, extracts features from the content in the payload, extracts the source address in the header and the timestamp in the optional field, and then maps the source address, timestamp and features extracted from the header, optional field and payload to the JSON format to complete the conversion of the multiple protocol data; The communication unit is used to establish a communication connection between the data acquisition unit and the database unit, and is used to send the data information after the protocol conversion to the database unit; The database unit is used to store the data information after the protocol conversion, and the database unit includes a data write-back module, and the data write-back module is used to write the data information after the transmission protocol conversion into the database, and the database unit includes a write-back logic module and a monitoring module, and the write-back logic module is used to regulate the written data information by direct writing, batch writing or asynchronous writing, and the monitoring module is used to detect the write-back operation, record the successful and failed operations, and trigger the alarm mechanism when the writing fails; The data write-back module is used to convert the data information in JSON format into a data vector in vector form, and write the data vector into the database.

[0010] Specifically, the multi-protocol interfaces on the interface module are electrically connected to interface expanders with unified interfaces, and the interface expanders are used to electrically connect multiple sensors or controllers of the same protocol. The multi-protocol interfaces on the interface module are electrically connected to the corresponding driver modules, and the driver modules receive and identify data information by driving the running program.

[0011] Specifically, the steps of the protocol conversion module are as follows: The data acquisition unit receives data information from sensors or controllers on various devices through the interface module; The driver module starts the running program to receive and identify the transmitted data information, and parses and processes the data information through the parsing function in the protocol library; The protocol conversion module extracts the header, payload, tail and optional fields of the parsed data information; The protocol conversion module generates a converted protocol format by mapping the source protocol to the target protocol and converting the corresponding source addresses, timestamps and features.

[0012] Specifically, the protocol conversion module extracts content features in the payload as follows: First, the parsed payload is filtered and cleaned to remove noise and dryness in the payload, and the outliers and missing values ​​are mean-filled; Then, feature extraction is performed on the filtered and cleaned payload to obtain features in the payload and remove irrelevant data; Then the extracted features are processed by binary encoding to generate binary data information; Finally, a name-value pairing relationship is established between the binary data information and the payload.

[0013] Specifically, the protocol conversion module maps the source address, timestamp and features to the JSON format by using field mapping, which corresponds the fields in the source protocol to the fields in the target protocol, and then maps the source address, timestamp and payload to the JSON format. When performing field mapping, the source address, timestamp and payload are first converted to a unified format type.

[0014] Specifically, the data write-back module converts the data information in JSON format into data information in vector form; First, the data write-back module parses and processes the data information in JSON format; Extract features in JSON format, including source address, timestamp, and payload; Convert the extracted source address, timestamp, and payload into a data vector; And the source address is calculated through a hash function to obtain a foreign key, and the source address is linked to the data vector through the foreign key.

[0015] Specifically, the direct writing is used to directly write the data vector into the data using an insert or update operation of the database; The batch write is used to merge multiple data vector entries into one batch processing request, and then write all the data vectors into the database; The asynchronous writing is used to put the data vector into the message queue and asynchronously write the data vector into the database; The direct write is used to write a data message of a single protocol into a database after conversion, the batch write is used to write multiple data messages of a single protocol into a database after conversion, and the asynchronous write is used to write multiple data messages of multiple protocols into a database after conversion. The asynchronous write adopts bubble sort and is written into the database in a first-in-first-out sorting manner.

[0016] Specifically, the database unit also includes a cache module, which is used to map the vector data in the cache module for caching before storing the vector data in the database through the direct writing, the batch writing or the asynchronous writing, and then write the vector data into the database, and detect and record the success or failure of writing the vector data into the database through the monitoring module. When the vector data is successfully written into the database, the cache record in the cache module is deleted. When the vector data fails to be written into the database, the cache record in the cache module is pulled up again for storage. After three repeated storage failures, an alarm mechanism is triggered.

[0017] Specifically, the communication unit is also connected to a remote terminal, and the remote terminal remotely controls each device through the communication unit, and the remote terminal inquires and operates the database unit through the communication unit.

[0018] Specifically, when the remote terminal remotely controls the device through the communication unit, the control instructions issued by the remote terminal are transmitted to the protocol conversion module in the data acquisition unit through the communication unit. The protocol conversion module parses and processes through the parsing function in the protocol library, obtains the destination address in the header, and performs protocol conversion according to the protocol of the target device of the destination address, and then encapsulates it through the encapsulation function, and finally transmits the control instructions to the target device.

[0019] Technical effects and advantages of the present invention: The present invention collects data information of various devices in the factory through a data acquisition unit, and the collected data information converts multiple protocols into a unified protocol format through a protocol conversion module, so as to facilitate the data information; After parsing the data information of the protocol, the protocol conversion module extracts some fields or features from the header, payload, tail and optional fields in the data information, removes useless information from the data information, improves the protocol conversion efficiency of the data information, and improves the efficiency of subsequent data information stored in the database. The extracted parts and features are mapped to the JSON format to complete the conversion process of multiple protocols into a unified protocol. The JSON format is convenient for viewing the data information content and for storage; The database unit is used to store the data information of the conversion protocol, and the data write-back module converts the data information in JSON format into a data vector in vector form, and writes the data vector into the database. By converting the data information into a data vector, it is convenient to improve the efficiency of subsequent query and viewing of the data information, and it is clear and concise. When writing back, it realizes the control of direct writing, batch writing or asynchronous writing. The monitoring module detects the write-back operation, records the successful and failed operations, and triggers the alarm mechanism when the write fails, so as to realize the monitoring operation of the write-back and record the data for subsequent tracking and auditing, and to provide data support for maintenance.

[0020] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the system structure provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, the data processing system for multi-protocol data collection and data write-back provided by the embodiment of the present invention includes a data collection unit, a communication unit and a database unit; The data acquisition unit is used to connect to sensors or controllers on various devices in the factory to collect data information from various devices and to remotely control and adjust the devices. The data acquisition unit includes an interface module, which is responsible for communicating with sensors or controllers on various devices and is provided with a variety of digital input and output interfaces and analog input and output interfaces. The data acquisition unit includes a driver module, which is used to drive the corresponding running program when receiving multiple protocol data to complete the acquisition of multiple protocol data. The data acquisition unit includes a protocol conversion module, which is used to convert multiple protocol data into the same protocol format. The data acquisition unit includes a protocol library, which contains parsing and encapsulation functions of various protocols for parsing and encapsulating multiple protocol data; After parsing the data information of multiple protocols, the protocol conversion module obtains the header, payload, tail and optional fields of the multiple protocol data, extracts features from the payload, extracts the source address from the header and the timestamp from the optional field, and then maps the source address, timestamp and features extracted from the header, optional field and payload to the JSON format, completing the conversion of multiple protocol data; The communication unit is used to establish a communication connection between the data acquisition unit and the database unit, and is used to send the data information after the protocol conversion to the database unit; The database unit is used to store the data information after the protocol conversion. The database unit includes a data write-back module. The data write-back module is used to write the data information after the transmission protocol conversion into the database. The database unit includes a write-back logic module and a monitoring module. The write-back logic module is used to control the written data information by direct writing, batch writing or asynchronous writing. The monitoring module is used to detect the write-back operation, record the successful and failed operations, and trigger the alarm mechanism when the write fails. The data write-back module is used to convert data information in JSON format into data vectors in vector form and write the data vectors into the database.

[0024] In this embodiment, preferably, the multi-protocol interfaces on the interface module are respectively electrically connected to an interface expander with a unified interface, the interface expander is used to electrically connect multiple sensors or controllers of the same protocol, and the multi-protocol interfaces on the interface module are respectively electrically connected to corresponding drive modules, and the drive modules receive and identify data information by driving the running program; It should be noted that, through the interface expander of the unified interface, it is convenient to provide more interfaces of the unified type of protocol, to facilitate the connection of multiple devices in the factory, and to realize the reception and identification of data information.

[0025] In this embodiment, preferably, the steps of the protocol conversion module are as follows: The data acquisition unit receives data information from sensors or controllers on various devices through the interface module; The driver module starts the running program to receive and identify the transmitted data information, and parses and processes the data information through the parsing function in the protocol library; The protocol conversion module extracts the header, payload, tail and optional fields of the parsed data information; The protocol conversion module generates the converted protocol format by mapping the source protocol to the target protocol and the corresponding source address, timestamp and characteristics; It should be noted that the data acquisition unit collects data information, receives and identifies the data information, facilitates the parsing of the data information, obtains the header, payload, tail and optional fields of the data information, facilitates the corresponding format conversion, and completes the protocol conversion processing of the data information.

[0026] In this embodiment, preferably, the protocol conversion module extracts content features in the payload as follows: First, the parsed payload is filtered and cleaned to remove noise and dryness in the payload, and the outliers and missing values ​​are mean-filled; Then, feature extraction is performed on the filtered and cleaned payload to obtain features in the payload and remove irrelevant data; Then the extracted features are processed by binary encoding to generate binary data information; Finally, the binary data information and the payload are paired to establish a name-value relationship; It should be noted that in order to reduce the string size of the payload, the payload is feature extracted to obtain useful data information in the payload, and irrelevant data information is deleted to improve the accuracy of the payload and reduce the space occupied. The features are converted into binary codes to facilitate protocol conversion processing, and a key pair relationship between name and value is established to facilitate one-dimensional vector storage of data. The calculation for filtering the payload is as follows: , in, Represented as a time point in the data information The corresponding original data, Represented as a time point in the data information The corresponding smoothing value is is a smoothing factor used to assign weights to new data points and historical data points, and Indicates data information in scale and location The transformation structure on is the conjugate of the wavelet transform, is the scale parameter, is the translation parameter; The calculation for cleaning the payload is as follows: Cleaning is detected using the following algorithm: , in, It is expressed as the calculation value of data information cleaning. Indicates data information. Represented as the mean of the data group in the data information, It is expressed as the standard deviation of the data group in the data information, and the judgment of abnormality is , then the data point is an outlier; After the data information is cleaned for anomaly detection, the data information is normalized and mapped between [0,1]. The normalization formula is as follows: , in, Represented as the minimum value in the data information, Represented as the maximum value in the data information, Represents data information; For the replacement of abnormal data, the average value is used for replacement: , in, is the amount of non-anomalous data.

[0027] In this embodiment, preferably, the protocol conversion module maps the source address, timestamp and feature to the JSON format by field mapping, which corresponds the fields in the source protocol to the fields in the target protocol, and then performs mapping, mapping the source address, timestamp and payload to the JSON format, and when performing field mapping, first performs type conversion on the source address, timestamp and payload to form a unified format type; It should be noted that in order to quickly complete the protocol conversion process and achieve complete conversion of data information, field mapping will be used to complete the conversion of the JSON format and keep the format of the fields of the protocol conversion unified, so as to improve the efficiency of successful protocol conversion and facilitate the success rate of storage and query.

[0028] In this embodiment, preferably, the data write-back module converts the data information in JSON format into data information in vector form; First, the data write-back module parses and processes the data information in JSON format; Extract features in JSON format, including source address, timestamp, and payload; Convert the extracted source address, timestamp, and payload into a data vector; And the source address is calculated through a hash function to obtain a foreign key, and the source address is linked to the data vector through the foreign key; It should be noted that the source address is used to generate a foreign key through a hash function, which makes it easy to connect the source address with the data vector through the foreign key, so that the data vector is easy to store and query, and the setting of the source address makes it easy to connect the data vector through the foreign key after indexing, so as to realize fast query of the data vector.

[0029] In this embodiment, preferably, direct writing is used to directly write the data vector into the data using an insert or update operation of the database; Batch write is used to merge multiple data vector entries into one batch request and then write all data vectors to the database; Asynchronous writing is used to put data vectors into the message queue and asynchronously write data vectors to the database; Direct writing is used to convert a piece of data information of a single protocol and write it into the database. Batch writing is used to convert multiple pieces of data information of a single protocol and write them into the database. Asynchronous writing is used to convert multiple pieces of data information of multiple protocols and write them into the database. Asynchronous writing uses bubble sorting and writes them into the database in a first-in-first-out manner. It should be noted that in order to improve the storage efficiency of data vectors, selective storage is performed according to the protocol conversion of data vectors, and bubble sorting is used for asynchronous writing to facilitate sequential writing of vector data.

[0030] In this embodiment, preferably, the database unit also includes a cache module, which is used to map the vector data in the cache module for caching before storing the vector data in the database by direct writing, batch writing or asynchronous writing, and then write the vector data into the database, and detect and record the success or failure of writing the vector data into the database through the monitoring module. When the vector data is successfully written into the database, the cache record in the cache module is deleted. When the vector data fails to be written into the database, the cache record in the cache module is pulled up again for storage. After three repeated storage failures, an alarm mechanism is triggered; It should be noted that in order to prevent the loss of vector data after a write failure, the vector data cannot be written and stored again. If the storage fails after three consecutive write operations, the alarm mechanism is triggered and the failure information is stored in the log to facilitate tracking and auditing, and to provide data support for maintenance.

[0031] In this embodiment, preferably, the communication unit is also connected to a remote terminal, and the remote terminal remotely controls each device through the communication unit, and the remote terminal queries and operates the database unit through the communication unit; It should be noted that the communication unit is connected to the remote terminal to facilitate the control of the storage of data information, the remote control of the device, and the viewing of the written-back data information in the database.

[0032] In this embodiment, preferably, when the remote terminal remotely controls the device through the communication unit, the control instruction issued by the remote terminal is transmitted to the protocol conversion module in the data acquisition unit through the communication unit. The protocol conversion module performs parsing processing through the parsing function in the protocol library, obtains the destination address in the header, and performs protocol conversion according to the protocol of the target device of the destination address, and then encapsulates it through the encapsulation function, and finally transmits the control instruction to the target device; When necessary, the control instructions of the remote terminal are converted into a protocol and encapsulated so that the control instructions of the remote terminal can adapt to the protocols of the equipment, sensors and controllers, so that the equipment, sensors and controllers can parse the control instructions and complete the control adjustment.

[0033] The specific steps of this application are: The data acquisition unit is used to connect sensors or controllers on various devices in the factory to collect data information and remotely control and adjust the devices. The data acquisition unit includes an interface module, which communicates with sensors or controllers on various devices through the interface module. The data acquisition unit includes a driver module, which is used to drive the corresponding running program when receiving multiple protocol data to complete the acquisition of multiple protocol data. The data acquisition unit includes a protocol conversion module, which is used to convert multiple protocol data into the same protocol format. The data acquisition unit includes a protocol library, which contains parsing and encapsulation functions of various protocols for parsing and encapsulating multiple protocol data. After parsing the data information of multiple protocols, the protocol conversion module obtains the header, payload, tail and optional fields of the multiple protocol data, extracts features from the content in the payload, extracts the source address in the header and the timestamp in the optional field, and then maps the source address, timestamp and features extracted from the header, optional field and payload to the JSON format to complete the conversion of multiple protocol data. The data acquisition unit receives data information from sensors or controllers on various devices through the interface module; The driver module starts the running program to receive and identify the transmitted data information, and parses and processes the data information through the parsing function in the protocol library; The protocol conversion module extracts the header, payload, tail and optional fields of the parsed data information; The protocol conversion module maps the source protocol to the target protocol, and generates the converted protocol format with the corresponding source address, timestamp and features; the data acquisition unit collects the data information, receives and identifies the data information, facilitates the parsing of the data information, obtains the header, payload, tail and optional fields of the data information, facilitates the corresponding format conversion, and completes the protocol conversion processing of the data information.

[0034] The protocol conversion module extracts the content features in the payload as follows: First, the parsed payload is filtered and cleaned to remove noise and dryness in the payload, and the outliers and missing values ​​are mean-filled; Then, feature extraction is performed on the filtered and cleaned payload to obtain features in the payload and remove irrelevant data; Then the extracted features are processed by binary encoding to generate binary data information; Finally, the binary data information and the payload are connected to establish a name-value pair relationship; in order to reduce the string size of the payload, the payload is feature extracted to obtain useful data information in the payload, and irrelevant data information is deleted to improve the accuracy of the payload and reduce the space occupied, and the features are converted into binary codes to facilitate protocol conversion processing, and a name-value pair relationship is established to facilitate one-dimensional vector storage of data; The calculation for filtering the payload is as follows: , in, Represented as a time point in the data information The corresponding original data, Represented as a time point in the data information The corresponding smoothing value is is a smoothing factor used to assign weights to new data points and historical data points, and Indicates data information in scale and location The transformation structure on is the conjugate of the wavelet transform, is the scale parameter, is the translation parameter; The calculation for cleaning the payload is as follows: Cleaning is detected using the following algorithm: , in, It is expressed as the calculation value of data information cleaning. Indicates data information. It is expressed as the mean of the data group in the data information, It is expressed as the standard deviation of the data group in the data information, and the judgment of abnormality is , then the data point is an outlier; After the data information is cleaned for anomaly detection, the data information is normalized and mapped between [0,1]. The normalization formula is as follows: , in, Represented as the minimum value in the data information, Represented as the maximum value in the data information, Represents data information; For the replacement of abnormal data, the average value is used for replacement: , in, is the amount of non-abnormal data; The database unit is used to store the data information after the protocol conversion. The database unit includes a data write-back module. The data write-back module is used to write the data information after the transmission protocol conversion into the database. The database unit includes a write-back logic module and a monitoring module. The write-back logic module is used to control the written data information by direct writing, batch writing or asynchronous writing. The monitoring module is used to detect the write-back operation, record the successful and failed operations, and trigger the alarm mechanism when the writing fails; the data write-back module is used to convert the data information in JSON format into a data vector in vector form, and write the data vector into the database; the data write-back module converts the data information in JSON format into data information in vector form; First, the data write-back module parses and processes the data information in JSON format; Extract features in JSON format, including source address, timestamp, and payload; Convert the extracted source address, timestamp, and payload into a data vector; The source address is calculated by a hash function to obtain a foreign key, and the source address is connected to the data vector through the foreign key; the source address is generated by a hash function to facilitate the connection between the source address and the data vector through the foreign key, so that the data vector is easy to store and query, and the setting of the source address facilitates the connection of the data vector through the foreign key after indexing, so as to realize fast query of the data vector; In the protocol conversion module, field mapping is used to map the source address, timestamp and features to the JSON format. The fields in the source protocol are matched with the fields in the target protocol, and then mapped to map the source address, timestamp and payload to the JSON format. When performing field mapping, the source address, timestamp and payload are first converted to form a unified format type. In order to quickly complete the protocol conversion process and achieve complete conversion of the data information, field mapping is used to complete the conversion process of the JSON format, and the format of the fields of the protocol conversion is kept unified, so as to improve the efficiency of the successful protocol conversion and facilitate the success rate of storage and query.

[0035] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data processing system for multi-protocol data collection and data write-back, characterized in that: It includes a data acquisition unit, a communication unit and a database unit; The data acquisition unit is used to connect to sensors or controllers on various devices in the factory to collect data information from various devices and to remotely control and adjust the devices. The data acquisition unit includes an interface module, which is responsible for communicating with sensors or controllers on various devices and is provided with a variety of digital input and output interfaces and analog input and output interfaces. The data acquisition unit includes a driver module, which is used to drive the corresponding running program when receiving a variety of protocol data to complete the acquisition of a variety of protocol data. The data acquisition unit includes a protocol conversion module, which is used to convert a variety of protocol data into the same protocol format. The data acquisition unit includes a protocol library, which contains parsing and encapsulation functions of various protocols for parsing and encapsulating a variety of protocol data; After parsing the data information of multiple protocols, the protocol conversion module obtains the header, payload, tail and optional fields of the multiple protocol data, extracts features from the content in the payload, extracts the source address in the header and the timestamp in the optional field, and then maps the source address, timestamp and features extracted from the header, optional field and payload to the JSON format to complete the conversion of the multiple protocol data; The communication unit is used to establish a communication connection between the data acquisition unit and the database unit, and is used to send the data information after the protocol conversion to the database unit; The database unit is used to store the data information after the protocol conversion, and the database unit includes a data write-back module, and the data write-back module is used to write the data information after the transmission protocol conversion into the database, and the database unit includes a write-back logic module and a monitoring module, and the write-back logic module is used to regulate the written data information by direct writing, batch writing or asynchronous writing, and the monitoring module is used to detect the write-back operation, record the successful and failed operations, and trigger the alarm mechanism when the writing fails; The data write-back module is used to convert the data information in JSON format into a data vector in vector form, and write the data vector into the database.

2. The data processing system for multi-protocol data acquisition and data write-back according to claim 1, characterized in that: The multi-protocol interfaces on the interface module are respectively electrically connected to an interface expander with a unified interface, and the interface expander is used to electrically connect multiple sensors or controllers of the same protocol. The multi-protocol interfaces on the interface module are respectively electrically connected to the corresponding driving modules, and the driving modules receive and identify data information by driving the running program.

3. The data processing system for multi-protocol data acquisition and data write-back according to claim 1, characterized in that: The steps of the protocol conversion module are as follows: The data acquisition unit receives data information from sensors or controllers on various devices through the interface module; The driver module starts the running program to receive and identify the transmitted data information, and parses and processes the data information through the parsing function in the protocol library; The protocol conversion module extracts the header, payload, tail and optional fields of the parsed data information; The protocol conversion module generates a converted protocol format by mapping the source protocol to the target protocol and converting the corresponding source addresses, timestamps and features.

4. The data processing system for multi-protocol data acquisition and data write-back according to claim 1, characterized in that: The protocol conversion module extracts content features in the payload as follows: First, the parsed payload is filtered and cleaned to remove noise and dryness in the payload, and the outliers and missing values ​​are mean-filled; Then, feature extraction is performed on the filtered and cleaned payload to obtain features in the payload and remove irrelevant data; Then the extracted features are processed by binary encoding to generate binary data information; Finally, a name-value pairing relationship is established between the binary data information and the payload.

5. The data processing system for multi-protocol data acquisition and data write-back according to claim 1, characterized in that: The protocol conversion module maps the source address, timestamp and features to the JSON format using field mapping, which corresponds the fields in the source protocol to the fields in the target protocol, and then maps the source address, timestamp and payload to the JSON format. When performing field mapping, the source address, timestamp and payload are first converted to a unified format type.

6. The data processing system for multi-protocol data collection and data write-back according to claim 1, characterized in that: The data write-back module converts the data information in JSON format into data information in vector form; First, the data write-back module parses and processes the data information in JSON format; Extract features in JSON format, including source address, timestamp, and payload; Convert the extracted source address, timestamp, and payload into a data vector; And the source address is calculated through a hash function to obtain a foreign key, and the source address is linked to the data vector through the foreign key.

7. The data processing system for multi-protocol data collection and data write-back according to claim 1, characterized in that: The direct writing is used to directly write the data vector into the data using the insert or update operation of the database; The batch write is used to merge multiple data vector entries into one batch processing request, and then write all the data vectors into the database; The asynchronous writing is used to put the data vector into the message queue and asynchronously write the data vector into the database; The direct write is used to write a data message of a single protocol into a database after conversion, the batch write is used to write multiple data messages of a single protocol into a database after conversion, and the asynchronous write is used to write multiple data messages of multiple protocols into a database after conversion. The asynchronous write adopts bubble sort and is written into the database in a first-in-first-out sorting manner.

8. The data processing system for multi-protocol data collection and data write-back according to claim 1, characterized in that:The database unit also includes a cache module, which is used to map the vector data in the cache module for caching before storing the vector data in the database through the direct writing, the batch writing or the asynchronous writing, and then write the vector data into the database, and detect and record the success or failure of writing the vector data into the database through the monitoring module. When the vector data is successfully written into the database, the cache record in the cache module is deleted. When the vector data fails to be written into the database, the cache record in the cache module is pulled up again for storage. After three repeated storage failures, an alarm mechanism is triggered.

9. The data processing system for multi-protocol data collection and data write-back according to claim 1, characterized in that: The communication unit is also connected to a remote terminal, and the remote terminal remotely controls each device through the communication unit, and the remote terminal inquires and operates the database unit through the communication unit.

10. The data processing system for multi-protocol data collection and data write-back according to claim 9, characterized in that: When the remote terminal remotely controls the device through the communication unit, the control instructions issued by the remote terminal are transmitted to the protocol conversion module in the data acquisition unit through the communication unit. The protocol conversion module performs parsing through the parsing function in the protocol library, obtains the destination address in the header, and performs protocol conversion according to the protocol of the target device of the destination address, and then encapsulates it through the encapsulation function, and finally transmits the control instructions to the target device.

Images