Production line multi-source heterogeneous stream data acquisition system and method

Abstract

The invention discloses a production line multi-source heterogeneous stream data acquisition system, and the system comprises a video data preprocessing module which intercepts continuous frames from a video and extracts effective features from the video frames; the time synchronization module is used for uniformly managing timestamps of the multi-source heterogeneous data and identifying a time sequence relationship among the data; the protocol analysis module is used for analyzing the data frame of the communication protocol and carrying out effective data interaction with different devices or systems; the data fusion module is used for integrating the data into a unified data system model; the data storage module is used for collecting, processing and distributing data in real time; and the data display module provides data display, dynamically screens and filters data, and has an alarm function. The problem that data timestamps are inconsistent is effectively solved, all collected data can be compared and analyzed under the unified time reference, diversified data formats from different devices and systems are efficiently processed, and integration and unified management of multi-source heterogeneous data are achieved.

Description

Technical Field

[0001] The present invention relates to the field of data acquisition systems, and particularly to a multi-source heterogeneous flow data acquisition system and method for a production line. Background Art

[0002] Multi-source heterogeneous flow data on a production line refers to a data set that is generated in real time by machine facilities and industrial sensors in multiple sources such as manufacturing execution systems, production monitoring systems, equipment operation and maintenance systems, product quality inspection systems, energy consumption management systems, etc. during the manufacturing process, and has different structures and formats. With the wide application of automation, informatization, and intelligent technologies in the manufacturing industry, these devices and systems continuously generate a large amount of data. Due to the diversity of the data sources and the differences in their structures and formats, the characteristics of multi-source heterogeneous flow data are formed. Since these data are generated in real time, they need to be processed and analyzed in a timely manner to achieve monitoring, optimization, and decision-making support for the production process.

[0003] Currently, during the operation of production lines in various industries, the acquisition and processing of multi-source heterogeneous flow data on the production line face various problems and challenges. First, the diversity of data sources and the non-uniformity of formats lead to a longer analysis cycle and lower efficiency of multi-source heterogeneous flow data analysis, and often involve complex protocol parsing to be compatible with different communication protocols and data frame formats. Second, the real-time requirements for flow data are also very high. It is difficult for a large number of acquisition devices and sensors on the production line to record and process data under a unified time reference, resulting in a large number of data disorders and timing problems caused by time errors. In addition, the massiveness and complexity of heterogeneous flow data also pose severe challenges to storage and management, and efficient data storage technologies and fast data retrieval capabilities are required to support the subsequent analysis and application of data. Finally, during the data fusion process, data from different sources, different formats, and different frequencies will cause a series of problems, including inconsistent data formats, incomplete data, and inaccurate data, etc. Summary of the Invention

[0004] To solve the above technical problems, on the one hand, the present invention provides a multi-source heterogeneous flow data acquisition system for a production line, and the acquisition system includes: A video data preprocessing module, configured to intercept continuous frames from a video and extract effective features from the video frames; A time synchronization module, configured to uniformly manage the timestamps of multi-source heterogeneous data and identify the timing relationships between the data; A protocol parsing module, configured to parse the data frames of communication protocols and perform effective data interaction with different devices or systems; A data fusion module, configured to integrate the data into a unified data system model; A data storage module, configured to perform real-time acquisition, processing, and distribution of data; A data display module, which is used to provide data display, dynamic screening and filtering of data, and has an alarm function.

[0005] Furthermore, the video data preprocessing module specifically includes collecting video surveillance data, processing the collected data, and inputting the processed data into the YOLOv5l model. The processing of the collected data includes annotating the data, dividing the data set, and extracting key data information.

[0006] Furthermore, the time synchronization module, that is, taking the time of the data acquisition computer as the reference for the time of all devices, specifically includes: When the time synchronization starts, the data acquisition computer sends a time synchronization message. The data acquisition computer sends a time synchronization request time T1. After receiving and parsing it, the microcontroller records its own current received request time T2. The microcontroller packs its own response time T3 and sends it to the data acquisition computer. After receiving it, the data acquisition computer records the request response time T4 and sends the request response time T4 to the microcontroller. The time of the microcontroller is corrected according to the time data. The formula is as follows: ; ; Wherein, is the transmission delay time, is the time deviation of the microcontroller.

[0007] Furthermore, the protocol parsing module specifically includes reading the frame protocol configuration file and performing KMP pattern matching with the parsed frame format definition as the input. The process of the KMP pattern matching includes receiving multi-source heterogeneous data streams and intercepting data segments according to the window length, and quickly matching the frame format template by using the KMP algorithm. The KMP algorithm locates the template frame header or format in the data stream through the matching table, determines the starting position of the data frame and parses the field value. If the match is successful, the field content is parsed according to the frame format template and the parsing result is output; if the match fails, the system performs an exception handling mechanism. For the template frames that are not matched or the frames whose field structures do not match the template definition, the system records and marks the abnormal data and gives a prompt.

[0008] Furthermore, the data fusion module specifically includes constructing a data system model and integrating the data into the data system model. The data system model adopts a class management architecture, and each data is composed of member variables and member functions. The formats of all data are unified, and duplicate data removal and anomaly detection are performed.

[0009] Furthermore, the data storage module specifically includes using Kafka stream processing technology to achieve real-time data collection, processing, and distribution; the data storage module also integrates the distributed search and analysis capabilities of Elasticsearch; the data storage module uses time sharding to store data in an orderly manner according to time order; the data storage module incorporates the LZ4 data compression algorithm for data compression.

[0010] Furthermore, the data display module specifically includes dynamically screening and filtering data, displaying data, real-time showing key data, and providing an alarm function.

[0011] In a second aspect, the present invention also provides a method for collecting multi-source heterogeneous stream data in a production line, The method includes, S1: Obtain multi-source target device information, where the information includes video surveillance data, on-site environmental data, analog switch data, and processing equipment process data; S2: Process the collected multi-source target device information; The data processing includes video data preprocessing, time synchronization, protocol parsing, data fusion, data storage, and data display; The video data preprocessing includes intercepting consecutive frames from the video and extracting effective features from the video frames; The time synchronization includes unified management of multi-source heterogeneous data timestamps and identification of the timing relationships between the data; The protocol parsing includes parsing the data frames of the communication protocol and performing effective data interaction with different devices or systems; The data fusion includes integrating the data into a unified data system model; the data system model adopts a class management architecture, and each data consists of member variables and member functions, unifying the format of all data and performing duplicate removal and anomaly detection; The data storage includes real-time data collection, processing, and distribution; The data display includes providing data display, dynamically screening and filtering data, and having an alarm function.

[0012] In a third aspect, the present invention also provides an electronic device, including a memory and a processor, where the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are executed by the processor to implement a method for collecting multi-source heterogeneous stream data in a production line as described above.

[0013] In a fourth aspect, the present invention also provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of a method for collecting multi-source heterogeneous stream data in a production line as described above are implemented.

[0014] Advantages of the present invention: By using modules such as time synchronization, protocol parsing, and data fusion, the present invention effectively solves the problem of inconsistent data timestamps, enabling all collected data to be compared and analyzed under a unified time benchmark, comprehensively and efficiently processing diverse data formats from different devices and systems, adapting to multiple data frame structures, and achieving the integration and unified management of multi-source heterogeneous data. The present invention has high compatibility and real-time performance in the acquisition and processing of multi-source heterogeneous stream data. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is the overall schematic diagram of the multi-source heterogeneous stream data acquisition system for the production line; Figure 2 is the schematic diagram of the model training and deployment process of the video data preprocessing module; Figure 3 is the schematic diagram of the principle of the time synchronization module; Figure 4 is the schematic diagram of the protocol parsing module process; Figure 5 is the schematic diagram of the data system model architecture; Figure 6 is the system architecture diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] To make the objectives, advantages, and features of the present invention more obvious, the following detailed description further elaborates on the present invention.

[0017] To more clearly illustrate the present invention, the following describes the present invention in further detail in conjunction with preferred embodiments and the drawings. Similar components in the drawings are denoted by the same reference numerals. Those skilled in the art should understand that the content specifically described below is illustrative rather than restrictive, and should not be used to limit the protection scope of the present invention.

[0018] The implementation of a multi-source heterogeneous stream data acquisition system and method for a production line includes hardware components, a video data preprocessing module, a time synchronization module, a protocol parsing module, a data fusion module, a data storage module, and a data display module.

[0019] The hardware components include an on-site host computer, a GD32 microcontroller, a switch, a serial server, a video processing computer, a disk array, and a data acquisition computer, which are responsible for real-time data acquisition and transmission, fusing data from different sources with different formats, structures, and characteristics, thereby ensuring that the system can collect and analyze multi-source heterogeneous stream data on the production line in real time; The implementation of the video data preprocessing module is to continuously intercept frames from the video at a preset frequency, aiming to reduce the data volume and shorten the processing time. The core model adopted by this module is the YOLOv5l network architecture, which can efficiently extract key features from video frames. The remote monitoring system on the production line can accurately capture core information such as valve opening, hopper opening and closing status, and equipment operation status.

[0020] The time synchronization module ensures that all acquisition devices record and process data under the same time reference. The time of all devices is based on the time of the data acquisition computer. The time synchronization module will perform time synchronization in various situations, including system startup, continuous system operation for 24 hours, packet delay of a certain device exceeding 0.3 seconds, manual trigger, etc. Through an accurate time synchronization mechanism, this module ensures that the data between the on-site host computer, microcontroller, serial server, and video host computer can be aligned in time, thus realizing the unified management of timestamps of multi-source heterogeneous data and identifying the timing relationship between each data.

[0021] The protocol parsing module designs a dynamic configuration method for data protocols, which can parse data frames of various communication protocols such as MODBUS, CAN, TCP / IP, RS-232, etc., enabling the system to effectively interact with different devices or systems. This module is configured in the form of a frame protocol configuration file, and uses the KMP algorithm to read templates from the frame protocol configuration file and perform fast protocol matching. It has generality and is convenient for later expansion. New protocol parsing rules can be easily added to the configuration file without modifying the core code of the acquisition system, improving the flexibility and maintainability of the system.

[0022] The data fusion module integrates data from different sources, different formats, and different structures into a unified data system model. By constructing a data system model, the data body model adopts a class management architecture, and each data is mainly composed of member variables and member functions. The module will perform operations such as deduplication and anomaly detection to ensure the accuracy and integrity of the collected data stream.

[0023] The data storage module utilizes Kafka stream processing technology to achieve real-time data collection, processing, and distribution, effectively improving the throughput and processing speed of the data stream. The module also deeply integrates the distributed search and analysis capabilities of Elasticsearch, enabling the system to quickly retrieve the required information from massive amounts of data. The module adopts a time sharding strategy to store data in an orderly manner according to time sequence, which not only ensures the timeliness of data but also facilitates subsequent data management and analysis. To optimize storage efficiency and reduce costs, the module also incorporates the LZ4 data compression algorithm, which has a very fast decompression speed and can significantly reduce the storage space occupied by data without affecting real-time stream processing and without changing the data quality.

[0024] The data visualization module mainly provides data display, capable of dynamically screening and filtering data, and presenting data intuitively using diverse charts such as line charts, pie charts, and comparison charts, facilitating trend analysis and comparison analysis of data. In addition, the module displays key data in real time and provides an alarm function to ensure the production safety of the production line.

[0025] As Figure 1 shown in the schematic diagram of the overall solution of the production line multi-source heterogeneous stream data acquisition system, it covers the sources of all multi-source heterogeneous data, the connection relationships between devices, etc.

[0026] The on-site upper computer in the described hardware composition is directly connected to the production line equipment, used to control and monitor the production process, and send the production equipment data and production process data to the data acquisition computer through a switch.

[0027] The GD32 microcontroller in the described hardware composition is used to collect some voltage, current, and digital quantity signals, and send the collected data to the data acquisition computer through an external network port via a switch.

[0028] The serial port server in the described hardware composition aggregates the data of sensors with RS485 interfaces such as temperature sensors, humidity sensors, and static electricity sensors, and sends the collected environmental data to the data acquisition computer through Ethernet via a switch.

[0029] The video processing computer in the described hardware composition is responsible for receiving and storing the video data collected from all the monitoring cameras on the production line, extracting the key features in the video through image feature extraction technology, and sending the extracted video features to the data acquisition computer through a switch.

[0030] The data acquisition computer in the described hardware composition is the center where all data converges. It is equipped with a data acquisition and processing platform, integrating modules such as time synchronization, protocol parsing, data fusion, data storage, and data display, ensuring that the data collected from various sensors and devices can be in a unified format, time-synchronized, data-fused, and securely stored, and at the same time presented to users through an intuitive interface, enabling the data acquisition computer to efficiently and accurately complete the acquisition and processing of multi-source heterogeneous stream data.

[0031] As Figure 2 shown in the schematic diagram of the model training and deployment process of the video data preprocessing module, Example 1: Taking the hopper opening and closing as an example, the key extraction modes for other device operating states are the same. The information video data preprocessing module collects video monitoring data through a camera. The video monitoring data is collected using a high-definition camera from Hikvision to monitor the production situation at various parts of the production line in real time.

[0032] In the described model training process, before the deployment of the YOLOv5l model, video acquisition and data annotation are required, and the dataset is divided. To ensure the generalization ability and stability of the model, the training set accounts for 75% and the test set accounts for 25% to obtain high-quality training data. Through the training and optimization of the YOLOv5l model, its accuracy and real-time performance in actual applications are ensured.

[0033] In the described video image preprocessing, the part of the video stream data that needs to extract key information is scaled and cropped into picture data with corresponding proportions and pixels. The frame rate is 10 frames, and the picture size is 640 640 pixels, and the processed image data is input into the trained YOLOv5l model.

[0034] In the described feature extraction process, 10 features are extracted from each video stream per second, and the mean of the 10 features is used as the final result. The model outputs key information such as valve opening, hopper opening and closing status, and device operating status, and the results are sent to the data acquisition computer through a switch.

[0035] As Figure 3 shown in the schematic diagram of the principle of the time synchronization module, the time of all devices is based on the time of the data acquisition computer. The time synchronization module will perform time synchronization in various situations, including system startup, the system working continuously for 24 hours, the packet delay of a certain device exceeding 0.3 seconds, manual triggering, etc. Through this time synchronization module, real-time problems caused by time errors are effectively avoided.

[0036] The implementation principle of the time synchronization module takes the GD32 microcontroller as the time synchronization object. The synchronization principle for other devices is the same. When time synchronization starts, the data acquisition computer sends time synchronization information, which includes the sending time T1 of the data acquisition computer (time synchronization request time). After receiving and parsing it, the GD32 microcontroller records its own current time T2 (received request time). Then, the GD32 microcontroller packs its own time T3 (response time) and sends it to the data acquisition computer. After receiving it, the data acquisition computer records the time T4 (request response time) and sends the T4 time to the GD32 microcontroller. The time of the GD32 microcontroller is corrected according to the above time data. The formula is as follows: ; ; Among them is the sending delay time, is the time deviation of the GD32 microcontroller. It modifies the device's own time according to the time deviation, effectively reducing data chaos or timing problems that may be caused by time asynchronization.

[0037] Such as Figure 4 As shown in the schematic diagram of the protocol parsing module process, the module supports configuration files in JSON, XML, or text formats by reading the frame protocol configuration file. The content includes information such as frame header definition, frame type definition, field structure, offset, field length, and check rule. For example, a JSON-format configuration file may contain a frame header (such as "AA55"), field names and lengths (such as the timestamp field occupies 8 bytes), and check rules. The parsed frame format definition will be used as input for subsequent generation of the partial match table.

[0038] For the partial match table in the protocol parsing module, the construction of the partial match table utilizes the prefix (such as frame header, source address, or destination address, etc.) and suffix (such as frame tail, check bit, etc.) characteristics of the frame format template string to avoid repeated comparison of characters when matching fails. By preprocessing the template string, a partial match table is generated and stored on the hard disk to improve subsequent matching efficiency.

[0039] For the KMP matching process in the protocol parsing module, it receives multi-source heterogeneous data streams and intercepts data segments according to a fixed window length, and uses the KMP algorithm to quickly match the frame format template. The algorithm locates the template frame header or format in the data stream through the partial match table, thereby determining the starting position of the data frame and parsing the field values. After successful matching, it parses the field content according to the frame format template, such as extracting the timestamp, data ID, and data value, etc., and verifies the checksum field to ensure the correctness of the data.

[0040] The described matching rule fails. To ensure the reliability of parsing, the system designs an exception handling mechanism. For template frames that are not matched or frames whose field structures do not conform to the template definition, the system will record and mark the abnormal data and trigger a prompt.

[0041] As Figure 5 shown in the schematic diagram of the data system model architecture, the data system model adopts a class management architecture. Each data is mainly composed of member variables and member functions. The instantiated objects of all acquisition measurement points on the production line inherit from the class of the data system model. The member variables include data source ID, upper threshold, lower threshold, acquisition frequency, minimum value, maximum value, mean value, variance, received data volume, data status, data integrity, and data. The member functions include maximum value calculation, minimum value calculation, mean value calculation, variance calculation, received data volume calculation, data status verification, and data integrity verification.

[0042] The described data system model can compress data streams with different frequencies to a fixed frequency and calculate data status, data integrity, etc. After being processed by this data system model, the originally scattered and differently formatted multi-source heterogeneous data streams are integrated and stored together in a single database file. For the original records of high-frequency data streams, they are classified and stored according to the frequency characteristics of the data.

[0043] Example 2: An electronic device includes a memory and a processor. The memory is used to store one or more computer instructions. Among them, the one or more computer instructions are executed by the processor to implement the method of interfering with a network scanner described above.

[0044] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the electronic device described above can refer to the corresponding process in the foregoing method embodiment and will not be elaborated here.

[0045] Example 3: A computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the steps of the method in Example 1 are implemented.

[0046] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, devices, or computer program products. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0047] The present invention is described with reference to the flowcharts and / or block diagrams of methods, terminal devices (systems), and computer program products according to the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate means for implementing the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0048] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0049] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0050] The above embodiments have described the technical solutions of the present invention in detail. Obviously, the present invention is not limited to the described embodiments. Based on the embodiments of the present invention, those skilled in the art can also make various changes accordingly, but any changes equivalent or similar to the present invention fall within the scope of protection of the present invention.

[0051] The content not described in detail in this specification belongs to the prior art well-known to those skilled in the art.

Claims

1. A multi-source heterogeneous flow data acquisition system for a production line, characterized in that, The acquisition system includes: A video data preprocessing module, which is used to intercept continuous frames from a video and extract effective features from the video frames; A time synchronization module, which is used for unified management of the timestamps of multi-source heterogeneous data and identification of the timing relationships between various data; A protocol parsing module, which is used to parse the data frames of the communication protocol and perform effective data interaction with different devices or systems; A data fusion module, which is used to integrate data into a unified data system model; A data storage module, which is used for real-time acquisition, processing, and distribution of data; A data display module, which is used to provide data display, dynamic screening, and filtering of data, and has an alarm function.

2. The multi-source heterogeneous flow data acquisition system for a production line according to claim 1, wherein The video data preprocessing module specifically includes collecting video surveillance data; processing the collected data; and inputting the processed data into the YOLOv5l model; The processing of the collected data includes annotating the data, dividing the data set, and extracting key data information.

3. The multi-source heterogeneous flow data acquisition system for a production line according to claim 2, wherein The time synchronization module, that is, taking the time of the data acquisition computer as the reference for the time of all devices, specifically includes, At the beginning of time synchronization, the data acquisition computer sends a time synchronization message. The data acquisition computer sends a time synchronization request time T1. After receiving and parsing it, the microcontroller records its own current received request time T2. The microcontroller packs its own response time T3 and sends it to the data acquisition computer. After receiving it, the data acquisition computer records the request response time T4 and sends the request response time T4 to the microcontroller. The time of the microcontroller is corrected according to the time data. The formula is as follows: ； ； Wherein, is the transmission delay time, is the microcontroller time deviation.

4. The multi-source heterogeneous flow data acquisition system for a production line according to claim 3, wherein The protocol parsing module specifically includes reading the frame protocol configuration file and using the parsed frame format definition as input for KMP pattern matching; The process of KMP pattern matching includes receiving multi-source heterogeneous data streams and intercepting data segments according to the window length, and using the KMP algorithm to quickly match the frame format template. The KMP algorithm locates the template frame header or format in the data stream through the matching table, determines the starting position of the data frame and parses the field value. If the match is successful, the field content is parsed according to the frame format template and the parsing result is output; If the match fails, the system performs an exception handling mechanism. For the template frames that are not matched or the frames whose field structures do not match the template definition, the system records and marks the abnormal data and gives a prompt.

5. The multi-source heterogeneous flow data acquisition system for a production line according to claim 4, wherein The data fusion module specifically includes constructing a data system model and integrating data into the data system model; the data system model adopts a class management architecture, and each data is composed of member variables and member functions. The formats of all data are unified, and duplicate data removal and anomaly detection are performed.

6. The multi-source heterogeneous flow data acquisition system for a production line according to claim 5, characterized in that, The data storage module specifically includes using Kafka stream processing technology to achieve real-time acquisition, processing, and distribution of data; the data storage module also integrates the distributed search and analysis capabilities of Elasticsearch; the data storage module uses time sharding to store data in an orderly manner according to time order; the data storage module incorporates the LZ4 data compression algorithm for data compression.

7. The multi-source heterogeneous flow data acquisition system for a production line according to claim 6, characterized in that, The data display module specifically includes dynamically screening and filtering data, displaying data, real-time presenting key data, and providing an alarm function.

8. A method for collecting multi-source heterogeneous flow data in a production line, characterized in that, The method includes: S1: Obtain multi-source target device information, where the information includes video surveillance data, on-site environmental data, analog switch quantity data, and processing equipment process data; S2: Perform data processing on the collected multi-source target device information; The data processing includes video data preprocessing, time synchronization, protocol parsing, data fusion, data storage, and data display; The video data preprocessing includes intercepting consecutive frames from the video and extracting effective features from the video frames; The time synchronization includes unified management of multi-source heterogeneous data timestamps and identifying the timing relationships between various data; The protocol parsing includes parsing the data frames of the communication protocol and performing effective data interaction with different devices or systems; The data fusion includes integrating the data into a unified data system model; the data system model adopts a class management architecture, and each data is composed of member variables and member functions, unifying the format of all data and performing duplicate removal and anomaly detection; The data storage includes real-time collection, processing, and distribution of data; The data display includes providing data display, dynamically screening and filtering data, and having an alarm function.

9. An electronic device, characterized in that, It includes a memory and a processor. The memory is used to store one or more computer instructions. Among them, the one or more computer instructions are executed by the processor to implement a method for collecting multi-source heterogeneous flow data in a production line as described in claim 8.

10. A readable storage medium, on which a computer program is stored, characterized in that: When the computer program is executed by a processor, it implements the steps of a method for collecting multi-source heterogeneous flow data in a production line as described in claim 8.

Images