Factory data acquisition method based on automatic adaptation of multiple protocols

Abstract

This invention relates to the field of industrial internet and industrial automation data acquisition technology, specifically a factory data acquisition method based on automatic adaptation of multiple protocols. This method is applied to an edge gateway and includes: acquiring continuous raw messages from factory field access devices; extracting physical layer information, link layer information, and application layer information from the raw industrial data stream to form protocol feature information; outputting protocol type identifiers and matching values ​​via a protocol comparison module; performing routing control based on dual thresholds, executing target protocol parsing, parallel trial parsing, or unknown protocol processing; converting the parsing results into data formats and appending standard timestamps and quality markers; and triggering a simplified extraction method to continuously execute factory data acquisition when the protocol parsing delay exceeds the threshold.

Description

Technical Field

[0001] This invention relates to the field of industrial internet and industrial automation data acquisition technology, specifically a factory data acquisition method based on automatic adaptation of multiple protocols. Background Technology

[0002] In the process of data acquisition in industrial fields, edge gateways usually need to connect to multiple devices such as programmable logic controllers, CNC equipment, and sensors at the same time, and uniformly receive, identify, parse and upload data from different sources. Therefore, automatic adaptive data acquisition solutions for multiple industrial communication protocols are gradually becoming the technical basis for continuous data acquisition on the factory side. Currently, factory data acquisition systems generally support pre-configuration of protocol types before targeted parsing. If unified access of multiple protocol devices is required, it is usually necessary to pre-bind communication protocols, fix parsing rules, or manually maintain the protocol library on the gateway side to obtain the acquired data that can be read by the upper-layer system. However, when processing multi-source industrial data streams using the above methods, protocol confirmation and rule configuration need to be completed before device access. This makes it difficult to identify and adaptively distribute continuous raw messages in real time when new devices are temporarily connected on-site, the protocol type is unclear, or the physical layer error rate exceeds the preset environment correction threshold. Furthermore, it is impossible to perform differentiated parsing, trial parsing, or unknown protocol transfer on the data stream based on the matching reliability, resulting in insufficient protocol adaptation capabilities. At the same time, this processing method usually lacks targeted control over parsing latency in scenarios with duplicate messages, incomplete message boundaries, and high load. It also lacks support for local accumulation, cloud updates, and online expansion of unknown protocols, leading to problems such as easy interruption of the acquisition process, excessive consumption of memory and computing resources, and limited system architecture expansion. Summary of the Invention

[0003] The purpose of this invention is to provide a factory data acquisition method based on automatic adaptation of multiple protocols, and to solve the following technical problems: It avoids the misjudgment caused by manual protocol binding on the acquisition side and interference from on-site lines, and can maintain the continuous operation of production line data acquisition and output a unified data format to reduce the complexity of upper-layer interface processing.

[0004] The objective of this invention can be achieved through the following technical solutions: A factory data acquisition method based on automatic adaptation of multiple protocols, applied to edge gateways, including: S100: Obtain the raw industrial data stream output by the access device, which contains continuous raw messages and nested fields, and extract its physical layer, link layer and application layer information to obtain protocol feature information. S200: Input the protocol feature information into the preset protocol comparison module and output the recognition result containing the protocol type identifier and matching value; S300: Obtain a pre-set first preset threshold and a second preset threshold, wherein the first preset threshold is greater than the second preset threshold; if the matching value is not less than the first preset threshold, route the original industrial data stream to the preset protocol parsing module, execute the first parsing branch step through the corresponding preset target protocol parser and output the protocol parsing result; if the matching value is less than the first preset threshold and not less than the second preset threshold, trigger the parallel trial parsing step, call the preset target protocol parser and the general message parser to perform trial parsing and output the corresponding protocol parsing result; S400, if the matching value is less than the second preset threshold, the original industrial data stream is determined to be unknown protocol data, its protocol feature information is stored in the preset local feature library, and the updated feature library is received to update the protocol comparison module online. S500, when generating protocol parsing results, combines the preset data format conversion unit to convert the protocol parsing results into a unified data format containing standard timestamps and quality markers, and outputs the final processing result; S600 monitors the protocol parsing latency when outputting the final processing result. When the actual protocol parsing latency of consecutive packets reaching a preset number threshold exceeds the preset parsing latency threshold, a simplified extraction method is triggered to stop extracting nested fields from consecutive original packets.

[0005] In one possible implementation, S110, obtain consecutive original packets sent from the same device address, and perform sliding window deduplication on the consecutive original packets; S120: If the checksums of adjacent messages in consecutive original messages are the same and the lengths are consistent, then only the first original message is retained as a message sample; otherwise, return to S110 to continue execution. S130, perform frame boundary segmentation on the packet sample based on the combined segmentation rule of delimiter and inter-frame idle time, and extract the corresponding physical layer information, link layer information and application layer information. S140, the extracted physical layer information, link layer information and application layer information are combined accordingly to generate protocol feature information and output the first processing result.

[0006] In one possible implementation, S210, the physical layer message error rate of the factory field communication network transmitting raw industrial data streams is obtained per unit time. S220 determines the line interference value based on the physical layer message error rate and the preset environmental correction coefficient; S230, Input the protocol feature information into the protocol comparison module to obtain the initial matching value output by each known supported protocol; S240 combines the pre-introduced correction model and line interference value to dynamically compensate and correct the initial matching value, and outputs a second processing result containing the protocol type identifier and the corrected matching value.

[0007] In one possible implementation, S310, if the matching value is greater than or equal to the first preset threshold, the original industrial data stream and protocol type identifier are routed to the preset protocol parsing module for protocol parsing, the first parsing branch step is executed, and the protocol parsing result is output; S320, if the matching value is less than the first preset threshold and greater than or equal to the second preset threshold, then the parallel trial parsing step is triggered and executed, by simultaneously calling the target protocol parser and the general message parser to perform trial parsing on the continuous original messages; S330, Check the parsing results for the length and data type of the data fields in the test parsing output; S340: If the data field length and data type check pass, accept the parsing result of the trial parser and output the protocol parsing result; otherwise, discard the continuous original messages and trigger an alarm signal.

[0008] In one possible implementation, S311, the protocol parsing rule file corresponding to the protocol type identifier is retrieved from the cache of the protocol parsing module; If S312 is not hit, on-demand loading is triggered, and the corresponding protocol parsing plugin is loaded from the local storage area into the cache area; S313, based on the protocol parsing plugin, performs parsing operations on the raw industrial data stream; S314 When the access device is detected to be disconnected and no corresponding protocol data stream is detected within the preset time window, the protocol parsing plugin is uninstalled and the memory resources of the edge gateway are released.

[0009] In one possible implementation, S410, when the amount of unknown protocol data in the local feature library reaches the preset batch size, the unknown protocol data is removed by field masking or truncation to remove sensitive content, while retaining message structure information and time sequence information. S420 will report the retained message structure information and time sequence information to the cloud server for summary and analysis; S430 evaluates update weights based on the number of valid identification data uploaded by each edge gateway and the total number of data in the cloud, and generates an overall update plan based on the update weights and the newly added identification data locally. S440: After the cloud-based identification and analysis is completed, it receives an updated feature library containing the corresponding relationships of the new protocol identification based on the total update volume, updates the local protocol comparison module online, and outputs the fourth processing result.

[0010] In one possible implementation, S510 extracts the local time of the arrival of the raw industrial data stream at the edge gateway as a standard timestamp; S520: If the underlying protocol verification result of the original industrial data stream passes, the matching value is mapped to the corresponding initial quality flag; if the verification result fails, the initial quality flag is set to an error state. S530, when the matching value is less than the first preset threshold and greater than or equal to the second preset threshold, and the parallel trial parsing step is passed, the initial quality mark is adjusted to the pending confirmation state to obtain the final quality mark; S540 maps the business data in the protocol parsing result into a key-value pair format, encapsulates and combines it with the standard timestamp and final quality mark, generates a unified data format with a unified interface format, and outputs the final processing result.

[0011] In one possible implementation, the S610 obtains the actual protocol parsing latency of the current packet in real time through the performance monitoring component built into the edge gateway; S620 compares the actual protocol parsing latency with the preset parsing latency threshold; S630, if the actual protocol parsing delay of consecutive preset number threshold messages is greater than the preset parsing delay threshold, then the processing delay is determined to be too high. S640 triggers a simplified extraction method, stopping the extraction of nested fields in the message and retaining only the fixed offset field for protocol identification until the actual protocol parsing delay is less than or equal to the preset parsing delay threshold.

[0012] In one possible implementation, S710 receives a feature import instruction from the local management interface of the edge gateway and obtains a manually imported customized protocol feature file. S720, based on customized protocol feature files, skips the cloud analysis process and forces an update to the custom rule base of the protocol comparison module; After receiving the update confirmation command, the S730 starts the continuous data acquisition process, automatically identifying and acquiring the raw industrial data streams from the access devices in real time based on the updated custom rule base, and completes the data acquisition.

[0013] The beneficial effects of this invention are: 1. This invention generates protocol feature information by extracting physical layer, link layer, and application layer information from the original industrial data stream. Based on the comparison of the matching value with a first preset threshold and a second preset threshold, it automatically performs differentiated routing control on the data stream, such as direct parsing, parallel trial parsing, or storage in a local feature library. At the same time, it calculates the line interference value by combining the physical layer message error rate of the factory's field communication network, and then corrects the initial matching value. This scheme effectively solves the limitation of traditional acquisition systems that require manual pre-binding of communication protocols or fixed parsing rules, avoids misjudgments caused by abnormal amplification of matching values ​​due to field line interference, and realizes automatic identification and adaptive diversion of industrial data streams from multiple sources. Attached Figure Description

[0014] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of the factory data acquisition method based on automatic adaptation of multiple protocols according to the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Please see Figure 1 A factory data acquisition method based on automatic adaptation of multiple protocols is applied to edge gateways, including: S100: Obtain the raw industrial data stream output by the access device, which contains continuous raw messages and nested fields, and extract its physical layer, link layer and application layer information to obtain protocol feature information. S200: Input the protocol feature information into the preset protocol comparison module and output the recognition result containing the protocol type identifier and matching value; S300: Obtain a pre-set first preset threshold and a second preset threshold, wherein the first preset threshold is greater than the second preset threshold; if the matching value is not less than the first preset threshold, route the original industrial data stream to the preset protocol parsing module, execute the first parsing branch step through the corresponding preset target protocol parser and output the protocol parsing result; if the matching value is less than the first preset threshold and not less than the second preset threshold, trigger the parallel trial parsing step, call the preset target protocol parser and the general message parser to perform trial parsing and output the corresponding protocol parsing result; S400, if the matching value is less than the second preset threshold, the original industrial data stream is determined to be unknown protocol data, its protocol feature information is stored in the preset local feature library, and the updated feature library is received to update the protocol comparison module online. S500, when generating protocol parsing results, combines the preset data format conversion unit to convert the protocol parsing results into a unified data format containing standard timestamps and quality markers, and outputs the final processing result; S600 monitors the protocol parsing latency when outputting the final processing result. When the actual protocol parsing latency of consecutive packets that reach a preset number threshold is greater than the preset parsing latency threshold, a simplified extraction method is triggered to stop extracting nested fields from consecutive original packets. The system acquires the raw industrial data stream output by the access device, containing continuous raw messages and nested fields. It extracts message information from the raw industrial data stream layer by layer to obtain protocol feature information including physical layer, link layer, and application layer information. The first processing result is then output, mainly including steps S110 to S140: S110: Obtain consecutive raw packets sent from the same device address, and perform sliding window deduplication on the consecutive raw packets; S120: If the checksums of adjacent messages in consecutive original messages are the same and the lengths are consistent, then only the first original message is retained as a message sample; otherwise, return to S110 to continue execution. S130, perform frame boundary segmentation on the packet sample based on the combined segmentation rule of delimiter and inter-frame idle time, and extract the corresponding physical layer information, link layer information and application layer information. S140, The extracted physical layer information, link layer information and application layer information are combined accordingly to generate protocol feature information and output the first processing result; The protocol feature information is input into a pre-deployed protocol comparison module for identification processing, and the output is a second processing result containing the protocol type identifier and the matching value, mainly including S210 to S240: S210, obtain the physical layer message error rate of the factory field communication network transmitting raw industrial data streams per unit time; S220 determines the line interference value based on the physical layer message error rate and the preset environmental correction coefficient; S230, Input the protocol feature information into the protocol comparison module to obtain the initial matching value output by each known supported protocol; S240, combined with the pre-introduced correction model and line interference value, dynamically compensates and corrects the initial matching value, and outputs a second processing result containing the protocol type identifier and the corrected matching value. The original industrial data stream is routed based on the matching value. A first preset threshold and a second preset threshold are obtained, and the first preset threshold is greater than the second preset threshold. If the matching value is greater than or equal to the first preset threshold, the original industrial data stream is routed to a preset protocol parsing module, the corresponding target protocol parser is called for parsing, the first parsing branch step is executed, and the protocol parsing result is output. If the matching value is less than the first preset threshold and greater than or equal to the second preset threshold, a parallel trial parsing step is triggered and executed. Simultaneously, the target protocol parser and the general message parser are invoked for trial parsing, and the protocol parsing result is output, mainly including S310 to S340: S310, if the matching value is greater than or equal to the first preset threshold, the original industrial data stream and protocol type identifier are routed to the preset protocol parsing module for protocol parsing, the first parsing branch step is executed, and the protocol parsing result is output; S320, if the matching value is less than the first preset threshold and greater than or equal to the second preset threshold, then the parallel trial parsing step is triggered and executed, by simultaneously calling the target protocol parser and the general message parser to perform trial parsing on the continuous original messages; S330, check the parsing result of the data field length and data type of the test parsing output; S340, if the data field length and data type check passes, accept the parsing result of the test parsing output and output the protocol parsing result; otherwise, discard the continuous original messages and trigger an alarm signal.

[0017] Further explanation: The edge gateway is installed on the factory production line side and connects to PLC, CNC and sensor devices via RS485 bus and industrial Ethernet respectively; Each device continuously sends raw industrial data streams. The gateway does not pre-define the specific protocol used by the access devices, but directly identifies, diverts, and parses the continuous raw messages on the receiving side to maintain the continuous operation of production line data acquisition. This implementation method is mainly used to illustrate the overall execution process of the method in the edge gateway. After the gateway receives continuous raw messages from the field devices, it first merges them according to the device address in the acquisition thread and sends messages from the same source into the feature extraction pipeline. This pipeline does not directly perform feature parsing on every frame of message. Instead, it first forms continuous message segments and then extracts physical layer information, link layer information and application layer information from the continuous message segments. After extraction, the three types of information are combined into a set of protocol feature information and written into the recognition buffer for use by the protocol comparison module. After the protocol comparison module outputs the protocol type identifier and the corresponding matching value, the gateway's routing control unit determines the subsequent processing path based on the range in which the matching value falls: if it is higher than the first preset threshold, it directly enters the corresponding protocol parsing module; if it is lower than the first preset threshold but not lower than the second preset threshold, it enters the parallel trial parsing branch. Data below the second preset threshold is recorded as unknown protocol data and transferred to the local feature library for storage; the parsed data is then uniformly sent to the data format conversion unit, where standard timestamps and quality markers are added to form a unified data format before being output to the upper layer system. At the same time, the performance monitoring component continuously counts the protocol parsing latency. When the latency continues to be higher than the preset parsing latency threshold, the feature extraction pipeline switches to a simplified extraction method, stops extracting nested fields in the message, and only retains fixed offset fields for subsequent identification until the latency falls back. From S110 to S140, under continuous data acquisition conditions at the factory site, the raw industrial data stream typically contains periodic repetitive messages. If deduplication is not performed first, the protocol comparison module will repeatedly process the same content, which will increase the system's computational load and amplify the impact of periodic messages from the same device on the recognition results. Therefore, in this embodiment, a sliding window is set within the same device address range; after reading consecutive original messages within the window, the checksum and message length are compared according to the adjacency relationship; when the two are the same, only the first original message is retained as a message sample, and the remaining duplicate messages are no longer included in the subsequent feature extraction. If they are not the same, continue sliding the window until a message sample that can be segmented is obtained; after the message sample is formed, the frame boundary is segmented according to the combination of delimiter and inter-frame idle time segmentation rule. The specific combinational logic is as follows: scan the preset delimiters in the data stream, such as a fixed byte sequence, to perform preliminary boundary positioning; The arrival time difference between the current byte and the previous byte is calculated as the inter-frame idle time. When this time difference is greater than a predetermined multiple of the character transmission period corresponding to the baud rate, it is determined as an auxiliary boundary. The system performs cross-validation based on the dual conditions of delimiter identification and idle time pause to segment complete frame boundaries. For example, for serial bus messages, the focus can be on identifying boundaries by combining inter-frame idle time; for industrial Ethernet messages, the focus can be on segmentation by combining the fixed byte sequence of the frame header. Under complex electromagnetic interference, the above two must be combined for combined verification and segmentation; the physical layer information, link layer information and application layer information are extracted from the segmented complete frames respectively, and combined into protocol feature information in a predetermined order; the protocol feature information enters the protocol comparison module; If a complete frame boundary is not obtained within a certain sampling period, the current message sample is not directly sent to the protocol comparison module, but is retained until the next consecutive message arrives and the boundary is filled in, so as to avoid distorted protocol features caused by message truncation or message sticking. The purpose of this implementation method is to avoid the high matching value caused by on-site line interference. The edge gateway counts the physical layer message error rate of the factory's on-site communication network within a unit of time and multiplies the error rate by a preset environmental correction coefficient to obtain the line interference value. After the protocol feature information enters the protocol comparison module, the protocol comparison module first outputs the initial matching value for each known protocol it supports. The correction amount is generated by multiplying the line interference value by a preset proportional coefficient and then subtracting it from the initial matching value to obtain the corrected matching value. Specifically, for example, if the edge gateway counts a physical layer packet error rate of 5% per unit time at a certain moment, i.e. 0.05, and assuming the preset environment correction factor is set to 100, the calculated line interference value is 5; if the preset ratio factor is 1.5, the corresponding correction amount is 7.5. When the protocol comparison module gives an initial matching value of 92 for a specific protocol, after deduction calculation, the corrected matching value is 84.5. Through this structured quantification calculation mechanism, the system directly transforms the physical state of the underlying link into a quantifiable numerical deduction item for application layer feature matching, avoiding the misjudgment of erroneously spliced ​​message features as high-confidence matches in environments where the physical layer message error rate exceeds the set threshold. When outputting, retain the protocol type identifier and its corresponding corrected matching value for subsequent routing control calls; if no valid physical layer packet error rate is counted within a unit of time, for example, when the field link has just been restored or the number of samples in the statistical period is insufficient, the physical layer packet error rate of the previous valid period can be used to participate in the current correction to ensure that the identification process is not interrupted. At the same time, the correction result is marked as a retained value for use in subsequent status checks; the above configuration is to ensure that when electromagnetic interference increases on site, the matching value will not directly enter the high-confidence parsing path due to accidental matching of local features of the original message; In this implementation of routing control, a dual-threshold traffic splitting method is used. If the corrected matching value is greater than or equal to the first preset threshold, the gateway directly sends the original industrial data stream and protocol type identifier to the preset protocol parsing module, executes the first parsing branch step, and outputs the protocol parsing result. If the corrected matching value is lower than the first preset threshold but greater than or equal to the second preset threshold, the gateway does not immediately accept the identification conclusion, but instead simultaneously calls the target protocol parser and the general message parser to perform trial parsing on the continuous original messages; after the results of the two trial parsings are summarized into the inspection unit, the length and data type of the output data field are checked. If the check passes, the parsing result is used as the protocol parsing result and the packet is transmitted. If the check fails, the current continuous original packets are discarded and an alarm signal is triggered. The focus of this check is to exclude cases of truncated fields, inconsistent field lengths, and mismatched data types. If the target protocol parser does not return a result within the current processing time limit during the parallel trial parsing, the current message will not enter the accept branch, but will be processed as failing the check, in order to avoid the gateway outputting an uncertain parsing result in an intermediate state; Specifically, the above steps continue to apply to the production line edge gateway; for raw industrial data streams with known protocol types and matching values ​​not less than the first preset threshold, the gateway can maintain a direct parsing path; for data streams with interference or insufficient protocol characteristics, the gateway performs secondary confirmation through parallel trial parsing branches. For data streams below the second preset threshold, the process switches to an unknown protocol processing path, without occupying the normal processing resources of the protocol parsing module; thus, the acquisition side can automatically adapt to multi-protocol access without pre-binding protocols manually.

[0018] In a preferred embodiment of the present invention, the original industrial data stream and protocol type identifier are routed to a preset protocol parsing module for protocol parsing, and the first parsing branch step is executed, mainly including S311 to S314: S311, retrieve the protocol parsing rule file corresponding to the protocol type identifier from the cache of the protocol parsing module; If S312 is not hit, on-demand loading is triggered, and the corresponding protocol parsing plugin is loaded from the local storage area into the cache area; S313, based on the protocol parsing plugin, performs parsing operations on the raw industrial data stream; S314, when the access device is detected to be disconnected and no corresponding protocol data stream is detected within the preset time window, the protocol parsing plugin is uninstalled and the memory resources of the edge gateway are released; Trigger the local and cloud update process, receive the updated feature library from the cloud to update the protocol comparison module online, and output the fourth processing result, mainly including S410 to S440: S410: When the amount of unknown protocol data in the local feature library reaches the preset batch size, sensitive content is removed from the unknown protocol data by field masking or truncation, while retaining message structure information and time sequence information. S420 will report the retained message structure information and time sequence information to the cloud server for summary and analysis; S430 evaluates update weights based on the number of valid identification data uploaded by each edge gateway and the total number of data in the cloud, and generates an overall update plan based on the update weights and the newly added identification data locally. S440: After the cloud-based identification and analysis is completed, it receives the updated feature library containing the corresponding relationships of the new protocol identification based on the total update volume, updates the local protocol comparison module online, and outputs the fourth processing result. Based on the protocol parsing results and combined with a pre-built data format conversion unit, data format conversion is performed to obtain a unified data format containing standard timestamps and quality markers, and the final processing result is output, mainly including S510 to S540: S510 extracts the local time of the arrival of the raw industrial data stream at the edge gateway as a standard timestamp; S520: If the underlying protocol verification result of the original industrial data stream passes, the matching value is mapped to the corresponding initial quality flag; if the verification result fails, the initial quality flag is set to an error state. S530, when the matching value is less than the first preset threshold and greater than or equal to the second preset threshold, and the parallel trial parsing step is passed, the initial quality mark is adjusted to the pending confirmation state to obtain the final quality mark; S540 maps the business data in the protocol parsing result into a key-value pair format, encapsulates and combines it with the standard timestamp and the final quality mark, generates a unified data format with a unified interface format, and outputs the final processing result. Further explanation: During normal production, several fixed PLCs and sensors are connected to the same production line for a long time. At the same time, during equipment debugging, maintenance or process switching, equipment using other protocols may be temporarily connected. At this point, the edge gateway needs to keep the existing protocol parsing thread running continuously, and on the other hand, it needs to load the new protocol and output data in a unified format without interrupting the existing data collection tasks. Although the first parsing branch can send high-matching value data streams into the protocol parsing module, if all protocol parsing rules are resident in memory, the memory usage of the edge gateway will continue to rise as the number of supported protocols increases; if the parsing service is required to be restarted every time a new protocol is added, it will cause data collection interruption; based on this limitation, on-demand loading and idle unloading processing methods have been added. Specifically, after the original industrial data stream and protocol type identifier are routed to the protocol parsing module, the parsing module first retrieves the protocol parsing rule file corresponding to the protocol type identifier in the cache; Configure the cache area as the storage area for the currently active protocol to restrict it to only storing parsing rules that are currently in the calling state; when a retrieval hits, the raw industrial data stream directly enters the corresponding protocol parsing plugin to perform parsing operations, and the parsing results are written to the parsing result buffer as protocol parsing results for subsequent conversion units to read; If the cache is not found, the parsing module retrieves the corresponding protocol parsing plugin from the local storage and loads it into the cache, and then performs parsing on the original industrial data stream. The purpose of this approach is to use the local storage area as a storage area for inactive parsing rules, only consuming memory resources when the corresponding protocol data stream actually appears on-site; to avoid frequent installation and uninstallation due to short-term device disconnection, when the access device is detected to be disconnected, the gateway does not immediately uninstall the protocol parsing plugin, but continues to monitor for the existence of the corresponding protocol data stream within a preset time window; The plugin is only uninstalled and the edge gateway's memory resources are released if no corresponding protocol data stream is detected within the window. If the device is quickly reconnected during the disconnection, the plugin in the original cache can still be reused directly without needing to be reloaded. This implementation describes the local and cloud update process for unknown protocol data. While simple storage of unknown protocol data can achieve local isolation, it cannot explain when to report, what content to report, or how the local identification capability is updated. To address this issue, this implementation method initiates an update process when the local feature library reaches a preset batch size. Once the batch size is reached, the gateway performs field masking or truncation on unknown protocol data, removing sensitive content and retaining only message structure information and time sequence information. The processed data is reported to the cloud server for aggregation and analysis. During the cloud analysis, the update is not determined directly based on the data of a single edge gateway. Instead, the update ratio is determined by combining the ratio of the number of valid identification data uploaded by each edge gateway to the total number of data in the cloud. Then, the update ratio is multiplied by the locally added identification data to obtain the total update amount. Based on the storage capacity and computing resource threshold limits of the edge gateway, redundant transmission of the full protocol library in the cloud is avoided; through the above processing logic, the proportion of effective data provided by the gateway to the cloud is directly mapped to the weight of the number of new feature library entries obtained. For example, if the total number of data entries in the cloud is 10,000, and the number of data entries uploaded by the edge gateway and effectively identified in the cloud is 200, then the update ratio is 0.02. Assuming that the gateway adds 500 new features during this analysis period, the total update count calculated by multiplication is 10. If the result contains a decimal, it will be rounded down. With this configuration, the cloud server is limited to prioritizing the screening and distribution of the 10 new protocol feature correspondences with the highest matching degree. This clarifies the technical motivation for algorithm selection and enables dynamic balanced allocation of cloud and edge resources. After the cloud identification and analysis are completed, the edge gateway receives the updated feature library containing the corresponding relationship of the new protocol identification based on the total update volume, and replaces or supplements the identification content in the local protocol comparison module online. The online update process does not interfere with the running protocol parsing thread, but only affects the new data stream that subsequently enters the protocol comparison module; If the cloud does not return an updated feature library within the current cycle, the local feature library will continue to retain unconfirmed unknown protocol data, waiting for the next update process to be triggered, in order to avoid the loss of local unknown protocol data due to a single communication interruption. Although the protocol parsing result has been completed, the field expression methods of different protocols are not consistent. If it is uploaded directly, the upper-layer MES or SCADA side will need to adapt to different formats, which will increase the complexity of interface processing. Therefore, this embodiment sets up a data format conversion unit in the edge gateway; this unit extracts the local time from the moment the original industrial data stream arrives at the edge gateway and uses it as a standard timestamp; it reads the underlying protocol verification result: when the verification passes, it maps the initial quality mark according to the matching value; The specific mapping logic is as follows: a mapping table between matching value ranges and quality levels is pre-established in the system. For example, when the matching value is greater than or equal to the first preset threshold, the initial quality label is directly mapped and assigned the first confidence state. When the matching value is less than the first preset threshold but greater than or equal to the second preset threshold, it is mapped and assigned to the second pending confirmation state, thereby achieving accurate and quantifiable quality classification; when the verification fails, the initial quality mark is directly set to the error state, and the quality level is no longer improved based on the matching value. For data that passes the parallel trial parsing step, since its matching value falls between the first preset threshold and the second preset threshold, even if the parsing check passes, it will not be output directly in the normal state. Instead, the initial quality mark will be adjusted to the pending confirmation state and used as the final quality mark. After the above processing is completed, the business data in the protocol parsing result is mapped into a key-value pair format, and then combined and encapsulated with the standard timestamp and the final quality mark to form a unified data format with a unified interface format before being output. If a message only yields some service fields after parsing, the conversion unit retains the parsed valid key-value pairs and attaches the corresponding quality flag to avoid discarding the entire frame of service data due to missing partial fields. In this production line scenario, data streams with high matching values ​​or confirmed by trial parsing work together; the protocol parsing plugin is loaded on demand and unloaded when idle, reducing the memory usage of the edge gateway; Unknown protocol data is reported in batches and the feature library is updated, so that subsequent similar data streams can be identified locally; parsed business data is uniformly encapsulated and provided to the upper-level system with a consistent data format; after this processing, the production line can maintain a unified data output during daily production, temporary maintenance and equipment access changes.

[0019] In a preferred embodiment of the present invention, load adjustment is performed on the output process of the final processing result, and the protocol parsing delay is monitored in real time. When the protocol parsing delay is greater than a preset parsing delay threshold, a simplified extraction method is triggered to stop extracting nested fields of the message to reduce the data parsing load and continue to perform factory data acquisition. This mainly includes steps S610 to S640: S610 uses the performance monitoring component built into the edge gateway to obtain the actual protocol parsing latency of the current packet in real time. S620 compares the actual protocol parsing latency with the preset parsing latency threshold; S630, if the actual protocol parsing delay of consecutive preset number threshold messages is greater than the preset parsing delay threshold, then it is determined that the processing delay exceeds the preset processing range. S640, triggers simplified extraction mode, stops extracting nested fields of the message, and only retains fixed offset field for protocol identification until the actual protocol parsing delay is less than or equal to the preset parsing delay threshold; The method also includes confirmation processing S700: S710, receiving feature import instructions from the local management interface of the edge gateway, and obtaining manually imported customized protocol feature files; S720, based on customized protocol feature files, skips the cloud analysis process and forces an update to the custom rule base of the protocol comparison module; After receiving the update confirmation command, the S730 starts the continuous data acquisition process, automatically identifying and acquiring the raw industrial data streams from the access devices in real time based on the updated custom rule base, and completes the data acquisition.

[0020] Further explanation: Edge gateways are deployed on the factory production line side for a long time. In addition to normal access to fixed equipment, they also need to collect and record temporary messages during the centralized online and maintenance phases of equipment. During this period, the number of messages increases and the types of protocols may change. The edge gateway must maintain uninterrupted data collection while allowing on-site engineers to access customized protocols locally. Under normal load, the processing flow can complete continuous processing from feature extraction to unified output. However, when the message density exceeds the preset density threshold, the number of parsing plugins exceeds the preset number threshold, or the number of application layer field nesting layers exceeds the preset number, if all messages are still processed according to the original extraction depth, the protocol parsing latency may continue to rise, further causing the final processing results to back up. To solve this limitation, a load adjustment process has been added. Specifically, the performance monitoring component built into the edge gateway obtains the actual protocol parsing latency of the current packet in real time; The starting point for latency measurement is the time stamp when the original message is assigned to the preset protocol parsing module by the routing control unit or when parallel trial parsing begins, and the ending point is the time stamp when the relevant module completes the parsing process and successfully outputs the protocol parsing result. The component obtains the exact actual protocol parsing latency by subtracting the time stamp of the ending point from the time stamp of the starting point. The system compares the latency with a preset parsing latency threshold. If the number of packets with a parsing latency greater than the preset parsing latency threshold is less than the preset number threshold, the system does not immediately switch the extraction method, but continues to process them in the current method to avoid unnecessary switching caused by transient fluctuations. Only when the actual protocol parsing delay of a series of messages that continuously reach the preset number threshold is greater than the preset parsing delay threshold, is it determined that the simplified extraction trigger condition is met. At this point, the gateway triggers a simplified extraction method, stopping the extraction of nested fields in the message and retaining only the fixed offset field for protocol identification; The purpose of this is to prioritize ensuring that the main process of identification and data collection continues to run, rather than fully expanding deep fields during periods of high load. The simplified extraction method is not permanent. The performance monitoring component continues to track the actual protocol parsing latency of subsequent packets. Once the latency drops to no more than the preset parsing latency threshold, the gateway can exit the simplified extraction method and restore the original extraction depth. If, during the simplified extraction process, the matching value of individual packets decreases due to insufficient fields, they will still be entered into the corresponding branch according to the double threshold rule, without changing the original identification and routing framework. This implementation method is used to illustrate the confirmation process of field engineers in the local management interface; relying solely on the cloud update process, when encountering certain customized protocols whose source is already clearly known on site, it is still necessary to wait for the cloud to complete the identification and analysis before updating the local protocol comparison module, and the processing cycle is longer than the preset update cycle; Based on this situation, the edge gateway sets up a local management interface to receive feature import instructions. After receiving the feature import instructions, the gateway obtains the manually imported customized protocol feature file and, based on the feature file, directly skips the cloud analysis process and forcibly updates the corresponding content to the custom rule base of the protocol comparison module. The custom rule base here exists alongside the existing identification content in the original protocol comparison module, and is given priority for matching the corresponding customized protocol data stream; After the update is completed, the gateway does not automatically switch to data collection immediately. Instead, it starts a continuous data collection process after receiving the update confirmation instruction and automatically identifies and collects the raw industrial data streams of the access devices in real time based on the updated custom rule base. The purpose of this approach is to retain a confirmation step after the on-site commissioning personnel import the customized protocol feature file, so that the file version and applicable equipment can be confirmed before the system is put into operation. If the imported customized protocol feature file conflicts with the custom rules in the current cache, the imported content will not overwrite the original rules before receiving the update confirmation instruction, so as to prevent unconfirmed data from directly affecting the actual network data collection on site. Specifically, they correspond to two types of field problems: the former is used to maintain the continuity of data collection when the message processing load increases, and the latter is used to provide local customized protocol access paths outside the cloud process; Neither of them changes the defined main processing order. Instead, they supplement the load pressure and on-site customized access based on the automatic identification and resolution already performed by the edge gateway.

[0021] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A factory data acquisition method based on automatic adaptation of multiple protocols, applied to edge gateways, characterized in that: include: S100: Obtain the raw industrial data stream output by the access device, which contains continuous raw messages and nested fields, and extract its physical layer, link layer and application layer information to obtain protocol feature information. S200, the protocol feature information is input into a preset protocol comparison module, and the recognition result containing the protocol type identifier and matching value is output; S300, obtain a preset first preset threshold and a second preset threshold, wherein the first preset threshold is greater than the second preset threshold; If the matching value is not less than the first preset threshold, the original industrial data stream is routed to the preset protocol parsing module, and the first parsing branch step is executed through the corresponding preset target protocol parser and the protocol parsing result is output; if the matching value is less than the first preset threshold and not less than the second preset threshold, the parallel trial parsing step is triggered, and the preset target protocol parser and general message parser are called to perform trial parsing and output the corresponding protocol parsing result. S400, if the matching value is less than the second preset threshold, the original industrial data stream is determined to be unknown protocol data, its protocol feature information is stored in a preset local feature library, and the updated feature library is received to update the protocol comparison module online. S500: When a protocol parsing result is generated, the protocol parsing result is converted into a unified data format containing standard timestamps and quality markers by combining the preset data format conversion unit, and the final processing result is output. S600, monitor the protocol parsing delay when outputting the final processing result. When the actual protocol parsing delay of consecutive packets reaching a preset number threshold is greater than the preset parsing delay threshold, trigger the simplified extraction method and stop extracting the nested fields of the consecutive original packets.

2. The method of claim 1, wherein, The process involves acquiring the raw industrial data stream output by the access device, which contains continuous raw packets and nested fields; extracting packet information from the raw industrial data stream layer by layer to obtain protocol feature information containing physical layer information, link layer information, and application layer information; and outputting the first processing result, mainly including steps S110 to S140. S110, obtain the consecutive original packets sent from the same device address, and perform sliding window deduplication on the consecutive original packets; S120: If the checksums of adjacent messages in the consecutive original messages are the same and the lengths are consistent, then only the first original message is retained as a message sample; otherwise, return to S110 to continue execution. S130, the message sample is segmented by frame boundaries based on the combined segmentation rule of delimiter and inter-frame idle time, and the corresponding physical layer information, link layer information and application layer information are extracted. S140, the extracted physical layer information, link layer information and application layer information are combined accordingly to generate protocol feature information and output the first processing result.

3. The method of claim 1, wherein, The protocol feature information is input into a pre-deployed protocol comparison module for identification processing, and a second processing result containing the protocol type identifier and matching value is output, mainly including S210 to S240: S210, Obtain the physical layer message error rate of the factory field communication network transmitting the original industrial data stream per unit time; S220, determine the line interference value based on the physical layer message error rate and the preset environmental correction coefficient; S230, The protocol feature information is input into the protocol comparison module to obtain the initial matching value output by the module for each known supported protocol; S240, the initial matching value is dynamically compensated and corrected by combining the pre-introduced correction model and the line interference value, and a second processing result containing the protocol type identifier and the corrected matching value is output.

4. The method of claim 1, wherein, The original industrial data stream is routed according to the matching value. A first preset threshold and a second preset threshold are obtained, where the first preset threshold is greater than the second preset threshold. If the matching value is greater than or equal to the first preset threshold, the original industrial data stream is routed to a preset protocol parsing module, the corresponding target protocol parser is called for parsing, a first parsing branch step is executed, and the protocol parsing result is output. If the matching value is less than the first preset threshold but greater than or equal to the second preset threshold, a parallel trial parsing step is triggered and executed, simultaneously calling the target protocol parser and a general message parser for trial parsing, and outputting the protocol parsing result. This mainly includes steps S310 to S340: S310, if the matching value is greater than or equal to the first preset threshold, the original industrial data stream and protocol type identifier are routed to the preset protocol parsing module for protocol parsing, the first parsing branch step is executed, and the protocol parsing result is output; S320, if the matching value is less than the first preset threshold and greater than or equal to the second preset threshold, then the parallel trial parsing step is triggered and executed, by simultaneously calling the target protocol parser and the general message parser to perform trial parsing on the continuous original messages; S330, Check the parsing results for the length and data type of the data fields output by the test parser; S340, if the data field length and data type check pass, then accept the parsing result of the trial parsing output and output the protocol parsing result; otherwise, discard the continuous original messages and trigger an alarm signal.

5. The method of claim 4, wherein, The original industrial data stream and protocol type identifier are routed to a preset protocol parsing module for protocol parsing, and the first parsing branch step is executed, mainly including S311 to S314: S311, retrieve the protocol parsing rule file corresponding to the protocol type identifier from the cache area of ​​the protocol parsing module; S312, if a match is not found, on-demand loading is triggered, and the corresponding protocol parsing plugin is loaded from the local storage area into the cache area; S313, perform parsing operations on the original industrial data stream based on the protocol parsing plugin; S314, when the access device is detected to be disconnected and no corresponding protocol data stream is detected within a preset time window, the protocol parsing plugin is uninstalled and the memory resources of the edge gateway are released.

6. The method of claim 1, wherein, Trigger the local and cloud update process, receive the updated feature library from the cloud to update the protocol comparison module online, and output the fourth processing result, mainly including S410 to S440: S410, when the amount of unknown protocol data in the local feature library reaches the preset batch size, sensitive content is removed from the unknown protocol data by field masking or truncation, while retaining message structure information and time sequence information; S420, the retained message structure information and time sequence information are reported to the cloud server for summary analysis; S430: Evaluate the update weight based on the number of valid identification data uploaded by each edge gateway and the total number of data in the cloud, and generate a total update plan based on the update weight and the newly added identification data locally; S440, after the cloud-based identification and analysis is completed, the updated feature library containing the corresponding relationship of the new protocol identification is received according to the total update amount, the local protocol comparison module is updated online, and the fourth processing result is output.

7. The method of claim 1, wherein, Based on the protocol parsing results, and combined with a pre-set data format conversion unit, data format conversion is performed to obtain a unified data format containing standard timestamps and quality markers, and the final processing result is output, mainly including S510 to S540: S510, extract the local time of the arrival of the original industrial data stream at the edge gateway as a standard timestamp; S520, if the underlying protocol verification result of the original industrial data stream is passed, then the matching value is mapped to the corresponding initial quality mark; If the verification result is unsuccessful, the initial quality mark is set to an error state; S530, when the matching value is less than the first preset threshold and greater than or equal to the second preset threshold, and the parallel trial parsing step is passed, the initial quality mark is adjusted to the pending confirmation state to obtain the final quality mark; S540, the business data in the protocol parsing result is mapped into a key-value pair format, and then encapsulated and combined with the standard timestamp and the final quality mark to generate a unified data format with a unified interface format and output the final processing result.

8. The method of claim 1, wherein, Load adjustment is performed on the output process of the final processing result, and the protocol parsing delay is monitored in real time. When the protocol parsing delay is greater than the preset parsing delay threshold, a simplified extraction method is triggered to stop extracting nested fields of the message to reduce the data parsing load and continue to perform factory data acquisition. This mainly includes S610 to S640: S610, the actual protocol parsing latency of the current packet is obtained in real time through the performance monitoring component built into the edge gateway; S620, compare the actual protocol parsing delay with the preset parsing delay threshold; S630, if the actual protocol parsing delay of consecutive preset number threshold messages is greater than the preset parsing delay threshold, then the processing delay is determined to be too high. S640, trigger the simplified extraction method, stop extracting nested fields of the message, and retain only the fixed offset field for protocol identification until the actual protocol parsing delay is less than or equal to the preset parsing delay threshold.

9. The method of claim 1, wherein, The method further includes a confirmation process S700: S710, receive the feature import instruction from the local management interface of the edge gateway, and obtain the manually imported customized protocol feature file; S720, based on the customized protocol feature file, skipping the cloud analysis process, and forcibly updating the custom rule library of the protocol comparison module; S730, after receiving the update confirmation instruction, starting the continuous collection process, and automatically identifying and collecting the original industrial data stream of the access device based on the updated custom rule library in real time, completing data collection.

Images