Multi-mode identification industrial protocol adaptation method and terminal

Abstract

The invention discloses a multi-mode identification industrial protocol adaptation method and a terminal, and the method comprises the steps: extracting a first multi-mode feature of a to-be-identified message, and determining a protocol type of the to-be-identified message according to the first multi-mode feature; whether a protocol with the same protocol type as the message to be identified exists in a preset protocol library or not is judged, if yes, a preset protocol template corresponding to the protocol with the same protocol type is selected, protocol conversion configuration is conducted on the message to be identified through the preset protocol template, and if not, candidate protocol templates corresponding to the message to be identified are created; and judging whether information that the candidate protocol template completes manual verification and correction is received, if so, obtaining a final protocol template according to the information, and performing protocol conversion configuration on the to-be-identified message through the final protocol template. According to the method, dynamic identification and matching of various industrial protocols can be supported, the workload of manual configuration is reduced, multi-source heterogeneous data in a multi-protocol scene can be rapidly and accurately identified and converted, and convenience is provided for interaction between devices.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and particularly relates to a multi-modal recognition-based industrial protocol adaptation method and a terminal. Background Art

[0002] Energy storage system integration involves multiple manufacturers' devices such as photovoltaic inverters, BMS, PCS, electricity meters, charging piles, and grid dispatching terminals. There are significant protocol differences (such as Modbus / TCP, CAN, IEC104, IEC61850, etc.), presenting heterogeneity. Moreover, protocol conversion between devices relies on manual configuration, resulting in a long debugging cycle. It is prone to system compatibility failures due to protocol version updates, and the data processing efficiency is low.

[0003] Furthermore, existing manual configurations generally adopt single-protocol recognition, such as recognition based on message header features. This method has a high misjudgment rate, especially in scenarios dealing with some private protocols. It is difficult to quickly and accurately complete data conversion when dealing with multi-source heterogeneous data in the protocol conversion of multiple manufacturers' devices. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: to provide a multi-modal recognition-based industrial protocol adaptation method and a terminal, which can quickly and accurately identify and convert multi-source heterogeneous data in a multi-protocol scenario, facilitating the interaction between different devices.

[0005] To solve the above technical problem, the technical solution adopted by the present invention is: A multi-modal recognition-based industrial protocol adaptation method includes the following steps: S1. Extract the first multi-modal feature of the message to be recognized, and determine the protocol type of the message to be recognized according to the first multi-modal feature; S2. Determine whether there is a protocol in the preset protocol library that is the same as the protocol type of the message to be recognized. If so, select the preset protocol template corresponding to the protocol with the same protocol type, and perform protocol conversion configuration on the message to be recognized through the preset protocol template. Otherwise, execute S3; S3. Create a candidate protocol template corresponding to the message to be recognized, and determine whether the information indicating that the candidate protocol template has completed manual verification and correction is received. If so, obtain the final protocol template according to the information, and perform protocol conversion configuration on the message to be recognized through the final protocol template; Determining the protocol type of the message to be recognized according to the first multi-modal feature specifically includes: Identifying the first multi-modal feature through a hybrid model of a graph neural network and a Transformer to obtain the protocol type of the message to be recognized. To solve the above technical problem, the technical solution adopted by the present invention is: An industrial protocol adaptation terminal for multimodal recognition includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the above-mentioned industrial protocol adaptation method for multimodal recognition is implemented.

[0006] The beneficial effects of the present invention are as follows: It provides an industrial protocol adaptation method and terminal for multimodal recognition. By using multimodal recognition and a preset protocol library in combination, it can support the dynamic recognition and matching of multiple industrial protocols, and then perform automated protocol conversion configuration, greatly reducing the manual configuration workload and improving the conversion efficiency. At the same time, an artificial verification and correction process for creating protocol templates is introduced. When an unknown protocol is recognized, a candidate protocol template is first created and then manually verified and corrected. The protocol conversion configuration of the message to be recognized is performed through the final protocol template, effectively dealing with unknown protocols. The combination of automation and manual operation can quickly and accurately identify and convert multi-source heterogeneous data in a multi-protocol scenario, providing convenience for the interaction between different devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is a schematic diagram of the steps of an industrial protocol adaptation method for multimodal recognition of the present invention; Figure 2 It is a schematic diagram of the software function distribution of an industrial protocol adaptation method for multimodal recognition of the present invention; Figure 3 It is a flowchart of an industrial protocol adaptation method for multimodal recognition of the present invention; Figure 4 It is a timing diagram of an industrial protocol adaptation method for multimodal recognition of the present invention; Figure 5 It is a schematic diagram of the composition of a preset protocol library of an industrial protocol adaptation method for multimodal recognition of the present invention; Figure 6 It is a system block diagram of an industrial protocol adaptation terminal for multimodal recognition of the present invention.

[0008] Reference Signs Explanation: 1. An industrial protocol adaptation terminal for multimodal recognition; 2. Memory; 3. Processor. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0009] To describe the technical content, achieved objectives, and effects of the present invention in detail, the following is described in conjunction with the embodiments and with reference to the accompanying drawings.

[0010] Please refer to Figures 1 to 5 , an industrial protocol adaptation method for multimodal recognition, includes the following steps: S1. Extract the first multimodal feature of the message to be recognized, and determine the protocol type of the message to be recognized according to the first multimodal feature; S2. Determine whether there is a protocol in the preset protocol library that is the same as the protocol type of the to-be-identified message. If so, select the preset protocol template corresponding to the protocol with the same protocol type, and perform protocol conversion configuration on the to-be-identified message through the preset protocol template; otherwise, execute S3. S3. Create a candidate protocol template corresponding to the to-be-identified message, and determine whether the information indicating that the candidate protocol template has completed manual verification and correction is received. If so, obtain the final protocol template according to the information, and perform protocol conversion configuration on the to-be-identified message through the final protocol template. As can be seen from the above description, the beneficial effects of the present invention are as follows: By using multi-modal recognition and a preset protocol library in combination, it is possible to support the dynamic recognition and matching of multiple industrial protocols, and then perform automated protocol conversion configuration, greatly reducing the manual configuration workload and improving the conversion efficiency. At the same time, the process of creating a protocol template with manual verification and correction is introduced. When an unknown protocol is recognized, a candidate protocol template is first created, and then it is manually verified and corrected. The to-be-identified message is subjected to protocol conversion configuration through the final protocol template, effectively dealing with unknown protocols; the combination of automation and manual operation can quickly and accurately identify and convert multi-source heterogeneous data in a multi-protocol scenario, providing convenience for the interaction between different devices.

[0011] Further, the specific process of creating a candidate protocol template corresponding to the to-be-identified message is as follows: Select the protocol with the highest adaptability of multi-modal features to the first multi-modal feature from the preset protocol library, and create a candidate protocol template according to the selected protocol.

[0012] As can be seen from the above description, by intelligently matching the protocol with the highest adaptability in the preset library as the basis of the candidate template, the time cost of manually modifying the candidate protocol template can be significantly reduced; the existing protocol structure is preferentially reused, which not only retains the verified reliable framework but also adapts to the differential features of new protocols through a dynamic adjustment mechanism. Manual work only needs to perform targeted correction on local parameters such as keyword fields and verification rules, avoiding repeated development of complete templates.

[0013] Further, the specific process of extracting the first multi-modal feature of the to-be-identified message is as follows: Identify the first multi-modal feature through a hybrid model of a graph neural network and a Transformer to obtain the protocol type of the to-be-identified message.

[0014] As can be seen from the above description, the hybrid model that combines graph neural networks and Transformers has significant advantages in extracting multi-modal features of the message to be recognized. Graph neural networks can efficiently capture the topological structure relationships between data elements in the message, accurately analyze the complex hierarchical and correlative features of industrial protocols, and transform the message structure into a structured feature sequence in the form of nodes and edges. The Transformer, with its powerful attention mechanism, can deeply explore the semantic information of the text fields in the message and extract key feature sequences from the text content such as protocol instructions and parameter descriptions. The combination of the two not only preserves the structural integrity of industrial protocol data but also realizes a deep understanding of semantic information, avoiding the limitations of a single model in extracting structural or semantic information.

[0015] Furthermore, before the step S1, it further includes: S0. Create a multi-modal input source, and obtain the message to be recognized through the multi-modal input source.

[0016] Furthermore, the multi-modal input source includes at least one of network messages, device logs, and topology documents.

[0017] As can be seen from the above description, creating a multi-modal input source before protocol recognition can dock various heterogeneous data sources in the industrial scenario with a unified data entry. Whether it is the time-series data from sensors, the device log text, or the binary messages transmitted over the network, they can all be efficiently obtained through this input source. This integrated data collection mode avoids problems such as complex interface adaptation and inconsistent data formats caused by decentralized collection of multi-source data, and significantly improves the data collection efficiency.

[0018] Furthermore, before the step S1, it further includes: Set a private field mapping table for introducing vendor extension rules on the preset protocol template corresponding to each protocol type in the preset protocol library.

[0019] As can be seen from the above description, setting a private field mapping table for introducing manufacturer extension rules on the preset protocol template can significantly improve the flexibility and compatibility of industrial protocol adaptation. On the one hand, industrial devices of different manufacturers often have personalized function extensions and data format differences on the basis of following the general protocol standard. The private field mapping table can provide an exclusive management space for these manufacturer-specific protocol rules, avoid protocol adaptation failures caused by rule conflicts, and ensure the effective interaction of device data of each manufacturer in a multi-protocol scenario. On the other hand, this mapping table builds a bridge between the standardized protocol and the manufacturer's custom extensions. When there is a new requirement for manufacturer protocol extension, there is no need to conduct large-scale reconstruction of the preset protocol template. Only by performing field mapping and rule configuration in the mapping table can the compatibility with the new extension rules be quickly achieved, greatly reducing the development cost and time cost of protocol adaptation. At the same time, it ensures the stability and scalability of the preset protocol library, providing strong support for the collaborative operation of diverse devices in the industrial Internet environment.

[0020] Further, before the S1, it also includes: Setting a version evolution record table on the preset protocol template corresponding to each protocol type in the preset protocol library; After the content of the preset protocol template is updated, updating the corresponding version evolution record table.

[0021] As can be seen from the above description, the version evolution record table can completely record every change detail during the process of the protocol template from the initial version to iterative updates, including information such as the modified content, modification time, modifying personnel, and modification reasons, providing a clear traceability path for the iterative evolution of the protocol and facilitating technicians to quickly locate historical version problems and summarize optimization experiences.

[0022] Further, after the S3, it also includes: S4. Adding the final protocol template to the preset protocol library.

[0023] As can be seen from the above description, the addition of the final protocol template enriches the resource reserve of the preset protocol library, makes the protocol types in the library more diverse, effectively expands the range of protocols that the system can handle, provides a ready-made matching template for subsequent encounters with similar or the same type of protocols, and greatly shortens the recognition and configuration time. With the continuous accumulation of protocol templates, when the system faces new protocols, it can achieve more accurate multi-modal feature matching and candidate template recommendation based on historical data, further reducing the cost of manual intervention and improving the level of automated adaptation.

[0024] Further, the extraction of the first multi-modal feature of the to-be-identified message also includes: Performing feature extraction on the to-be-identified message through a structure parser, a timing analyzer, and a text miner.

[0025] As can be seen from the above description, by using a preset structure parser, a timing analyzer, and a text miner to extract features from the message, it is possible to achieve professional and refined parsing of multi-modal features.

[0026] Please refer to Figure 6 , an industrial protocol adaptation terminal 1 for multi-modal recognition, including a memory 2, a processor 3, and a computer program stored on the memory 2 and executable on the processor 3. When the processor 3 executes the computer program, the above-mentioned industrial protocol adaptation method for multi-modal recognition is realized.

[0027] As can be seen from the above description, the beneficial effects of the present invention are as follows: By using multi-modal recognition and a preset protocol library in combination, it is possible to support the dynamic recognition and matching of multiple industrial protocols, and then perform automated protocol conversion configuration, greatly reducing the manual configuration workload, improving the conversion efficiency. At the same time, an artificial verification and correction protocol template creation process is introduced. When an unknown protocol is recognized, a candidate protocol template is first created, and then it is manually verified and corrected. The protocol conversion configuration of the message to be recognized is performed through the final protocol template, effectively dealing with unknown protocols; the combination of automation and manual operations can quickly and accurately identify and convert multi-source heterogeneous data in a multi-protocol scenario, providing convenience for the interaction between different devices.

[0028] Please refer to Figures 1 to 4 , the first embodiment of the present invention is: An industrial protocol adaptation method for multi-modal recognition, as Figure 1 and Figure 4 shown, includes the following steps: S0. Create a multi-modal input source, and obtain the message to be recognized through the multi-modal input source. Among them, the multi-modal input source includes at least one of network message capture, device log parsing, and topology document import.

[0029] In this embodiment, the descriptions of the respective components of the multi-modal input source are as follows: 1. Network message capture: Collect protocol traffic such as Modbus / TCP and IEC104 through a mirror port; 2. Device log parsing: Use Logstash to collect the operation logs of battery management system or power conversion system devices; 3. Topology document import: Parse the SCD (Substation Configuration Description) file to obtain the device communication topology.

[0030] S1. Extract the first multi-modal feature of the message to be recognized, and determine the protocol type of the message to be recognized according to the first multi-modal feature; Such as Figure 3As shown in the figure, a structure parser, a timing analyzer, and a text miner are used to extract features from the message to be recognized. Among them, the structure parser can match the message head and tail identifiers based on regular expressions. The timing analyzer can calculate the sliding window standard deviation of the message interval time. The text miner can extract key semantic tags from the device document using the TF-IDF algorithm.

[0031] After extraction, the obtained first multi-modal features include structural features, timing features, statistical features, and text features; a dynamic protocol recognition engine is created, and a graph neural network model is used to recognize the structural features and timing features to obtain topological embedding vectors; a Transformer is used to recognize the statistical features and text features to obtain cross-modal vectors, and then through feature splicing and classification, the protocol type of the message to be recognized is identified.

[0032] S2. Determine whether there is a protocol in the preset protocol library that is the same as the protocol type of the message to be recognized. If so, select the preset protocol template corresponding to the protocol with the same protocol type, and perform protocol conversion configuration on the message to be recognized through the preset protocol template; otherwise, execute S3. S3. Create a candidate protocol template corresponding to the message to be recognized, and determine whether the information indicating that the candidate protocol template has completed manual verification and correction has been received. If so, obtain the final protocol template according to the information, and perform protocol conversion configuration on the message to be recognized through the final protocol template.

[0033] In this embodiment, when a preset number of continuously received messages cannot match the corresponding protocol type and protocol template in the preset protocol library, the protocol with the highest adaptability of multi-modal features to the first multi-modal features is selected from the preset protocol library, and a candidate protocol template is created according to the selected protocol. Then, the candidate protocol template is sent to the operation and maintenance platform, and the operation and maintenance personnel perform verification and modification on it to obtain the final protocol template. Moreover, a new parser is set for the final protocol template.

[0034] In this embodiment, a semantic understanding and conversion module is created, and the message to be recognized is converted using the preset protocol template or the final protocol template and the parser to obtain a standardized control instruction output, thereby realizing communication control between different devices.

[0035] S4. Add the final protocol template to the preset protocol library.

[0036] In this embodiment, as Figure 2 shown, an online learning increment module is created. When the final protocol template is successfully used to convert the message to be recognized, incremental learning is triggered, and the final protocol template is added to the preset protocol library.

[0037] Please refer to Figure 5 , Embodiment 2 of the present invention is as follows: A method for adapting industrial protocols for multimodal recognition. Based on the first embodiment above, each preset protocol template in the preset protocol library includes a basic protocol template, a semantic mapping graph, a vendor extension, and a version evolution record table.

[0038] Among them, the basic protocol template mainly includes a protocol type, a meta-model structure, field verification rules, and standard semantic definitions; the protocol type specifies the communication protocol followed by data transmission, determines the data transmission format, communication method, and interaction rules. The protocol type determines the data collection method, such as TCP, UDP, HTTP, Modbus, MQTT, etc.; guides the logic of data parsing; and provides connection parameters, such as port number, timeout setting, authentication method, and other parameters.

[0039] The vendor extension includes a vendor ID, a private field mapping table, an extended bit interpretation rule, and a checksum algorithm library; the vendor ID is a unique identifier assigned to the device manufacturer to identify the products or services of a specific vendor; the private field mapping table is the correspondence between the vendor-defined data fields and the standard protocol fields, used to interpret the "private fields" not defined in the protocol. The extended bits are the undefined bits reserved in the protocol (such as flag bits, reserved bits), and the vendor can customize their meanings. The extended bit interpretation rule clarifies the semantics and usage of these bits. The checksum algorithm library contains the vendor-defined error detection and data integrity verification methods, used to supplement or replace the standard verification mechanism of the basic protocol.

[0040] The semantic mapping graph includes source protocol instructions, semantic similarity, and a transformation weight matrix. The source protocol instructions refer to the data fields, instruction sets, and operation specifications defined in the original data source (such as devices, APIs, files). It describes the physical representation and access method of the data. The semantic similarity measures the similarity degree of two data elements (such as fields, entities) in terms of business meaning, rather than simple syntactic matching. When there is a lack of explicit mapping rules, potential corresponding relationships can be automatically discovered through semantic analysis, and it can also solve the problems of different meanings with the same name or the same meaning with different names. The transformation weight matrix is a multi-dimensional array used to quantify the credibility and priority of different mapping paths, and solve the problems of polysemy and conflicts in semantic mapping.

[0041] Please refer to Figure 6 , the third embodiment of the present invention is: An industrial protocol adaptation terminal 1 for multimodal recognition, including a memory 2, a processor 3, and a computer program stored on the memory 2 and executable on the processor 3. When the processor 3 executes the computer program, it implements the method for adapting industrial protocols for multimodal recognition in the first or second embodiment In summary, the industrial protocol adaptation method and terminal for multi-modal recognition provided by the present invention create a multi-modal input source before protocol recognition, which can dock various heterogeneous data sources in the industrial scenario with a unified data entry. Whether it is the time-series data from sensors, device log texts, or binary packets transmitted over the network, they can all be efficiently obtained through this input source. By using multi-modal recognition and a preset protocol library in combination, it can support the dynamic recognition and matching of multiple industrial protocols, and then perform automated protocol conversion configuration, greatly reducing the manual configuration workload and improving the conversion efficiency. At the same time, the protocol template creation process that introduces manual verification and correction can effectively handle unknown protocols. The combination of automation and manual operation can quickly and accurately identify and convert multi-source heterogeneous data in a multi-protocol scenario, providing convenience for the interaction between different devices. In addition, by setting a private field mapping table for introducing manufacturer extension rules on the preset protocol template, the flexibility and compatibility of industrial protocol adaptation can be significantly improved. On the one hand, industrial devices of different manufacturers often have personalized function extensions and data format differences on the basis of following the general protocol standard. The private field mapping table can provide a dedicated management space for these manufacturer-specific protocol rules, avoiding protocol adaptation failures caused by rule conflicts and ensuring the effective interaction of device data of each manufacturer in a multi-protocol scenario.

[0042] The above are only the embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in the related technical field, shall be similarly included in the patent protection scope of the present invention.

Claims

1. An industrial protocol adaptation method for multimodal recognition, characterized in that It includes the following steps: S1. Extract the first multi-modal feature of the message to be recognized, and determine the protocol type of the message to be recognized according to the first multi-modal feature; S2. Determine whether there is a protocol in the preset protocol library that is the same as the protocol type of the message to be recognized. If so, select the preset protocol template corresponding to the protocol with the same protocol type, and perform protocol conversion configuration on the message to be recognized through the preset protocol template. Otherwise, execute S3; S3. Create a candidate protocol template corresponding to the message to be recognized, and determine whether the information indicating that the candidate protocol template has completed manual verification and correction is received. If so, obtain the final protocol template according to the information, and perform protocol conversion configuration on the message to be recognized through the final protocol template; The specific method for determining the protocol type of the message to be recognized according to the first multi-modal feature is: Recognize the first multi-modal feature through a hybrid model of a graph neural network and a Transformer to obtain the protocol type of the message to be recognized.

2. The industrial protocol adaptation method for multimodal recognition according to claim 1, wherein The specific method for creating a candidate protocol template corresponding to the message to be recognized is: Select the protocol with the highest adaptability of multi-modal features to the first multi-modal feature from the preset protocol library, and create a candidate protocol template according to the selected protocol.

3. The industrial protocol adaptation method for multimodal recognition according to claim 1, characterized in that Before S1, it further includes: S0. Create a multi-modal input source, and obtain the message to be recognized through the multi-modal input source.

4. The industrial protocol adaptation method for multimodal recognition according to claim 3, wherein The multi-modal input source includes at least one of network messages, device logs, and topology documents.

5. The industrial protocol adaptation method for multimodal recognition according to claim 1, characterized in that Before S1, it further includes: Set a private field mapping table for introducing manufacturer extension rules on the preset protocol template corresponding to each protocol type in the preset protocol library.

6. The industrial protocol adaptation method for multimodal recognition according to claim 1, wherein Before S1, it further includes: Set a version evolution record table on the preset protocol template corresponding to each protocol type in the preset protocol library; After the content of the preset protocol template is updated, update the corresponding version evolution record table.

7. The industrial protocol adaptation method for multimodal recognition according to claim 1, characterized in that After S3, it further includes: S4. Add the final protocol template to the preset protocol library.

8. A multimodal recognition-based industrial protocol adaptation method according to claim 1, characterized in that The extraction of the first multi-modal feature of the message to be recognized further includes: Extract features of the message to be recognized through a structure parser, a timing analyzer, and a text miner.

9. An industrial protocol adaptation terminal for multimodal recognition, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a multi-modal recognition-based industrial protocol adaptation method according to any one of claims 1 to 8.

Images