Implementation method for efficient code stream analysis in SCADA (supervisory control and data acquisition) system

Abstract

The invention discloses a method for realizing high-efficiency code stream analysis in an SCADA (supervisory control and data acquisition) system, which relates to the technical field of monitoring and data acquisition and remarkably improves the efficiency and real-time performance of data analysis by means of automatically generating an analysis library, optimizing a code stream capture mechanism, quickly matching an analysis process, visually displaying a tree structure and the like. Meanwhile, the complexity of system maintenance is reduced, the reliability of the system is enhanced, system faults caused by manual analysis errors are reduced, the accuracy and reliability of data processing are improved, the innovation is beneficial to improving the overall performance and user experience of the SCADA system, and the system can better meet the requirements of the modern industrial environment.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring and data acquisition, and in particular to a method for realizing efficient code stream parsing in a SCADA system. Background Art

[0002] In modern SCADA systems, real-time data monitoring and control functions are the core components. SCADA systems are usually divided into modules during the development process, and the final complete system consists of multiple independent and closely connected modules. These modules are connected through data streams and realize efficient exchange of information. The development and debugging of modules often require the collaboration of multiple groups or departments, and the entire process usually requires the participation of multiple people in order to successfully complete the debugging task. In this process, in order to ensure the efficient operation of the system, the transmitted data must be effectively parsed and processed. However, traditional code stream parsing methods often face some technical challenges when facing complex structural information and multi-module collaboration. Therefore, it is particularly important to invent a method for implementing efficient code stream parsing in SCADA systems.

[0003] The prior art also has the following defects, which are specifically reflected in:

[0004] 1. In the prior art, there are significant differences in the data structures and transmission formats used by various modules, not only in terms of data types and structures, but also in terms of serialization and encoding methods. Traditional parsing methods are often difficult to adapt to this diversity, which reduces the efficiency of data parsing and increases the probability of data loss or parsing errors. The SCADA system has strict requirements on data processing and response time. The prior art takes a long time to package and process data, especially when processing high-frequency data streams, which will cause response delays, thereby reducing the system's real-time monitoring capabilities and overall performance.

[0005] 2. The modular design of modern SCADA systems usually requires the collaboration of multiple development teams, but the modules developed by each team are not unified in terms of function and message definition, which increases the complexity of debugging and integration, and reduces the accuracy and consistency of data during the data transmission process between modules. In a multi-department collaboration environment, the interaction between different modules is complex, which increases the difficulty of the debugging and maintenance process. Developers often need to spend a lot of time to find and fix potential problems, reducing work efficiency.

[0006] 3. The existing system lacks visualization capabilities. Although the system can complete data analysis, it lacks effective visualization display methods, which makes it difficult for users to intuitively understand and analyze the specific content and status of the data flow, increasing the difficulty for users to monitor and analyze data. The existing parsing library relies on manual definition, which increases the cost of maintenance. When the system is updated, it is necessary to manually adjust the parsing logic, which reduces work efficiency, increases the probability of errors, and thus reduces the stability and reliability of the system. Summary of the invention

[0007] The object of the present invention is to provide a method for implementing efficient code stream parsing in a SCADA system, thereby solving the problems existing in the background technology.

[0008] To solve the above technical problems, the present invention adopts the following technical solution: The present invention provides a method for implementing efficient code stream parsing in a SCADA system, comprising: step 1, message ID and message body definition.

[0009] Step 2: Generate a parsing library.

[0010] Step 3: Connect to the server and unpack the message.

[0011] Step 4: Locate the message structure information.

[0012] Step 5: Generate description objects in real time.

[0013] Step 6: Fill in the Description object field.

[0014] Step 7: Tree structure display.

[0015] Preferably, the message ID and message body definition are specifically implemented by using a standard format to define the ID and structure of each message, including field name, data type, length and order.

[0016] Preferably, the generating of the parsing library is specifically implemented by: developing a code generation tool, automatically generating the parsing library according to the definition file, and establishing a mapping relationship between the message ID and the message structure.

[0017] Preferably, the method of connecting to the server and unpacking the message is specifically implemented by: connecting to the server through socket programming, continuously monitoring the incoming code stream, unpacking the message according to the protocol, and decomposing the code stream into independent messages.

[0018] Preferably, the positioning message structure information is specifically implemented by: building an efficient indexing mechanism to quickly search for the structure information corresponding to the message ID.

[0019] Preferably, the real-time generation of description objects is specifically implemented by: utilizing a reflection mechanism to dynamically create description objects according to structure information, and using a dynamic type system to flexibly create and modify description objects.

[0020] Preferably, the method of filling the description object field is as follows: processing the difference between the network byte order and the host byte order, ensuring the correct byte order of the multi-byte data type, parsing the value from the code stream according to the offset and data type of the field and filling it into the corresponding field of the description object.

[0021] Preferably, the tree structure display is specifically implemented by developing a tree structure UI component to display the hierarchical structure of the description object.

[0022] The beneficial effects of the present invention are:

[0023] 1. In the present invention, the parsing efficiency is improved. By automatically generating a parsing library and a message ID mapping relationship, the parsing speed of the code stream data is optimized, ensuring that the system can quickly respond to data under real-time requirements and reduce development and maintenance costs. The automated parsing and dynamic update mechanisms are adopted to reduce the maintenance workload of developers when message definitions and message formats change, reduce labor costs and error rates, enhance system reliability, accurately match messages, and automatically generate field information, reduce system failures caused by manual parsing errors, and improve the accuracy and reliability of data processing.

[0024] 2. In the present invention, the inter-module collaboration capability is improved. Through clear message definitions and automatic mapping relationships, the collaboration between different modules and departments is promoted, the scalability and adaptability of the system are enhanced, the visual display is optimized, and the parsing results are displayed in a tree structure, so that users can understand complex data structures more intuitively, improve the efficiency of data analysis, and facilitate quick decision-making; reduce memory usage, and optimize memory usage during the parsing process to ensure that the system can remain stable when processing high-frequency or large-capacity data, avoiding performance bottlenecks caused by insufficient memory.

[0025] 3. In the present invention, multiple data types are supported, and multiple data formats and structures are supported in design. It can flexibly respond to the data requirements of different modules, enhance the versatility of the system to achieve real-time monitoring and decision-making, and the fast data parsing capability meets the SCADA system's demand for real-time monitoring, allowing operators to obtain key information and make decisions in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 The present invention is a schematic flow chart of the steps for implementing the method.

[0028] Figure 2 Implement a flow chart for your code.

[0029] Figure 3 Describe the UML class diagram for the message.

[0030] Figure 4 Display a flow chart for a tree structure.

[0031] Figure 5 Display the core class diagram for the tree structure. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Reference Figure 1 As shown, the present invention provides a method for implementing efficient code stream parsing in a SCADA system, including: step one, message ID and message body definition.

[0034] In a specific embodiment, the message ID and message body definition are specifically implemented by using a standard format to define the ID and structure of each message, including field name, data type, length, and order.

[0035] It should be noted that the version number needs to be added to ensure backward compatibility when updating.

[0036] In order to reduce the maintenance work of developers on the mapping relationship between message IDs and messages, certain constraints are imposed on message definitions, as follows:

[0037] Define a macro to represent the message ID: Define a macro MSG_ID to represent the message ID. The macro accepts the module and ID as parameters and generates a unique identifier.

[0038] Message structure definition: For each message ID, define a corresponding structure to describe the specific content of the message, and add a base structure field. The base field has a member id to save the ID generated by the above macro definition.

[0039] When parsing the message definition, the system records all message IDs that match the MSG_ID macro definition and associates them with the corresponding structure names. When matching the message ID, the system looks for the corresponding string, such as "SYS_INFO_REQ", and then appends "_STRU" to the string to form the final message structure name.

[0040] Step 2: Generate a parsing library.

[0041] In a specific embodiment, the generation of the parsing library is specifically implemented by developing a code generation tool, automatically generating the parsing library according to the definition file, and establishing a mapping relationship between the message ID and the message structure.

[0042] It should be noted that the use of template engines to quickly generate code makes it easy to regenerate the parsing library when the definition file is updated, without the need for manual coding.

[0043] The parsing library is generated by parsing the contents of the C++ header file (.h) and outputting the parsing results in JSON format to the specified file. Specifically, it extracts the information of the structure, enumeration, and macro definition in the header file. Through the CppHeaderParser library, read and parse the contents of the C++ header file to obtain the class, attribute, enumeration, and macro definition. Convert the parsed information into a structured JSON format for subsequent processing or viewing. During the parsing process, if an error is encountered, the error information will be recorded and other files will continue to be processed.

[0044] The specific generation process of generating the parsing library is as follows: S1, parsing of the message definition file, S2, creation and filling of global variables, S3, processing of the parsing results, S4, storage of the parsing library, and S5, performance optimization.

[0045] The parsing of the message definition file requires parsing the message definition file to extract definition information such as macro information, enumeration type and structure type. These files may be scattered in multiple files, so it is necessary to traverse all files under the message definition folder to ensure that all related message definitions are parsed.

[0046] The creation and filling of the global variables define a global dictionary to store all message definition information parsed from the message definition file. This global variable will serve as a central repository for storing parsing results. For example, a hash table or structure array can be used in C language.

[0047] The processing of the parsing results, for each successfully parsed message definition file, creates a new type according to typedef and saves it to the global dictionary. At the same time, macro definitions and enum definitions are created and saved in the global dictionary. The integer value in the macro definition is calculated, and a message ID is generated according to specific rules. These message IDs will be used to identify specific message types. Structure definitions are created and saved in the global dictionary. These structure definitions are simple descriptive information used for the generation of subsequent code stream parsing mapping relationships. The parsed macro information, enumeration type, structure type, and message ID are integrated to generate a mapping relationship between a message ID and a structure message. This mapping relationship is the core of the parsing library, which allows the system to quickly find the corresponding structure definition based on the message ID.

[0048] The parsing library is saved by using the populated global dictionary as the parsing library and saving it to the hard disk in JSON format or other suitable data formats. In this way, the parsing library can be quickly loaded and queried at runtime.

[0049] The performance optimization mentioned above takes into account the performance problem. The traditional parsing library based on relational database may encounter performance bottlenecks when processing large structures. Therefore, consider using a key-value based parsing library implementation solution, which can significantly reduce memory usage and parsing time and improve system response speed.

[0050] refer to Figure 2 , the code can be divided into several main parts:

[0051] a. Import the required modules and function definitions.

[0052] b. File parsing logic, output the file name being parsed, create a dictionary output_json to store the parsing results, and use the CppHeaderParser class to parse the header file. If an error occurs during the parsing process, the exception is captured and the error information is recorded in output_json.

[0053] c. Extract information, extract enumeration and typedef definitions from header files, and perform necessary mapping processing to extract macro definitions into a dictionary with the key being the macro name and the value being the macro definition.

[0054] d. Parse the structure, traverse the C++ classes, extract the name and public properties of each class, process each property, and skip if the property is static. Update the property type according to the property type and the previously defined typedefs. Add all extracted properties to the props list of the structure. Add the parsed structure information to the structs list.

[0055] e. Create the final output, add the parsed results of the file path, structure, macro, and enumeration to output_json, and return the generated JSON object.

[0056] f. Main program logic. In the main function, define a result list. Traverse the command line parameters and check whether each parameter is a valid C++ header file or directory. For each .h file, call parseFile() to parse it and add the result to the result list. If it is a directory, use os.walk() to traverse all files, find the header file and parse it. Finally, write the result to the specified JSON file.

[0057] By automatically parsing and extracting definition information from C++ header files, it is easy to use and analyze. By structuring the data into JSON format, it can simplify the subsequent processing of the code and facilitate interaction with other tools or programs.

[0058] In the present invention, the parsing efficiency is improved, and the parsing speed of the code stream data is optimized by automatically generating a parsing library and a message ID mapping relationship, ensuring that the system can quickly respond to data under real-time requirements and reduce development and maintenance costs. The automated parsing and dynamic update mechanisms are adopted to reduce the maintenance workload of developers when message definitions and message formats change, reduce labor costs and error rates, enhance system reliability, accurately match messages, and automatically generate field information, reduce system failures caused by manual parsing errors, and improve the accuracy and reliability of data processing.

[0059] Step 3: Connect to the server and unpack the message.

[0060] In a specific embodiment, the method of connecting to the server and unpacking the message is as follows: connecting to the server through socket programming, continuously monitoring the incoming code stream, unpacking the message according to the protocol, and decomposing the code stream into independent messages.

[0061] It should be noted that the protocol has a fixed length, a delimiter, and a CRC check.

[0062] In order to achieve fast code stream capture in the SCADA system, the following specific practical solutions can be adopted:

[0063] 1. Design of network communication module, using Socket programming to establish connection with the server and continuously monitor the incoming code stream. A non-blocking IO (Input / Output) model can be used to improve efficiency, allowing the system to process other tasks while waiting for network response.

[0064] 2. Message unpacking logic, according to the communication protocol of the SCADA system, implement the message unpacking logic. This usually involves identifying the start and end identifiers of the message, and processing the delimiter or length field between messages.

[0065] 3. An efficient data parser, sends a protocol parser to parse the received data layer by layer according to the protocol specification, from the physical layer to the application layer, and extract useful information.

[0066] 4. Concurrency control: In high-concurrency scenarios, use current limiting algorithms such as Semaphore or RateLimiter to control the number of concurrency to ensure stable system operation.

[0067] 5. Exception handling: Capture possible exceptions and use try...catch statements or Promise to handle exceptions in asynchronous operations to ensure stability and reliability during code stream capture.

[0068] Step 4: Locate the message structure information.

[0069] In a specific embodiment, the positioning message structure information is specifically implemented by building an efficient indexing mechanism to quickly search for the structure information corresponding to the message ID.

[0070] It should be noted that the index mechanism is such as a hash table. A cache strategy is implemented for frequently queried message IDs and their structure information to reduce search time.

[0071] Step 5: Generate description objects in real time.

[0072] In a specific embodiment, the real-time generation of description objects is specifically implemented by: using a reflection mechanism to dynamically create description objects according to structure information, and using a dynamic type system to flexibly create and modify description objects.

[0073] refer to Figure 3 , the description information of the message in the parsing library, the fields in the message body may be basic types, for nested structure messages or array messages, there is only the message name, no detailed information of the message, the user needs to get the specific message, so the message needs to be expanded to obtain the detailed information of each field. Study the structure of the message, the message contains basic messages, and other messages may be nested. According to the design pattern, the composite pattern can be used to implement the detailed definition of the message.

[0074] Step 6: Fill in the Description object field.

[0075] In a specific embodiment, the method for filling the description object field is as follows: processing the difference between the network byte order and the host byte order, ensuring the correct byte order of the multi-byte data type, parsing the value from the code stream according to the offset and data type of the field and filling it into the corresponding field of the description object.

[0076] Step 7: Tree structure display.

[0077] In a specific embodiment, the tree structure display is specifically implemented by developing a tree structure UI component to display the hierarchical structure of the description object.

[0078] It should be noted that interactive operations such as expanding or collapsing tree nodes and searching for specific messages are provided to enhance the user experience and ensure that the UI is updated in real time to reflect the latest data.

[0079] In the present invention, the inter-module collaboration capability is improved, and through clear message definitions and automatic mapping relationships, the collaboration between different modules and departments is promoted, the scalability and adaptability of the system are enhanced, and the visual display is optimized. The parsing results are displayed in a tree structure, so that users can understand complex data structures more intuitively, improve the efficiency of data analysis, and facilitate quick decision-making; reduce memory usage, and by optimizing memory usage during the parsing process, ensure that the system can remain stable when processing high-frequency or large-capacity data, avoiding performance bottlenecks caused by insufficient memory.

[0080] This implementation uses a tree structure based on the QT MVC architecture to display message details. Figure 4 , the implementation process is as follows:

[0081] A. The user clicks on the bitstream corresponding to an item and obtains detailed information about the bitstream through the above bitstream parsing solution.

[0082] B. Bind the code stream and the expanded detailed information into a Model.

[0083] C. Organize data in the Model object and provide an interface to the outside world.

[0084] D. QTreeView displays through the interface provided by Model. Figure 5 .

[0085] The commonly used parsing library generation solution based on relational database is adopted. When parsing large structures such as those with the number of domains reaching the 100,000 level, the memory usage reaches the 100 MB level and the parsing time reaches the minute level. When the user clicks on the interface to view detailed information, it will be stuck for several minutes from parsing the code stream to displaying it. The implementation solution of the key-value-based parsing library reduces the memory usage to tens of MB and the parsing time to the ms level. It takes less than 1 second for users to go from clicking on the code stream to parsing the code stream and finally displaying it in a tree structure. The user interface has a slight lag, but it does not affect the use, and the operation fluency is within an acceptable range.

[0086] Copied variables can be reused between different groups and projects, which enables the reuse of variables between projects and groups and greatly improves programming efficiency.

[0087] In the present invention, multiple data types are supported, and multiple data formats and structures are supported in design. It can flexibly respond to the data requirements of different modules, enhance the versatility of the system to achieve real-time monitoring and decision-making, and the fast data parsing capability meets the SCADA system's demand for real-time monitoring, allowing operators to obtain key information and make decisions in a timely manner.

[0088] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they shall all fall within the protection scope of the present invention.

Claims

1. A method for implementing efficient code stream parsing in a SCADA system, characterized in that: include: Step 1: Message ID and message body definition; Step 2: Generate parsing library; Step 3: Connect to the server and unpack the message; Step 4: Locate the message structure information; Step 5: Generate description objects in real time; Step 6: Fill in the description object field; Step 7: Tree structure display.

2. The method for implementing efficient code stream parsing in a SCADA system according to claim 1, characterized in that: The message ID and message body definition are specifically implemented as follows: using a standard format to define the ID and structure of each message, including field name, data type, length and order.

3. The method for implementing efficient code stream parsing in a SCADA system according to claim 1, characterized in that: The specific implementation method of generating the parsing library is as follows: developing a code generation tool, automatically generating the parsing library according to the definition file, and establishing a mapping relationship between the message ID and the message structure.

4. The method for implementing efficient code stream parsing in a SCADA system according to claim 1, characterized in that: The method of connecting to the server and unpacking the message is as follows: connecting to the server through socket programming, continuously monitoring the incoming code stream, unpacking the message according to the protocol, and decomposing the code stream into independent messages.

5. The method for implementing efficient code stream parsing in a SCADA system according to claim 1, characterized in that: The method for locating the message structure information is as follows: constructing an efficient indexing mechanism to quickly search for the structure information corresponding to the message ID.

6. The method for implementing efficient code stream parsing in a SCADA system according to claim 1, characterized in that: The real-time generation of description objects is specifically implemented by: utilizing the reflection mechanism to dynamically create description objects according to structure information, and using the dynamic type system to flexibly create and modify description objects.

7. The method for implementing efficient code stream parsing in a SCADA system according to claim 1, characterized in that: The method for filling the description object field is as follows: processing the difference between the network byte order and the host byte order, ensuring the correct byte order of the multi-byte data type, parsing the value from the code stream according to the offset and data type of the field and filling it into the corresponding field of the description object.

8. The method for implementing efficient code stream parsing in a SCADA system according to claim 1, characterized in that: The tree structure display is specifically implemented by developing a tree structure UI component to display the hierarchical structure of the description object.

Images