Data analysis method, related device and electronic equipment

Abstract

The invention provides a data analysis method, a related device and a computer program product, and the method comprises the steps: responding to the received to-be-analyzed data which does not indicate a communication protocol, and reading the message features of the to-be-analyzed data; based on the message features, determining a protocol type of a target communication protocol for analyzing the to-be-analyzed data; and in response to the fact that the protocol type is a standard protocol type, analyzing the to-be-analyzed data by using a standard analysis template corresponding to the target communication protocol, and generating analyzed data. Therefore, the protocol adaptation efficiency can be improved, and the access cost of new equipment and new devices can be reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for parsing data, related devices, and electronic equipment. Background Art

[0002] With the continuous development and popularization of intelligent technology, the demand for new device access is growing across all industries. In fields such as industrial automation, the Internet of Things, smart homes, and smart cities, interoperability between devices has become a key factor in improving production efficiency, optimizing resource allocation, and ensuring system stability and security.

[0003] Accordingly, the smooth and efficient integration of new devices directly impacts the overall system performance and stability, and thus product quality and service reliability. To achieve rapid device integration and efficient collaboration, access protocol configuration has become a key technical issue. For example, in the marine sector, protocol adaptation for multi-source, heterogeneous devices remains a core challenge in data collection as ships become more intelligent.

[0004] Therefore, how to improve the efficiency of protocol adaptation and reduce the access cost of new equipment and new devices is worthy of attention and an urgent need. Summary of the Invention

[0005] Various aspects of this application provide a data parsing method, related apparatus, and electronic device that can adaptively determine a parsing template based on the protocol type of the communication protocol corresponding to the data to be parsed, thereby completing automatic parsing and intelligent mapping of the data to be parsed. This can improve the efficiency of protocol adaptation and reduce the cost of integrating new equipment and devices.

[0006] In one aspect of the present application, a method for parsing data is provided, comprising: in response to receiving data to be parsed that does not indicate a communication protocol, reading message features of the data to be parsed; based on the message features, determining a protocol type of a target communication protocol for parsing the data to be parsed; in response to the protocol type being a standard protocol type, parsing the data to be parsed using a standard parsing template corresponding to the target communication protocol to generate parsed data.

[0007] Another aspect of the present application provides a device for parsing data, including: a message feature reading module, configured to read the message features of the data to be parsed in response to receiving the data to be parsed that does not indicate a communication protocol; a protocol type determination module, configured to determine the protocol type of the target communication protocol used to parse the data to be parsed based on the message features; a first data parsing module, configured to parse the data to be parsed using a standard parsing template corresponding to the target communication protocol in response to the protocol type being a standard protocol type, and generate parsed data.

[0008] Another aspect of the present application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for parsing data provided above.

[0009] In another aspect of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored. The computer program instructions can be executed by a processor to implement the method for parsing data provided above.

[0010] In another aspect of the present application, a computer program product includes a computer program having computer program instructions stored thereon. When the computer program is executed by a processor, the method for parsing data as provided above can be implemented.

[0011] In the solution provided by the embodiment of the present application, first, in response to receiving data to be parsed that does not indicate a communication protocol, the message characteristics of the data to be parsed are read; then, based on the message characteristics, the protocol type of the target communication protocol used to parse the data to be parsed is determined; finally, in response to the protocol type being a standard protocol type, the standard parsing template corresponding to the target communication protocol is used to parse the data to be parsed to generate parsed data. As a result, based on the protocol type of the communication protocol corresponding to the data to be parsed, the parsing template can be adaptively determined to complete the automatic parsing and intelligent mapping of the data to be parsed. As a result, the efficiency of protocol adaptation can be improved and the access cost of new equipment and new devices can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0014] Figure 1 A flowchart of a data parsing process provided in an embodiment of the present application;

[0015] Figure 2 A flowchart of another data parsing process provided in an embodiment of the present application;

[0016] Figure 3A flowchart illustrating a specific implementation of the data parsing process in a specific application scenario provided by an embodiment of the present application;

[0017] Figure 4 A schematic diagram of the structure of a device for parsing data provided in an embodiment of the present application;

[0018] Figure 5 The figure is a schematic diagram of the structure of an electronic device suitable for implementing the solution in the embodiment of the present application.

[0019] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0020] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0021] In a typical configuration of the present application, the terminal and the equipment of the service network each include one or more processors (CPUs), input / output interfaces, network interfaces and memories.

[0022] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0023] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. The information can be computer program instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc-read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0024] As discussed above, how to improve the efficiency of protocol adaptation and reduce the access cost of new equipment and devices is worthy of attention and an urgent need.

[0025] In some solutions, after a new device or apparatus is connected to a data system, a manually written communication protocol is provided to the data system for communicating with the new device or apparatus. This allows the data system to subsequently parse and identify the data provided by the new device or apparatus through this communication analysis. However, this approach not only requires a high level of manual effort, but also limits the ability to add new devices and apparatuses to manual work, resulting in high integration costs and making it difficult to integrate new devices and apparatuses flexibly and efficiently.

[0026] To this end, an embodiment of the present application provides a method for parsing data. The method first responds to receiving data to be parsed that does not indicate a communication protocol, reads the message characteristics of the data to be parsed; then, based on the message characteristics, determines the protocol type of the target communication protocol used to parse the data to be parsed; finally, in response to the protocol type being a standard protocol type, uses the standard parsing template corresponding to the target communication protocol to parse the data to be parsed to generate parsed data. Thus, based on the protocol type of the communication protocol corresponding to the data to be parsed, the parsing template can be adaptively determined to complete the automatic parsing and intelligent mapping of the data to be parsed. Thus, the efficiency of protocol adaptation can be improved and the access cost of new equipment and new devices can be reduced.

[0027] In practical scenarios, the execution subject of this method can be a user device, or a device formed by integrating a user device and a network device via a network, or an application running on such a device. User devices include but are not limited to computers, mobile phones, tablets, smart watches, wristbands, and other terminal devices. Network devices include but are not limited to network hosts, single network servers, multiple network server clusters, or cloud computing-based computer clusters. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a group of loosely coupled computers forming a virtual computer.

[0028] When the execution subject is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules, or as a single software or software module, and is not specifically limited here.

[0029] Figure 1 A data parsing process 100 provided in an embodiment of the present application is shown. The process 100 includes at least the following processing steps:

[0030] (Step) S101, in response to receiving data to be parsed that does not indicate a communication protocol, reading a message feature of the data to be parsed;

[0031] In an embodiment of the present application, the execution subject of the above-mentioned method (or process) for implementing data parsing, after receiving the data to be parsed, first determines whether a communication protocol has been pre-indicated for the data to be parsed. For example, after receiving the data to be parsed, the execution subject may determine the sending device that sent the data to be parsed. Then, it determines whether a communication protocol that can be used has been pre-configured with the sending device.

[0032] Accordingly, if such a communication protocol is configured, the execution entity can determine a corresponding parsing template based on the parsing rules indicated by the communication protocol, and then use the parsing template to complete the parsing of the data to be parsed.

[0033] If such a communication protocol is not configured or indicated, the execution entity can respond thereto by reading the message features of the data to be parsed. For example, the message features can be protocol header fields, such as Internet Protocol (IP) header, Transmission Control Protocol (TCP) header, User Datagram Protocol (UDP) header, etc. For example, the message features can also be identifier fields, such as function codes in the Modbus protocol, message identifiers in the Controller Area Network (CAN) protocol, etc. For example, the message features can be fixed data length fields and checksum fields. For example, the message features can be timestamps, synchronization sequence numbers, etc.

[0034] In some embodiments, when the execution subject reads the message features in this step, it can, as an alternative or substitute, read the message features of the data to be parsed through traffic sniffing.

[0035] For example, when the data to be parsed is input, the executing entity can use a traffic sniffing tool (for example, Wireshark, Tcpdump, Tshark, etc.) to capture the data stream of the data to be parsed, and then parse it to determine the contents included therein, such as the protocol header field (or protocol identifier), data bits, timestamp, etc., to obtain the above-mentioned message characteristics.

[0036] Therefore, by using the traffic sniffing method, after the execution subject receives the data to be parsed, it can quickly and comprehensively identify the data to be parsed, or in other words, the information included in the traffic, so as to determine the type of communication protocol and provide support for subsequent data processing and security analysis.

[0037] S102, determining a protocol type of a target communication protocol for parsing the data to be parsed based on the message characteristics;

[0038] In an embodiment of the present application, after reading the message features based on the above S101, the execution entity can use these message features to determine the protocol type of the target communication protocol used to parse the data to be parsed. For example, the execution entity can use the message features to determine the expression form of the data to be parsed, and then determine the protocol type of the corresponding target communication protocol to be used.

[0039] For example, the execution subject may determine the protocol type to which the corresponding communication protocol belongs based on whether it includes a protocol header field, a timestamp for verification, etc.

[0040] In some embodiments, the protocol type includes at least a standard protocol type. The communication protocol under the standard protocol type can be, for example, a common standard and public network communication protocol, such as TCP protocol, UDP protocol, etc.

[0041] In some embodiments, the protocol type may also include a vendor protocol type. A vendor protocol type communication protocol may be a protocol provided to the execution entity by the "vendor" that allows communication with the execution entity based on their long-term usage needs. Typically, communications under this vendor protocol type may be partially non-standard, such as proprietary CAN extended frames.

[0042] A private CAN extended frame typically refers to a frame format customized by a manufacturer based on a standard CAN extended frame to meet its own needs. This extended frame may involve modifications to the identifier, data format, control field, and other aspects to improve communication performance in specific application scenarios or meet specific functional requirements.

[0043] Therefore, through the communication protocol of the manufacturer protocol type, these "manufacturers" that are allowed to communicate can use non-standard protocols to communicate according to their needs, thereby improving the configuration flexibility of new equipment.

[0044] In some embodiments, protocol types may also include completely non-standard custom protocols, or proprietary protocols. For example, these are communication protocols that are completely customized by a manufacturer or organization and are not approved or defined by a standardization organization. Custom protocols are often tailored to specific application scenarios, hardware, or software requirements, and are generally not universal or compatible across manufacturers.

[0045] Therefore, by classifying custom protocol types, the execution entity can also complete the processing of communication protocols of such custom protocol types based on the corresponding processing strategy (the specific processing strategy will be discussed in detail below) and complete the parsing of the data to be parsed. This allows the execution entity to access and process new devices under such custom protocols.

[0046] In some embodiments, in the process of executing this step and determining the protocol type of the target communication protocol used to parse the data to be parsed, in order to improve the determination efficiency and quality of the execution entity, a communication protocol library (for example, a protocol fingerprint library) can also be pre-selected and maintained.

[0047] Accordingly, in the communication protocol library, the executing entity can choose to directly use the "protocol header field" as the dimension to mark the pre-collected standard communication protocols and / or manufacturer communication protocols to determine the protocol header fields corresponding to each standard communication protocol and manufacturer communication protocol.

[0048] Subsequently, if the execution subject's message features include a "protocol header field", the execution subject can use the protocol header field in the message features to query whether the communication protocol library includes the target standard communication protocol for parsing the data to be parsed, and whether it includes the target manufacturer communication protocol for parsing the data to be parsed, that is, to query whether the above-mentioned target communication protocol is the standard communication protocol or the manufacturer communication protocol in the communication protocol library (if so, the standard communication protocol that is hit can be called the "target standard communication protocol", and similarly, the manufacturer communication protocol that is hit can be called the "target manufacturer communication protocol").

[0049] In some embodiments, when it is difficult to collect protocol header fields in advance, or when it is desired to use more dimensional information to mark the standard communication protocols and / or manufacturer communication protocols in the communication protocol library, the communication protocol library can also be marked using the "verification method" as a dimension to mark the pre-collected standard communication protocols and / or manufacturer communication protocols separately or in combination with the "protocol header field". Thus, after completing the parsing of the message features, it is possible to adaptively query whether the communication protocol library includes the target standard communication protocol for parsing the data to be parsed, and whether it includes the target manufacturer communication protocol for parsing the data to be parsed, based solely on the "verification method", "protocol header field", or a combination of the "verification method" and "protocol header field".

[0050] For example, the inspection method can check whether the "data length field" in the message characteristics meets the length corresponding to the standard communication protocol or the manufacturer's communication protocol, whether the content recorded in the inspection field meets the preset content corresponding to the standard communication protocol or the manufacturer's communication protocol, etc.

[0051] Accordingly, if the executing entity can query the above-mentioned target standard communication protocol in the communication protocol library, the executing entity can respond to this and determine that the protocol type of the target communication protocol used to parse the data to be parsed is a standard protocol type (in fact, as discussed above, the target standard communication protocol is the target communication protocol).

[0052] Similarly, if the executing entity can query the above-mentioned target manufacturer communication protocol in the communication protocol library, the executing entity can respond to this and determine that the protocol type of the target communication protocol used to parse the data to be parsed is the manufacturer communication protocol (similarly, the target manufacturer communication protocol is the above-mentioned target communication protocol).

[0053] Accordingly, if the execution entity does not find the above-mentioned target standard communication protocol and the above-mentioned target manufacturer communication protocol in the communication protocol library, the execution entity can respond to this and determine that the protocol type of the target communication protocol used to parse the data to be parsed is a custom protocol type, that is, classify the target communication protocol as a custom protocol type.

[0054] Therefore, by maintaining the communication protocol library, the communication protocols under known standard protocol types and manufacturer protocol types can be maintained, so that the execution entity can use the communication protocol library to efficiently and accurately classify the target communication protocols used to parse the data to be parsed.

[0055] S103 , in response to the protocol type being a standard protocol type, parsing the data to be parsed using a standard parsing template corresponding to the target communication protocol to generate parsed data.

[0056] In an embodiment of the present application, if the protocol type of the target communication protocol is a standard protocol type, the execution entity may respond to this by using a standard parsing template corresponding to the target communication protocol to parse the data to be parsed and generate parsed data.

[0057] The standard parsing template can be pre-configured corresponding to the standard communication protocol to record the parsing rules of the corresponding standard communication protocol for the data to be parsed. Accordingly, because the standard communication protocol is often well-known and standardized, the execution entity can construct the standard parsing template through the parsing rules (description information) corresponding to the pre-maintained standard communication protocol. Alternatively, in some embodiments, if a certain standard communication protocol has been provided with a corresponding standard parsing template, the execution entity can choose to directly obtain the standard parsing template and use it.

[0058] For example, a parsing template, such as a standard parsing template, can record the format that data should have under the corresponding standard communication protocol (for example, the structure of the data frame, the length of the identifier, the location of the data field, etc.). A parsing template is typically a structured file or rule that describes how to extract each field in the data to be parsed according to the definition of the standard protocol. The parsing template defines the type, length, location, and other information of each field according to the provisions of the communication protocol.

[0059] For example, the parsing template may list the name, data type, length, parsing method, etc. of each field in a table format. Alternatively, in some embodiments, the parsing template may directly define the specific operations for parsing each data packet, such as byte order, bit field, field separator, etc.

[0060] Accordingly, in this step, after determining the target standard communication protocol, the executing entity can choose to call the standard parsing template corresponding to the target standard communication protocol, and use the standard parsing template to parse the data to be parsed under the target communication protocol, so as to extract the valid information expected to be transmitted (for example, sensor data) and generate parsed data.

[0061] Then, the method for parsing data provided by the present application first responds to receiving data to be parsed that does not indicate a communication protocol, reads the message characteristics of the data to be parsed; then, based on the message characteristics, determines the protocol type of the target communication protocol used to parse the data to be parsed; finally, in response to the protocol type being a standard protocol type, uses the standard parsing template corresponding to the target communication protocol to parse the data to be parsed to generate parsed data. Thus, based on the protocol type of the communication protocol corresponding to the data to be parsed, the parsing template can be adaptively determined to complete the automatic parsing and intelligent mapping of the data to be parsed. Thus, the efficiency of protocol adaptation can be improved and the access cost of new equipment and new devices can be reduced.

[0062] Correspondingly, in this way, the device (new device) can be hot-swapped at any time relative to the execution subject, and global service interruption will not be caused by the connection or disconnection of the device.

[0063] In some embodiments, if the execution entity can also divide the protocol type of the target communication protocol used to parse the data to be parsed into a manufacturer protocol type, then in such a case, the execution entity can adaptively complete the parsing of the data to be parsed under the manufacturer protocol type based on the "self-generated template" method, thereby enabling the data to be parsed under the manufacturer protocol type (or new equipment) to be efficiently connected to the execution entity without being directly and explicitly indicated and configured by humans as the (target) communication protocol.

[0064] For easier understanding, please refer to Figure 2 . Figure 2 Another data parsing process 200 provided in an embodiment of the present application is shown. The process 200 includes at least the following processing steps:

[0065] S201, in response to receiving data to be parsed that does not indicate a communication protocol, reading a message feature of the data to be parsed;

[0066] S202, determining a protocol type of a target communication protocol for parsing the data to be parsed based on the message characteristics;

[0067] The above S201-S202 and Figure 1 Steps S101-S102 shown are consistent. For the same contents, please refer to the corresponding parts of the previous embodiment and will not be repeated here.

[0068] S203, in response to the protocol type being a manufacturer protocol type, extracting key value segments from the data to be parsed based on entropy analysis and fuzzy matching algorithm;

[0069] Specifically, as discussed above, since the vendor protocol type is usually a protocol defined by a device or manufacturer and may not be a public standard protocol, the vendor protocol may have a unique data structure, encoding method, and identifier format.

[0070] Therefore, in this step, the execution subject may first calculate the entropy value of each byte in the data to be parsed. For example, the execution subject may estimate the entropy value by calculating the probability distribution of each byte.

[0071] Correspondingly, in the calculation results, if the entropy value is higher, it means that the uncertainty or information content of this part of the data is greater.

[0072] Then, the execution entity can extract the "high entropy segments" whose entropy values ​​are greater than or equal to a predetermined entropy threshold (for example, the entropy threshold can be determined based on the criteria of what is considered to be "key information"), and further extract the key value segments from the data to be parsed based on the fuzzy matching algorithm.

[0073] In the fuzzy matching algorithm, the executing entity can determine the parts that are closest to the known pattern from the "high entropy segment" based on a predetermined matching pattern (for example, a matching pattern constructed based on a known structure or data example of the manufacturer's protocol type. For example, certain fields may be numbers, timestamps, or device IDs, etc.), and use these parts as "key value segments."

[0074] S204, generating a temporary parsing template based on the key value segment;

[0075] Specifically, after determining the key value segments based on the above S203, the execution entity may choose to analyze the extracted key value segments and then assign a name to each field.

[0076] The execution entity then determines the data type corresponding to each field based on the data specifically included in the field. For example, if the data is an X-bit integer, and the ID of a new device or apparatus is an X-bit integer, then the field may correspond to the ID of the new device. For example, if the data is an Y-bit unsigned integer, and sensor data is typically a Y-bit unsigned integer, then the execution entity may determine that the field is "sensor data."

[0077] Then, the execution entity can determine the corresponding parsing rules for these fields by analyzing the actual data length and position of each field, and generate a temporary parsing template for parsing the key value segment, so that the execution entity can also complete the parsing in the form of a "parsing template".

[0078] The temporary parsing template may be a form of the above-mentioned parsing template, which can also record the format that the data should have under the corresponding standard communication protocol, and will not be repeated here.

[0079] In some embodiments, a template generation model trained based on sample key value segments and sample temporary parsing templates (for example, the sample generation model can be constructed using a long short-term memory network) can be selected to process the key value segments to generate corresponding temporary parsing templates. In this way, the pre-trained template generation model can be used to parse the key value segments with high quality and generate the required temporary parsing templates.

[0080] S205: Parsing the data to be parsed using the temporary parsing template to generate parsed data.

[0081] Specifically, after the temporary parsing template is generated, the execution entity may utilize the temporary parsing template similarly to the above discussion to process the data to be parsed according to the parsing rules indicated in the temporary parsing template to generate parsed data.

[0082] Therefore, by generating a temporary parsing template, the execution entity has the ability to adapt and self-parse the data to be parsed under the manufacturer's protocol type, so that new equipment and new devices of the manufacturer's protocol type can also be efficiently and flexibly connected to the execution entity.

[0083] In some embodiments, after generating the temporary parsing template, the execution entity may also choose to provide the temporary parsing template to the target device (eg, the management or maintenance party of the execution entity, or the provider of new equipment or new devices).

[0084] Accordingly, the user of the target device can make adjustments based on the temporary parsing template, generate adjustment information, and then feed back the adjustment information to the execution entity. In some embodiments, the adjustment information may include dimension information and / or alarm threshold information.

[0085] If the target device receives adjustment information for the temporary parsing template, the execution entity may respond to the adjustment information and update the temporary parsing template based on the adjustment information. Accordingly, the execution entity may, as an alternative or alternative, use the updated temporary parsing template to parse the data to be parsed and generate parsed data during the subsequent execution of S205.

[0086] Therefore, when the user or subscriber of the target device has different control or management requirements, for example, the user can conveniently and efficiently add the requirements by adjusting the temporary parsing template.

[0087] In some embodiments, for the custom protocol type, since it is often completely non-standard, the execution entity may also choose to provide a standardized interface (Common Application Binary Interface, CABI for short) so that the provider of the data to be parsed can provide a custom parsing template for parsing the data to be parsed by calling the standardized interface.

[0088] For example, the provider can encapsulate a custom parsing template in the form of a dynamic library (.so / dll) and provide it to the execution entity by calling CABI. Accordingly, the execution entity can generate parsed data by parsing the data to be parsed using the custom parsing template. This ensures the stability of the standard protocol while retaining expansion space for customized needs, allowing new devices and equipment based on custom communication protocols to be smoothly connected to the execution entity.

[0089] In some embodiments, for the case of custom protocol types, if the execution entity does not read the custom parsing template, the execution entity can also similarly choose to use a method such as generating a temporary parsing template to try to self-generate such a custom parsing template to improve the system's device compatibility.

[0090] Similarly, in the case of a manufacturer agreement type, if the provider of the new device or equipment also provides a parsing template that can be used, the execution entity can also choose to use the parsing template for parsing.

[0091] Based on any of the above embodiments, because the execution subject may be connected to multiple devices and apparatuses at the same time, in such a case, the execution subject may continuously receive multiple data to be parsed at the same time (or within a short period of time).

[0092] In such cases, the execution entity can choose to determine the data type corresponding to each piece of data to be parsed after determining the protocol type corresponding to each piece of data to be parsed, based on the content that protocol type typically communicates. For example, if communication protocol A is often used to communicate with sensor B, the execution entity can determine that the data under communication protocol A is "sensor data."

[0093] After determining the data type, the execution entity can sort the processing and parsing priorities of each data to be parsed according to the (predetermined) type priority, and the parsing order corresponding to each data to be parsed.

[0094] Then, the execution subject may sequentially call corresponding parsing templates (eg, target parsing template, temporary parsing template, custom parsing template) for each data to be parsed based on the parsing order to perform parsing sequentially.

[0095] As a result, more important and higher-priority data to be parsed can be processed first to avoid the situation where key data cannot be parsed in time due to the increase in data volume, which may lead to scenario risks (for example, collision risks, etc.).

[0096] Based on any of the above embodiments, the execution entity may also select a standard lightweight data exchange format, such as JSON (JavaScript Object Notation), as the data format of the parsed data during the process of generating the parsed data. This allows all types and sources of parsed data to be output as parsed data in a standardized format, so that each parsed data can be smoothly transferred within the execution entity and to devices connected to the execution entity.

[0097] In order to deepen the understanding, this application also combines a specific application scenario. Figure 3 A flowchart of the process of implementing data parsing in this specific application scenario is given. Figure 3 Process 300 is shown.

[0098] Specifically, in process 300, if the execution subject receives data to be parsed 310 (e.g., data to be parsed 310 does not directly indicate a communication protocol), the execution subject may first execute S301 to read the message characteristics of the data to be parsed. For example, based on S301, the execution subject may read message characteristics 315.

[0099] Then, the execution subject continues to execute S302 to determine the protocol type of the target communication protocol 320 for parsing the data to be parsed 310 based on the message feature 315 .

[0100] Correspondingly, if the target communication protocol 320 is a standard protocol type, the execution subject may choose to continue executing S303 , and parse the data to be parsed 310 using the standard parsing template 325 corresponding to the target communication protocol 320 to generate parsed data 350 .

[0101] If the target communication protocol 320 is a manufacturer protocol type, the execution subject may choose to continue executing S304 after S302, and extract the key value segment 330 from the data to be parsed 310 based on entropy analysis and fuzzy matching algorithm.

[0102] After S304 , the execution entity may continue to execute S305 to generate a temporary parsing template 335 based on the key value segment 330 .

[0103] After S305 , the execution entity may continue to execute S306 to parse the to-be-parsed data 310 using the temporary parsing template 335 to generate parsed data 350 .

[0104] If the target communication protocol 320 is a custom protocol type, the execution subject may choose to continue executing S307 after S302, and read the custom parsing template 340 packaged in the form of a dynamic library associated with the data to be parsed 310 through a standardized interface.

[0105] Then, after S307 , the execution body may continue to execute S308 to parse the to-be-parsed data 310 using the custom parsing template 340 to generate parsed data 350 .

[0106] The present application also provides a device for parsing data. The structure of the device is as follows: Figure 4The device 400 shown in FIG. 4 includes: a message feature reading module 410 configured to, in response to receiving data to be parsed that does not indicate a communication protocol, read the message features of the data to be parsed; a protocol type determination module 420 configured to, based on the message features, determine the protocol type of the target communication protocol used to parse the data to be parsed; and a first data parsing module 430 configured to, in response to the protocol type being a standard protocol type, parse the data to be parsed using a standard parsing template corresponding to the target communication protocol to generate parsed data.

[0107] In some embodiments, the device 400 also includes: a key value segment extraction module, configured to extract key value segments from the data to be parsed based on entropy analysis and fuzzy matching algorithm in response to the protocol type being a manufacturer protocol type; a temporary template generation module, configured to generate a temporary parsing template based on the key value segments; and a second data parsing module, configured to use the temporary parsing template to parse the data to be parsed to generate parsed data.

[0108] In some embodiments, the device 400 also includes: a temporary template providing module, configured to provide a temporary parsing template to the target device; a temporary template updating module, configured to update the temporary parsing template based on the adjustment information provided by the target device for the temporary parsing template in response to receiving the adjustment information, thereby obtaining an updated temporary parsing template, wherein the adjustment information includes: dimensional information and / or alarm threshold information; and the second data parsing module is further configured to parse the data to be parsed using the updated temporary parsing template to generate parsed data.

[0109] In some embodiments, the device 400 also includes: a custom template reading module, configured to read a custom parsing template encapsulated in the form of a dynamic library and associated with the data to be parsed through a standardized interface in response to the protocol type being a custom protocol type; a third data parsing module, configured to use the custom parsing template to parse the data to be parsed and generate parsed data.

[0110] In some embodiments, the protocol type determination module 420 includes: a communication protocol library query submodule, configured to query whether the communication protocol library includes a target standard communication protocol for parsing the data to be parsed and whether it includes a target manufacturer communication protocol for parsing the data to be parsed based on the protocol header field and / or verification method in the message characteristics; a protocol type determination submodule, configured to, in response to the communication protocol library including the target standard communication protocol, determine that the protocol type of the target communication protocol for parsing the data to be parsed is determined to be a standard protocol type; or, in response to the communication protocol library including the target manufacturer communication protocol, determine that the protocol type of the target communication protocol for parsing the data to be parsed is determined to be a manufacturer protocol type; or, in response to the communication protocol library not including the target standard communication protocol and the target manufacturer communication protocol, determine that the protocol type of the target communication protocol for parsing the data to be parsed is determined to be a custom protocol type.

[0111] In some embodiments, the message feature reading module 410 is further configured to, in response to receiving data to be parsed that does not indicate a communication protocol, read message features of the data to be parsed through traffic sniffing.

[0112] In some embodiments, the device 400 also includes: a data type determination module, configured to determine the data type of the data to be parsed based on the protocol type in response to the existence of multiple data to be parsed; and a parsing order determination module, configured to determine the parsing order corresponding to each data to be parsed based on type priority sorting.

[0113] In some embodiments, the data format of the parsed data is a standard lightweight data exchange format.

[0114] The embodiment of this device corresponds to the method embodiment shown in the above figures. First, in response to receiving data to be parsed that does not indicate a communication protocol, the message characteristics of the data to be parsed are read; then, based on the message characteristics, the protocol type of the target communication protocol used to parse the data to be parsed is determined; finally, in response to the protocol type being a standard protocol type, the standard parsing template corresponding to the target communication protocol is used to parse the data to be parsed to generate parsed data. As a result, based on the protocol type of the communication protocol corresponding to the data to be parsed, the parsing template can be adaptively determined to complete the automatic parsing and intelligent mapping of the data to be parsed. As a result, the efficiency of protocol adaptation can be improved and the access cost of new equipment and new devices can be reduced.

[0115] Based on the same inventive concept, an electronic device, a readable storage medium, and a computer program product are also provided in an embodiment of the present application. The method corresponding to the electronic device can be the method for parsing data in the aforementioned embodiment, and its principle of solving the problem is similar to that of the method. The electronic device provided in an embodiment of the present application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the aforementioned multiple embodiments of the present application.

[0116] An electronic device can be a user device, or a device formed by integrating a user device and a network device via a network, or an application running on any of the above devices. User devices include, but are not limited to, computers, mobile phones, tablets, smart watches, wristbands, and other terminal devices. Network devices include, but are not limited to, network hosts, single network servers, multiple network server clusters, or cloud computing-based computer clusters, and can be used to implement some of the processing functions required for setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computers.

[0117] Figure 5 The structure of an electronic device suitable for implementing the method and / or technical solution in the embodiment of the present application is shown. The electronic device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 into the random access memory (RAM) 503. In the RAM 503, various programs and data required for system operation are also stored. The CPU 501, ROM 502 and RAM 503 are connected to each other through a bus 505. An input / output (I / O) interface 504 is also connected to the bus 505.

[0118] The following components are connected to the I / O interface 504: an input section 506 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, etc.; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, etc., and a speaker; a storage section 508 including one or more computer-readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet.

[0119] In particular, the methods and / or embodiments in the embodiments of the present application can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the method of the present application are performed.

[0120] Another embodiment of the present application further provides a computer-readable storage medium and a computer program product, on which computer program instructions are stored. The computer program instructions can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of the present application.

[0121] Specifically, the present embodiment can adopt any combination of one or more computer-readable media.Computer-readable media can be computer-readable signal media or computer-readable storage media.Computer-readable storage media can be, for example, systems, devices or components including but not limited to electricity, magnetism, light, electromagnetic, infrared, or semiconductors, or any combination thereof.More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination thereof.In this document, computer-readable storage media can be any tangible medium containing or storing a program that can be used by an instruction execution system, device or device or used in combination with it.

[0122] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0123] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0124] The computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0125] The flow chart or block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the equipment, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code include one or more executable instructions for realizing the logical function of the specification. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated system for hardware that performs the function or operation of the specification, or can be implemented with a combination of dedicated hardware and computer instructions.

[0126] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules and units is only a logical function division. There may be other division methods in actual implementation. For example, with units as an example, for example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0128] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0129] In addition, the functional modules and units in the various embodiments of the present application may be integrated into a single processing module or unit, or each module or unit may exist physically separately, or two or more units may be integrated into a single module or unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional modules or units.

[0130] The above-mentioned integrated modules and units implemented in the form of software functional modules and units can be stored in a computer-readable storage medium. The above-mentioned software functional modules and units are stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute some steps of the methods of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program code.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0132] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

Claims

1. A method for parsing data, comprising: In response to receiving data to be parsed that does not indicate a communication protocol, reading a message feature of the data to be parsed; Determining, based on the message characteristics, a protocol type of a target communication protocol for parsing the data to be parsed; In response to the protocol type being a standard protocol type, the data to be parsed is parsed using a standard parsing template corresponding to the target communication protocol to generate parsed data.

2. The method according to claim 1, further comprising: In response to the protocol type being a manufacturer protocol type, extracting key value segments from the data to be parsed based on entropy analysis and fuzzy matching algorithms; generating a temporary parsing template based on the key value segment; The data to be parsed is parsed using the temporary parsing template to generate the parsed data.

3. The method according to claim 2, further comprising: Providing the temporary parsing template to the target device; In response to receiving adjustment information provided by the target device for the temporary parsing template, updating the temporary parsing template based on the adjustment information to obtain an updated temporary parsing template, wherein the adjustment information includes: dimension information and / or alarm threshold information; and The step of parsing the data to be parsed by using the temporary parsing template to generate the parsed data includes: The updated temporary parsing template is used to parse the data to be parsed to generate the parsed data.

4. The method according to claim 2, further comprising: In response to the protocol type being a custom protocol type, reading a custom parsing template associated with the data to be parsed and encapsulated in a dynamic library form through a standardized interface; The data to be parsed is parsed using the custom parsing template to generate the parsed data.

5. The method according to claim 4, wherein The determining of the protocol type of the target communication protocol for parsing the data to be parsed includes: Based on the protocol header field and / or the verification method in the message feature, query whether the communication protocol library includes a target standard communication protocol for parsing the data to be parsed, and whether it includes a target manufacturer communication protocol for parsing the data to be parsed; In response to the communication protocol library including the target standard communication protocol, determining that the protocol type of the target communication protocol used to parse the data to be parsed is the standard protocol type; or In response to the communication protocol library including the target manufacturer's communication protocol, determining that the protocol type of the target communication protocol used to parse the data to be parsed is the manufacturer's protocol type; or In response to the communication protocol library not including the target standard communication protocol and the target manufacturer communication protocol, the protocol type of the target communication protocol used to parse the data to be parsed is determined to be the custom protocol type.

6. The method according to claim 1, wherein Reading the message features of the data to be parsed includes: The message characteristics of the data to be parsed are read through traffic sniffing.

7. The method according to claim 1, further comprising: In response to the presence of a plurality of data to be parsed, determining a data type of the data to be parsed based on the protocol type; Based on the type priority sorting, the parsing order corresponding to each of the to-be-parsed data is determined.

8. The method according to any one of claims 1 to 7, wherein The data format of the parsed data is a standard lightweight data exchange format.

9. A device for parsing data, comprising: a message feature reading module, configured to, in response to receiving data to be parsed that does not indicate a communication protocol, read message features of the data to be parsed; a protocol type determination module configured to determine a protocol type of a target communication protocol for parsing the data to be parsed based on the message characteristics; The first data parsing module is configured to, in response to the protocol type being a standard protocol type, parse the data to be parsed using a standard parsing template corresponding to the target communication protocol to generate parsed data.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images