Communication protocol reversing method based on digital power grid public document

Abstract

The embodiment of the invention provides a communication protocol reversing method based on a digital power grid public document, and the method comprises the steps: determining a target document comprising a communication protocol; inputting each text block included in the target document into a first network model to mark each text block to obtain a first tag sequence; converting each text block into a text vector through a second network model, and marking the text vector to obtain a second tag sequence; optimizing the first tag sequence and the second tag sequence to obtain a target tag sequence; and determining a finite-state machine of the communication protocol based on the target label sequence, wherein reversing of the communication protocol is completed. Through the method and the device, the problem of low reverse efficiency of the communication protocol in the prior art is solved, and the effect of improving the reverse efficiency and accuracy of the communication protocol is achieved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of information security, and in particular, to a communication protocol reversal based on a public document of a digital power grid. Background Art

[0002] Reverse engineering of communication protocols is a critical task, for example, in the field of digital grid communications, it involves parsing and understanding the structure and characteristics of digital grid protocols. Traditional methods mainly rely on manual analysis and interpretation of protocol specification documents, however, this method has the problems of low efficiency, long time consumption and limited by the level of artificial intelligence.

[0003] It can be seen from this that the related art has the problem of low reverse efficiency in determining the communication protocol.

[0004] Currently, no effective solution has been proposed for the above-mentioned problems existing in the related technologies. Summary of the invention

[0005] The embodiment of the present invention provides a communication protocol reversal based on a public document of a digital power grid, so as to at least solve the problem of low efficiency of the communication protocol reversal existing in the related art.

[0006] According to one embodiment of the present invention, a communication protocol reverse engineering method based on a digital power grid public document is provided, comprising: determining a target document including a communication protocol; inputting each text block included in the target document into a first network model to mark each text block to obtain a first label sequence, wherein the first label sequence includes each text block and label information of the text block; converting each text block into a text vector through a second network model, and marking the text vector to obtain a second label sequence, wherein the second label sequence includes each text vector and label information of the text vector; optimizing the first label sequence and the second label sequence to obtain a target label sequence; determining a finite state machine of the communication protocol based on the target label sequence to complete the reverse engineering of the communication protocol.

[0007] According to another embodiment of the present invention, a communication protocol reverse engineering device based on a digital power grid public document is provided, comprising: a first determination module, used to determine a target document including a communication protocol; a first marking module, used to input each text block included in the target document into a first network model to mark each text block to obtain a first label sequence, wherein the first label sequence includes each text block and label information of the text block; a second marking module, used to convert each text block into a text vector through a second network model, and mark the text vector to obtain a second label sequence, wherein the second label sequence includes each text vector and label information of the text vector; an optimization module, used to optimize the first label sequence and the second label sequence to obtain a target label sequence; a second determination module, used to determine a finite state machine of the communication protocol based on the target label sequence to complete the reverse engineering of the communication protocol.

[0008] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0009] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0010] According to yet another embodiment of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the method described in each embodiment of the present application are implemented.

[0011] Through the present invention, a target document including a communication protocol is determined; each text block included in the target document is input into the first network model to mark each text block to obtain a first label sequence, wherein the first label sequence includes each text block and label information of the text block; each text block is converted into a text vector through the second network model, and the text vector is marked to obtain a second label sequence, wherein the second label sequence includes each text vector and label information of the text vector; the first label sequence and the second label sequence are optimized to obtain a target label sequence; and a finite state machine of the communication protocol is determined based on the target label sequence. Since each text block included in the target document can be input into the first network model to mark the text block to obtain a first label sequence, the text block can also be converted through the second network model to obtain a text vector, and the text vector is marked to obtain a second label sequence, the first label sequence and the second label sequence are optimized to obtain a target label sequence, and the finite state machine of the communication protocol is determined according to the target label sequence to complete the reverse of the communication protocol. The first label sequence is obtained through the first network model, and the second label sequence is obtained through the second network model, which improves the speed of determining the label sequence, optimizes the first label sequence and the second label sequence, determines the finite state machine according to the optimized target label training, and determines the finite state machine using the optimized label sequence, which is conducive to improving the accuracy of determining the finite state machine. Therefore, the problem of low efficiency of communication protocol reverse engineering existing in the related art can be solved, and the effect of improving the efficiency and accuracy of communication protocol reverse engineering can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a hardware structure block diagram of a mobile terminal of a communication protocol reverse method based on a public document of a digital power grid according to an embodiment of the present invention;

[0013] Figure 2 is a flow chart of a communication protocol reverse engineering method based on a digital power grid public document according to an embodiment of the present invention;

[0014] Figure 3 is a flow chart of a communication protocol reverse method based on a digital power grid public document according to a specific embodiment of the present invention;

[0015] Figure 4 It is a structural block diagram of a communication protocol reverse device based on a digital power grid public document according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0017] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0018] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of a communication protocol reverse engineering method based on a public document of a digital power grid according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0019] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the communication protocol reverse method based on the public document of the digital power grid in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0020] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0021] In this embodiment, a communication protocol reverse engineering method based on the public documents of the digital power grid is provided. Figure 2 is a flow chart of a communication protocol reverse engineering method based on a digital power grid public document according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0022] Step S202, determining a target document including a communication protocol;

[0023] Step S204: input each text block included in the target document into the first network model to label each text block to obtain a first label sequence, wherein the first label sequence includes each text block and label information of the text block;

[0024] Step S206, converting each of the text blocks into a text vector through a second network model, and labeling the text vector to obtain a second label sequence, wherein the second label sequence includes each of the text vectors and label information of the text vectors;

[0025] Step S208, optimizing the first tag sequence and the second tag sequence to obtain a target tag sequence;

[0026] Step S210, determining the finite state machine of the communication protocol based on the target tag sequence, and completing the reversal of the communication protocol.

[0027] In the above embodiments, the communication protocol inverse method can be applied in the field of digital power grid, in the field of network communication, and in other fields where standard protocols exist. When the communication protocol inverse method is applied in different fields, the types of communication protocols may be different. For example, when applied in the field of digital power grid, the communication protocol may be a digital power grid communication protocol, and the target document is a document including the digital power grid communication protocol. When applied in the field of network communication, the communication protocol may be a network communication protocol, and the target document is a document including the network communication protocol. In the invention, the determination method of the finite state machine is applied in the field of digital power grid, the communication protocol is a digital power grid communication protocol, and the target document is a document including the digital power grid communication protocol.

[0028] In the above embodiment, the target document may be a public document in the field of digital power grid, including standards, protocols, research papers, technical forums, blogs, etc. The target document may include specifications and descriptions related to various digital power grid communication protocols. Target documents related to digital power grid can be collected in real time from various public sources in the field of digital power grid, such as standard organization websites, academic paper databases, technical forums, etc., to ensure that the latest and comprehensive digital power grid document resources can be obtained, providing a sufficient material basis for the analysis and processing of subsequent steps.

[0029] After the above embodiment, after obtaining the target document, the target document can be segmented, such as segmenting text paragraphs to obtain multiple text blocks. Each text block is input into a sequence-to-sequence model, and the sequence-to-sequence model may include a first network model and a second network model. The initial sequence-to-sequence model may be first trained to parse the digital power grid specification document and mark key information, such as states, events, variables, etc. Both the first network model and the second network model may adopt a BIO (beginning, internal, external) label system, which can accurately identify entities in the text and assign corresponding labels to them. The first network model may be a training convolutional neural network conditional random field, and the second network model may be a neural CRF model based on BERT embedding. The initial models of the first network model and the second network model may be trained through a large number of digital power grid documents to adapt to the language characteristics of the digital power grid field. The training of the first network model, i.e., the convolutional neural network conditional random field, may process a set of extracted features on each text unit to learn sequence-to-sequence mapping. It models the prediction as a probabilistic graph model, specifically considering the sequential dependencies in the prediction. The second network model, a neural CRF model based on BERT embedding, can be trained by enhancing the BERT encoder with GRU and CRF layers. The BERT encoder is used to create a chunk-level representation from a sequence of words, and then the GRU is used to calculate the sequence of embedded text units to obtain the representation. Among them, the neural CRF model based on BERT embedding can use the collected digital power grid document information to convert the domain-specific technical language into distributed word representations.

[0030] In the above embodiment, the output of the sequence-to-sequence model, i.e., the first label sequence and the second label sequence, can be post-processed and optimized, including removing noise, repairing errors, optimizing structure, etc., to obtain a target label sequence. For example, rules can be applied to correct some simple situations that the model fails to recognize, such as the extraction of states, events, and variables, to ensure the accuracy and completeness of the extracted information. Then, the various components of the finite state machine, including the state set, input-output set, initial state, and state transition set, can be extracted from the output of the post-processed and optimized sequence-to-sequence model, i.e., the target label sequence.

[0031] In the above embodiment, the determined finite state machine can be applied in the reverse engineering of the communication protocol. When performing the reverse engineering of the communication protocol, the finite state machine can be used to analyze and model the behavior of the protocol. By observing the data interaction and state transfer of the protocol, it can be abstracted into a finite state machine model, thereby helping to understand the working principle and design logic of the protocol. At the same time, the finite state machine can also be used to verify the results of reverse engineering. By simulating and testing the finite state machine model, the accuracy and completeness of the analysis results can be verified. Therefore, the reverse engineering of the communication protocol is closely related to the finite state machine. The finite state machine provides a formalized modeling and analysis tool for reverse engineering, which helps to understand and crack the working principle of the communication protocol.

[0032] Through the present invention, a target document including a communication protocol is determined; each text block included in the target document is input into the first network model to mark each text block to obtain a first label sequence, wherein the first label sequence includes each text block and the label information of the text block; each text block is converted into a text vector through the second network model, and the text vector is marked to obtain a second label sequence, wherein the second label sequence includes each text vector and the label information of the text vector; the first label sequence and the second label sequence are optimized to obtain a target label sequence; the finite state machine of the communication protocol is determined based on the target label sequence to complete the reverse of the communication protocol. Since each text block included in the target document can be input into the first network model to mark the text block to obtain the first label sequence, the text block can also be converted through the second network model to obtain a text vector, and the text vector is marked to obtain a second label sequence, the first label sequence and the second label sequence are optimized to obtain a target label sequence, and the finite state machine of the communication protocol is determined according to the target label sequence. The first label sequence is obtained through the first network model, and the second label sequence is obtained through the second network model, which improves the speed of determining the label sequence, optimizes the first label sequence and the second label sequence, determines the finite state machine according to the optimized target label training, and determines the finite state machine using the optimized label sequence, which is conducive to improving the accuracy of determining the finite state machine. Therefore, the problem of low efficiency of communication protocol reverse engineering existing in the related art can be solved, and the effect of improving the efficiency and accuracy of communication protocol reverse engineering can be achieved.

[0033] Optionally, the execution subject of the above steps can be a background processor, or other devices with similar processing capabilities, or a machine that at least integrates a data processing device, wherein the data processing device may include terminals such as computers and mobile phones, but is not limited to this.

[0034] In an exemplary embodiment, each of the text blocks is converted into a text vector by a second network model, and the text vector is marked, and obtaining a second label sequence includes: converting the text block into the text vector by a bidirectional encoder representation converter model included in the second network model; marking the text vector by a gated cyclic unit and a conditional random field layer included in the second network model to obtain the label information of the text vector; determining the sequence consisting of the text vector and the label information of the text vector as the second label sequence. In this embodiment, the second network model may include a bidirectional encoder representation converter (BidirectionalEncoder Representations from Transformers, BERT) model, a gated cyclic unit (GRU) and a conditional random field layer (CRF layer). The second network model combines BERT, a gated cyclic unit (GRU) and a CRF layer to better capture the contextual information of the text sequence and mark it using the dependencies between sequences. The text unit is converted into a representation, i.e., a text vector, using a BERT encoder, and then marked by a GRU layer and a CRF layer. By jointly training the parameters of the BERT encoder, GRU layer, and CRF layer, the model can be effectively optimized to achieve accurate labeling and information extraction of digital power grid protocol documents.

[0035] In the above embodiment, the second network model can receive a text block as input and output a sequence of tags corresponding to the grammar. In order to label the text, the BIO (beginning, internal, external) tag can be used. The text block corresponds to the paragraph in the RFC document. The paragraph is first divided into smaller units (such as single words, chunks, or phrases). Each unit is then mapped to a specific label. The convolutional neural network conditional random field processes a set of extracted features on each chunk. The conditional random field models the prediction as a probabilistic graphical model; the chained conditional random field specifically considers the sequential dependencies in the prediction. Let y be the label sequence and x be the input sequence of text units. It is hoped to maximize the conditional probability:

[0036] where f is the eigenvector x t The scoring function is learned with parameter vector θ. p(y,x) represents the joint probability of a given input sequence x and label sequence y, that is, the probability that the input sequence and label sequence appear at the same time; p(x,y) represents the joint probability of a given label sequence y and input sequence x, that is, the probability that the label sequence and input sequence appear at the same time. y' represents all possible label sequences, which is used to sum all possible label sequences; ∑ y' p(y', x) represents the sum of the joint probabilities of all possible label sequences given the input sequence x; y trepresents the tth label in the label sequence, and y t-1 represents the t-1th label in the label sequence. They are used to represent the label of the current position and the label of the previous position respectively. In order to understand θ, the negative log-likelihood log P(y, x) is minimized. The partition function is calculated by using the forward-backward algorithm: Z(x) = ∑ y′ p(y′,x). Z(x) represents the sum of the joint probabilities of all possible label sequences given an input sequence x, where y′ represents all possible label sequences. In a conditional random field (CRF), Z(x) is used to normalize the conditional probability p(y|x) to ensure that the sum of the probabilities of all possible label sequences is 1.

[0037] The neural CRF model (NEURALCRF) based on BERT embedding combines the gated recurrent unit (GRU) and CRF layers to better capture the contextual information of the text sequence and use the dependencies between sequences for labeling. The text units are converted into representations using the BERT encoder and then labeled through the GRU layer and the CRF layer. By jointly training the parameters of the BERT encoder, GRU layer, and CRF layer, the model can be effectively optimized to achieve accurate labeling and information extraction of digital power grid protocol documents.

[0038] We can add a CRF layer on these representations by replacing the conditional probability function f: Among them, x t Indicates the input of the text unit, h t is the representation of the text unit computed by the model, and P is the learned parameter matrix representing the transition between labels. As in the case of RNNCRF, the negative log-likelihood log P(y, x) is minimized to jointly learn the parameters of the BERT encoder, GRU layers, and the transfer matrix P.

[0039] In an exemplary embodiment, before converting the text block into the text vector by the bidirectional encoder representation converter model included in the second network model, the method further includes: obtaining a target grammar for describing the structure of the document of the communication protocol; obtaining a training document including the communication protocol; performing unsupervised learning on the initial bidirectional encoder representation converter model based on the target grammar and the training document to obtain the bidirectional encoder representation converter model. In this embodiment, a general target grammar can be defined in the digital power grid protocol document to describe the structure and characteristics of the network protocol, covering relevant concepts such as states, events, and variables. Among them, the target grammar may include a state definition tag: used to annotate the names of states, events, and variables related to each protocol. This tag identifies and defines various states in the text, including network connection states, protocol execution states, etc. Through this tag, the various states described in the document can be clearly identified, and a basis for marking and identification is provided for subsequent steps. Event definition tag: used to reference states and events in the text, link to marked states and events, and clarify the terms used in the text. This tag associates the events involved in the text with defined states and variables to help readers understand the various operations and processes described in the document. State machine labels: A set of two labels used to represent the logic of the state machine. These include transition and variable labels. Word embedding models can be pre-trained: Technical language embedding models are built using unsupervised learning algorithms. These labels are used to describe the two parts of the state machine, including transitions between states and the use of variables. Transition labels: Used to describe the transition relationship between states in a state machine. In the digital grid protocol document, transition labels can identify the transition from one state to another, describing the change of state during the execution of the protocol. Through transition labels, the transition conditions and triggering events between states can be clearly defined. Variable labels: Used to identify and describe the variables or parameters involved in the digital grid protocol. These variables may include various parameters used in the protocol, configuration options, status information, etc. Through variable labels, the various variables involved in the protocol can be clearly defined and their use and management during the execution of the protocol can be guided.

[0040] In the above embodiment, the BERT model can be unsupervisedly learned based on the training documents and the target grammar. By utilizing the collected digital power grid document information, a technical language embedding model, such as the BERT model, is constructed. The goal of this model is to convert the proprietary technical language in the field of digital power grids into distributed word representations. The advantage of doing this is that the semantic information between words can be captured, thereby better understanding the text content. Through the BERT model, words in the field of digital power grids are represented as distributed vectors, capturing the semantic relationships and meanings between them, and converting proprietary terms in the field of digital power grids into vector representations that can be understood by computers, providing a basis for subsequent model training and semantic analysis.

[0041] In an exemplary embodiment, optimizing the first label sequence and the second label sequence to obtain a target label sequence includes: determining the state text included in the first label sequence and the second label sequence, wherein the state text includes state information and verbs or directional prepositions; determining the event text included in the first label sequence and the second label sequence, wherein the event text includes action verbs; determining the first text included in the first label sequence and the second label sequence, wherein the first text is not span-marked and the first text includes state text or event text; determining the second text included in the first label sequence and the second label sequence, wherein the second text is not span-marked; optimizing the state text, the event text, the first text, and the second text to obtain the target label sequence. In this embodiment, a set of rules can be used to correct some simple situations that the model fails to recognize. These rules are used to apply to the classification output to correct related situations by flipping labels. These rules include: transition span marking: looking for text units that mention states. If the unit mentions a state and contains transition verbs or directional prepositions, mark the unit as a transition span. Action span marking: looking for text units that mention events. If the unit mentions an event and contains an action verb (e.g. send, receive), mark the unit as an action span. Unmarked span marking: Mark any remaining unmarked spans that mention a state or event as a trigger. Variable marking: Mark any remaining unmarked spans as variable names.

[0042] In the above embodiment, the state text may be a mentioned state, that is, the state of the protocol, such as LISTEN is the listening state, and RESPONSE is the responding state. The unmarked span may be understood as a span label that is not recognized by the second network model. The target name may depend on the specific protocol, for example, a sliding window variable in the TCP protocol, a congestion control variable, etc., and all the remaining unmarked content may be identified as a variable.

[0043] In an exemplary embodiment, the first label sequence and the second label sequence are optimized to obtain a target label sequence, including: marking the state text as a transition span; marking the event text as an action span; marking the first text as an unmarked span; marking the second text as a target name; and determining a sequence including the state text and the transition span, the event text and the action span, the first text and the unmarked span, the second text and the target name as the target label sequence. In this embodiment, transition span marking: First, find a text unit that mentions a state. If the unit mentions a state and contains a transition verb or a directional preposition, the unit is marked as a transition span. This helps to identify the part of the text that describes the state transition. For example, when the text describes "from CLOSED to OPEN state", "from CLOSED to OPEN" is identified as a transition span to indicate the state transition from the CLOSED state to the OPEN state. Action span marking: Secondly, find a text unit that mentions an event. If the unit describes an event and contains an action verb, the unit is marked as an action span. This helps identify the part of the text that describes the event action. For example, when the text describes "sending a packet", mark "sending a packet" as an action span to indicate the event that performs the send action. Unmarked span marking: Mark any remaining unmarked spans and mention the state or event as the trigger. Variable marking: Mark any remaining unmarked spans with the variable name, i.e. the target name.

[0044] The introduction of these tags enables the model to more accurately identify state transitions and event actions in the text, thereby improving the overall tagging quality and parsing effect. By using these rules, some situations that the model fails to accurately identify can be corrected, further improving the performance and effect of the inverse method.

[0045] In an exemplary embodiment, determining the finite state machine of the communication protocol based on the target tag sequence includes: determining a state set, an input-output event set, and a transition relationship set included in the target tag sequence; and determining the state set, the input-output event set, and the transition relationship set as the finite state machine. In this embodiment, the components of the finite state machine can be extracted from the output of the sequence model after post-processing optimization. It can include a state set, an input-output set, an initial state, and a state transition set. By analyzing the tag sequence output by the model, the structure of the finite state machine is identified, providing important support for the subsequent development and application of digital power grid systems. These transition relationships describe the transitions between states and the events associated therewith. A state set can be extracted from the output, some of which may be marked as initial states. All states can be extracted from the output of the model, and some of them can be identified as initial states. These states represent the states of the system at different stages, and the initial state identifies the initial state of the system. According to the information in the model output, the input and output event sets are extracted. The output of the model is analyzed to identify the input and output event sets. The input event represents the external input received by the system, and the output event represents the external output generated by the system. By extracting these event sets, we can understand the behavior and function of the system. Analyze the transition information in the model output to extract the set of transition relations between states. These transition relations describe the system's transition process between different states and the events associated with them. By parsing this information, we can construct a transition diagram of the finite state machine to fully describe the system's behavior and state transitions.

[0046] The protocol state machine, that is, the finite state machine can be expressed as P = <S,I,O,s 0 ,T>. The finite state set S contains a finite number of states, which is used to represent the various states that the system may be in. The finite input set I contains a finite number of inputs, which represents the external inputs received by the system. The finite output set O contains a finite number of outputs, which represents the external outputs generated by the system. And the output set does not overlap with the input set I. The initial state s 0 ∈S, represents the initial state of the system. A finite set of transitions It contains a finite number of transition relations between system states. Each transition relation consists of three parts: the starting state, the transition label (which can be empty, timeout, or input / output), and the target state.

[0047] In an exemplary embodiment, determining a target document including a communication protocol includes: obtaining a first initial document including the communication protocol; performing data cleaning and format standardization operations on the first initial document to obtain a second initial document; and preprocessing the second initial document using natural language processing technology to obtain the target document. In this embodiment, firstly, a wide range of public documents in the field of digital power grids are collected, including standards, protocols, research papers, etc. These documents may involve specifications and descriptions of various digital power grid communication protocols. Then, the collected documents are preprocessed, including removing format errors, normalizing text formats, segmenting text paragraphs, etc., to facilitate processing and analysis in subsequent steps.

[0048] In the above embodiment, digital grid-related documents can be collected in real time from various public sources in the field of digital grid, such as standard organization websites, academic paper databases, technical forums, etc., to obtain the first initial document and store it as structured data. Ensure that the latest and comprehensive digital grid document resources can be obtained, providing sufficient material basis for the analysis and processing of subsequent steps. The collected documents, i.e., the first initial document, are subjected to text cleaning and format standardization, including removing HTML tags, converting to a unified coding format, removing non-text content, etc. Eliminate noise and interference in the document to obtain a second initial document, ensuring that subsequent processing can be based on clean and standardized text data. Preprocess the standardized text data, i.e., the second initial document, including natural language processing technologies such as word segmentation, stem extraction, and part-of-speech tagging. Convert the original text data into a form that can be processed by a computer, extract the key information and features therein, and prepare for model building and training in subsequent steps.

[0049] The following is an explanation of the communication protocol reverse engineering method based on the public documents of the digital power grid in combination with a specific implementation method. Based on the communication protocol reverse engineering technology based on the public documents of the digital power grid, the DNP3 protocol is reverse engineered. DNP3 (Distributed Network Protocol) is a commonly used communication protocol for remote monitoring and control, and is widely used in power systems.

[0050] Figure 3 is a flow chart of a communication protocol reverse method based on a digital power grid public document according to a specific embodiment of the present invention, such as Figure 3 As shown, the process includes:

[0051] Step S302, receiving documents, text cleaning and format standardization.

[0052] When collecting DNP3 protocol-related documents, we covered its official standard documents, technical specifications, RFC documents, and related research papers. These documents contain key information such as the communication mechanism, data format, and message structure of the DNP3 protocol. Taking the official standard document as an example, it contains content such as data frame structure, function code, and control field. For example, a DNP3 data frame usually includes the following tags:

[0053] Control Field: Contains the control information of the DNP3 data frame, which is used to indicate the frame type, master-slave communication mode, etc.

[0054] Address Field: Used to indicate the source address and destination address of the data frame to determine the target device for communication.

[0055] Function Code: Indicates the operation type of the data frame, such as reading data, writing data, etc.

[0056] Data Field: Contains the actual data content, such as information collected by sensors, control commands, etc.

[0057] Step S304: text preprocessing: In the preprocessing stage, HTML tags and non-text content in the document are first removed, and then the encoding format of the document is unified to UTF-8 to ensure the accuracy and consistency of subsequent processing.

[0058] Step S306, pre-training word embedding model: using unsupervised learning algorithm to build a technical language embedding model, representing words in the field of digital power grid as distributed vectors, capturing the semantic relationship and meaning between them.

[0059] A technical language embedding model, namely the BERT model, was built based on the collected DNP3 protocol document information to convert the proprietary terms and concepts in the DNP3 protocol into distributed word representations. This embedding model can convert the terms in the protocol into vector representations, thereby capturing the semantic relationship and meaning between the terms. For example, terms such as "DNP3 data frame", "master station", and "slave station" are converted into continuous vector representations for computer understanding and processing.

[0060] Step S308, training a convolutional neural network conditional random field model (RNNCRF): using a convolutional neural network and a CRF layer to process a set of extracted features on each text unit, thereby learning a sequence-to-sequence mapping.

[0061] Step S310, training a BERT-embedded GRU conditional random field model (GRUCRF): The BERT encoder is enhanced with GRU and CRF layers to create chunk-level representations from word sequences.

[0062] When training the sequence-to-sequence model, the preprocessed DNP3 specification document was used and the key information was annotated. The model is designed to identify key fields and operation types in the data frame, such as the control field, function code, etc. The model can accurately mark these fields and operation types as corresponding events or states. For example, in data frame parsing, the model can accurately identify information such as communication mode and address type in the control field and mark them as corresponding events. At the same time, the model can also identify function codes, such as read data, write data and other operation types, and mark them as corresponding states.

[0063] Step S312, rule correction: adopt a set of rules to correct some simple cases that the model fails to recognize, including transition and variable marking.

[0064] After the model is output, it is post-processed and optimized to improve the accuracy and completeness of the extracted information. Detect and correct labeling errors caused by unclear or ambiguous document descriptions, remove noise in the model output, and optimize the structure of the output results. For example, when parsing a data frame, there may be errors or missing control domain labels, and the post-processing process can correct these errors. At the same time, there may be incomplete or erroneous parsing of some data frames. At this time, these erroneous data can be cleared to ensure the accuracy and completeness of the output results.

[0065] Step S314, finite state machine extraction: Extract state set: extract state set from output, some of which may be marked as initial state. Extract input and output event set: extract input and output event set according to the information in the model output. Parse transition relationship: parse the transition information in the model output, extract the transition relationship set of the finite state machine, and describe the transition between states and the events associated with them.

[0066] The finite state machine of the DNP3 protocol is extracted from the post-processed optimized sequence to the sequence model output. This includes state sets, input and output sets, initial states, and state transition sets. For example, in the data frame parsing process, the state machine can describe the state transition sequence in which the master sends a command to the slave and waits for a response. Among them, the state set includes the master state and the slave state, the input and output set includes the data frame and the response frame, and the state transition set describes the communication process between the master and the slave.

[0067] Step S316, digital grid dedicated protocol finite state machine result.

[0068] In the aforementioned embodiments, technologies such as the digital grid-specific XML tagging syntax, the BERT model, the convolutional neural network conditional random field (RNNCRF), and the BERT-embedded neural CRF model (GRUCRF) are utilized to improve the reverse engineering process of the digital grid communication protocol.

[0069] First, the digital grid-specific XML tagging syntax provides a basis for parsing and understanding digital grid protocol documents. This syntax defines the general structure and characteristics of the protocol, including relevant concepts such as states, events, and variables. By using a dedicated tagging syntax, key information in the protocol document can be extracted more accurately and efficiently, providing a reliable foundation for subsequent reverse engineering.

[0070] Secondly, the BERT model is a natural language processing model that can convert text into semantically rich vector representations. The BERT model is used to learn technical language embedding in the field of digital power grids, so as to better understand and analyze text information in protocol documents. By pre-training the BERT model, complex semantics and contextual information in the document can be captured, improving the accuracy and efficiency of reverse engineering.

[0071] Third, Convolutional Neural Network Conditional Random Field (RNNCRF) is a probabilistic graph model for sequence labeling tasks. (RNNCRF) is used to label text blocks as specific grammatical tags, as well as to identify states, events, and other important elements in the text. By using (RNNCRF), the association between text blocks can be established, and the information in the protocol document can be converted into actionable sequence tags, providing an important preprocessing step for subsequent finite state machine extraction.

[0072] In addition, the BERT-embedded neural CRF model (GRUCRF) combines the semantic representation ability of the BERT model and the sequence labeling ability of the CRF model. GRUCRF is used to label the sequence to identify and extract the finite state machine structure in the protocol document. By combining the semantic understanding of the BERT model and the sequence labeling ability of the CRF model, GRUCRF can more accurately infer the transition relationship between states and achieve accurate extraction and modeling of the protocol structure.

[0073] The present invention can achieve the following beneficial effects:

[0074] Improved accuracy: By converting public documents in the field of digital power grid into intermediate representations and applying a series of rules for post-processing and optimization, the accuracy of information extraction can be improved. The use of sequence-to-sequence models for training, combined with technologies such as semantic role taggers, effectively captures key information in documents, thereby reducing errors and inaccuracies.

[0075] Enhanced information comprehensiveness: The finite state machine extraction step can completely extract the state set, input and output event set, and the transition relationship set between states from the optimized intermediate representation. Such extraction ensures that the extracted finite state machine has complete information and can fully describe the state changes and event conversion process of the digital power grid system, providing sufficient reference for system development and maintenance.

[0076] Efficiency optimization: A series of optimization methods are adopted, including post-processing of intermediate representations, optimization training of models, and further processing of extraction results, to ensure the efficiency of the entire information extraction process. These optimization measures can speed up the information extraction process and reduce the time cost of system development and maintenance.

[0077] Improved flexibility: Due to the modular design concept, each step is independent of each other, so it has high flexibility. Users can selectively apply each step according to actual needs, or make customized adjustments according to specific field requirements, so as to better adapt to different application scenarios and requirements.

[0078] Enhanced versatility: The technical methods and processing procedures adopted by the present invention have strong versatility and can be applied not only to the field of digital power grids, but also to text information extraction tasks in other fields. By changing the input data of the model and adjusting relevant parameters, the present invention can be easily applied to text information extraction work in other fields, providing a universal solution for information analysis and application in related fields.

[0079] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0080] In this embodiment, a communication protocol reverse device based on the public document of the digital power grid is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0081] Figure 4 is a structural block diagram of a communication protocol reverse device based on a digital power grid public document according to an embodiment of the present invention, such as Figure 4 As shown, the device comprises:

[0082] A first determination module 402, configured to determine a target document including a communication protocol;

[0083] A first labeling module 404 is used to input each text block included in the target document into the first network model to label each text block to obtain a first label sequence, wherein the first label sequence includes each text block and label information of the text block;

[0084] A second labeling module 406 is used to convert each of the text blocks into a text vector through a second network model, and label the text vector to obtain a second label sequence, wherein the second label sequence includes each of the text vectors and label information of the text vector;

[0085] An optimization module 408, configured to optimize the first tag sequence and the second tag sequence to obtain a target tag sequence;

[0086] The second determination module 410 is used to determine the finite state machine of the communication protocol based on the target tag sequence, and has completed the reverse of the communication protocol.

[0087] In an exemplary embodiment, the second labeling module 406 can be implemented in the following manner: converting each of the text blocks into a text vector through a second network model, and labeling the text vector to obtain a second label sequence: converting the text block into the text vector through a bidirectional encoder representation converter model included in the second network model; labeling the text vector through a gated recurrent unit and a conditional random field layer included in the second network model to obtain label information of the text vector; and determining a sequence consisting of the text vector and the label information of the text vector as the second label sequence.

[0088] In an exemplary embodiment, the device can also be used to obtain a target grammar for the structure of a document describing the communication protocol before converting the text block into the text vector through the bidirectional encoder representation converter model included in the second network model; obtain a training document including the communication protocol; and perform unsupervised learning on an initial bidirectional encoder representation converter model based on the target grammar and the training document to obtain the bidirectional encoder representation converter model.

[0089] In an exemplary embodiment, the optimization module 408 can optimize the first label sequence and the second label sequence to obtain a target label sequence in the following manner: determine the status text included in the first label sequence and the second label sequence, wherein the status text includes status information and verbs or directional prepositions; determine the event text included in the first label sequence and the second label sequence, wherein the event text includes action verbs; determine the first text included in the first label sequence and the second label sequence, wherein the first text is not span-marked and the first text includes status text or event text; determine the second text included in the first label sequence and the second label sequence, wherein the second text is not span-marked; optimize the status text, the event text, the first text and the second text to obtain the target label sequence.

[0090] In an exemplary embodiment, the optimization module 408 can optimize the first label sequence and the second label sequence to obtain a target label sequence in the following manner: marking the state text as a transition span; marking the event text as an action span; marking the first text as an unmarked span; marking the second text as a target name; and determining a sequence including the state text and the transition span, the event text and the action span, the first text and the unmarked span, the second text and the target name as the target label sequence.

[0091] In an exemplary embodiment, the second determination module 410 can implement the finite state machine of determining the communication protocol based on the target tag sequence in the following manner: determining the state set, input-output event set, and transition relationship set included in the target tag sequence; and determining the state set, the input-output event set, and the transition relationship set as the finite state machine.

[0092] In an exemplary embodiment, the first determination module 402 can determine the target document including the communication protocol in the following manner: obtain a first initial document including the communication protocol; perform data cleaning and format standardization operations on the first initial document to obtain a second initial document; and use natural language processing technology to pre-process the second initial document to obtain the target document.

[0093] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0094] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0095] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0096] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0097] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0098] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of the method in each embodiment of the present application when the computer program is executed by a processor.

[0099] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0100] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A communication protocol reverse engineering method based on digital power grid public documents, characterized in that: include: determining target documents including communication protocols; Input each text block included in the target document into the first network model to label each text block to obtain a first label sequence, wherein the first label sequence includes each text block and label information of the text block; Convert each of the text blocks into a text vector through a second network model, and label the text vector to obtain a second label sequence, wherein the second label sequence includes each of the text vectors and label information of the text vector; Optimizing the first tag sequence and the second tag sequence to obtain a target tag sequence; A finite state machine of the communication protocol is determined based on the target tag sequence to complete the reversal of the communication protocol.

2. The method according to claim 1, characterized in that Each of the text blocks is converted into a text vector through a second network model, and the text vector is labeled to obtain a second label sequence including: Converting the text block into the text vector by a bidirectional encoder representation converter model included in the second network model; Labeling the text vector by using a gated recurrent unit and a conditional random field layer included in the second network model to obtain label information of the text vector; A sequence consisting of the text vector and the label information of the text vector is determined as the second label sequence.

3. The method according to claim 2, characterized in that Before converting the text block into the text vector by the bidirectional encoder representation converter model included in the second network model, the method further includes: Obtaining a target grammar for describing the structure of a document of the communication protocol; obtaining a training document including the communication protocol; An initial bidirectional encoder representation converter model is unsupervisedly learned based on the target grammar and the training document to obtain the bidirectional encoder representation converter model.

4. The method according to claim 1, characterized in that: Optimizing the first tag sequence and the second tag sequence to obtain a target tag sequence includes: Determining state text included in the first tag sequence and the second tag sequence, wherein the state text includes state information and a verb or a directional preposition; Determining event texts included in the first tag sequence and the second tag sequence, wherein the event texts include action verbs; Determine a first text included in the first tag sequence and the second tag sequence, wherein the first text is not span-marked and includes a state text or an event text; Determine a second text included in the first tag sequence and the second tag sequence, where the second text is not span-marked; The state text, the event text, the first text, and the second text are optimized to obtain the target label sequence.

5. The method according to claim 4, characterized in that Optimizing the first tag sequence and the second tag sequence to obtain a target tag sequence includes: marking the state text as a transition span; marking the event text as an action span; marking the first text as an unmarked span; marking the second text as a target name; A sequence including the state text and the transition span, the event text and the action span, the first text and the unlabeled span, the second text and the target name is determined as the target label sequence.

6. The method according to claim 1, characterized in that The finite state machine for determining the communication protocol based on the target tag sequence includes: Determine a state set, an input and output event set, and a transition relationship set included in the target tag sequence; The state set, the input-output event set, and the transition relationship set are determined as the finite state machine.

7. The method according to claim 1, characterized in that Target documents that include communication protocols include: Acquire a first initial document including the communication protocol; Performing data cleaning and format standardization operations on the first initial document to obtain a second initial document; The second initial document is preprocessed using natural language processing technology to obtain the target document.

8. A communication protocol reverse device based on digital power grid public documents, characterized in that: include: A first determination module, used to determine a target document including a communication protocol; A first labeling module, used for inputting each text block included in the target document into a first network model to label each text block to obtain a first label sequence, wherein the first label sequence includes each text block and label information of the text block; A second labeling module, used for converting each of the text blocks into a text vector through a second network model, and labeling the text vector to obtain a second label sequence, wherein the second label sequence includes each of the text vectors and label information of the text vector; an optimization module, configured to optimize the first tag sequence and the second tag sequence to obtain a target tag sequence; The second determination module is used to determine the finite state machine of the communication protocol based on the target tag sequence to complete the reversal of the communication protocol.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images