Message service rule extraction method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a message service rule extraction method and device, electronic equipment and a storage medium, and relates to the field of communication. Obtaining at least one service message of a to-be-extracted service rule; encoding the at least one service message to obtain an encoded message; and inputting the coded message into a trained service rule extraction model to obtain a target service rule corresponding to the at least one service message, the target service rule comprising a feature value used for identifying a fingerprint feature of the service message, the characteristic value is determined from the service message based on semantic analysis of the service message. By adopting the mode, the target service rule corresponding to the service message can be automatically extracted through the trained service rule extraction model, and the trained service rule extraction model can automatically adapt to the message of the new service, so that the service message can be conveniently analyzed based on the extracted service message rule, and the efficiency of network operation and maintenance is improved.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular to a method, device, electronic device and storage medium for extracting message service rules. Background Art

[0002] With the vigorous development of mobile Internet, the traffic of data services has shown explosive growth, and the types of data services have also continued to increase, posing great challenges to network security and network operation and maintenance. Perceiving and identifying the traffic of data services is the basis for improving network security and network maintenance efficiency. Currently, deep packet inspection is the main traffic identification technology.

[0003] Deep packet inspection relies on the business rules of business packets. Business rules are the fingerprint information of the data packet protocol in the data packet. Deep packet inspection can detect the data packets in the business flow according to the business rules, so as to determine the application information carried by the business flow. The detection of new packet protocols depends on the corresponding new packet rules. The traditional way to obtain and maintain these business rules is to build an automated guard system to extract business rules. The automated guard system in the relevant technology can extract and disassemble the rules into small components. For example, after pre-processing the business message, the message is grouped by clustering method, and then each group of messages is used to extract some rules preset by experts by multi-sequence alignment, common substrings, field domain inference, etc. Because the implementation of these systems depends on expert experience design, on the one hand, the detection effect for massive messages is poor, and the output rules can often only be used as a reference, and experts are still required to verify and screen them later. On the other hand, the system built based on expert experience lacks intelligent adaptive capabilities and cannot automatically adapt to the messages of new services. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method, device, electronic device and storage medium for extracting message business rules.

[0005] To solve the above technical problems, the embodiments of the present application are implemented through the following aspects.

[0006] According to a first aspect of an embodiment of the present disclosure, a method for extracting message service rules is provided, the method comprising:

[0007] Obtain at least one business message of the business rule to be extracted;

[0008] Encoding the at least one service message to obtain an encoded message;

[0009] The encoded message is input into a trained business rule extraction model to obtain a target business rule corresponding to the at least one business message, wherein the target business rule includes a feature value for identifying a fingerprint feature of the business message, and the feature value is determined from the business message based on a semantic analysis of the business message.

[0010] According to a second aspect of an embodiment of the present disclosure, a device for extracting message service rules is provided, including:

[0011] An acquisition module, used to acquire at least one business message of the business rule to be extracted;

[0012] An encoding module, used to encode the at least one service message;

[0013] A determination module is used to input the encoded message into a trained business rule extraction model to obtain a target business rule corresponding to at least one business message, wherein the target business rule includes a feature value for identifying a fingerprint feature of the business message, and the feature value is determined from the business message based on a semantic analysis of the business message.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and computer executable instructions stored in the memory and executable on the processor, wherein the computer executable instructions, when executed by the processor, implement the following steps: obtaining at least one business message of a business rule to be extracted; encoding the at least one business message to obtain an encoded message; inputting the encoded message into a trained business rule extraction model to obtain a target business rule corresponding to the at least one business message, wherein the target business rule includes a feature value for identifying a fingerprint feature of the business message, wherein the feature value is determined from the business message based on a semantic analysis of the business message.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, implements the following steps: obtaining at least one business message for business rules to be extracted; encoding the at least one business message to obtain an encoded message; inputting the encoded message into a trained business rule extraction model to obtain a target business rule corresponding to the at least one business message, wherein the target business rule includes a feature value for identifying a fingerprint feature of the business message, and the feature value is determined from the business message based on a semantic analysis of the business message.

[0016] The technical solution provided by the embodiments of the present disclosure is as follows: obtaining at least one business message of the business rules to be extracted; encoding the at least one business message to obtain an encoded message; inputting the encoded message into a trained business rule extraction model to obtain a target business rule corresponding to the at least one business message, wherein the target business rule includes a feature value for identifying a fingerprint feature of the business message, and the feature value is determined from the business message based on a semantic analysis of the business message. By adopting the above method, the target business rule corresponding to the business message can be automatically extracted through the trained business rule extraction model, and the trained business rule extraction model can automatically adapt to the message of the new business, so as to facilitate the analysis of the business message based on the extracted business message rules and improve the efficiency of network operation and maintenance.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0018] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 A schematic diagram showing a flow chart of a method for extracting message service rules provided in an embodiment of the present application;

[0021] Figure 2 Another flow chart of the method for extracting message service rules provided in an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram of a method for encoding a service message provided in an embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram showing the structure of a coded message provided in an embodiment of the present application is shown;

[0024] Figure 5 A flow chart showing a method for training a message service rule model provided in an embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram showing the structure of an encoded second training message sample provided in an embodiment of the present application is shown;

[0026] Figure 7 Another flow chart of the method for training a message service rule model provided in an embodiment of the present application is shown;

[0027] Figure 8 A schematic diagram of a message rule verification system provided by an embodiment of the present application is shown;

[0028] Fig. 9 A schematic diagram showing a system architecture of reinforcement learning provided by an embodiment of the present application;

[0029] Fig.10 A schematic diagram of a process of training optimization of reinforcement learning provided by an embodiment of the present application is shown;

[0030] Fig.11 A block diagram of a device for extracting message service rules provided in an embodiment of the present application is shown;

[0031] Fig.12 A block diagram showing another device for extracting message service rules provided in an embodiment of the present application;

[0032] Fig.13 A schematic diagram of the hardware structure of an electronic device for executing the method for extracting message business rules provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0034] First, some necessary explanations are given on the professional terms used in this application.

[0035] Message service rules are one or more identification information used to characterize the fingerprint characteristics of service messages. The identification information may include a feature field and a feature value used to identify the fingerprint characteristics of the service message. The feature value may be a fragment of the message, a regular expression matching part or all of the message, a port, a length field, an address information, and other basic features or a combination of multiple basic features. Those skilled in the art can understand that for the same service message, the message service rules may include one or more identification information. For example, for HTTP service, "Host = XXXX.com" and "User-Agent = XXXXmobile" may be two identification information included in the message service rules of the service message. Taking "Host = XXXX.com" as an example, "Host = " is a feature field that characterizes the fingerprint characteristics of the service message, and "XXXX.com" is a feature value that characterizes the fingerprint characteristics of the service message. For different service types, the feature fields of the identification information contained in the message service rules may be the same or different. For example, for FTP service, the feature fields that are the same or different from those of HTTP service may be included. The number of identification information contained in the message service rules may be the same or different.

[0036] Figure 1 A flow chart showing a method for extracting message service rules provided in an embodiment of the present application is shown. Figure 1 The method in the embodiment can be executed by an electronic device provided with a device for extracting message service rules, and the electronic device can be a terminal device or a designated server. Figure 1 As shown, the method may include the following steps.

[0037] In step S101, at least one service message of a service rule to be extracted is obtained.

[0038] In some embodiments, the messages may be screened using preset screening rules to obtain at least one service message for which the service rule is to be extracted.

[0039] For example, the above-mentioned preset screening rules can be flow rules, including a first address, a first port, a second address, a second port and a protocol type. Taking HTTP service as an example, the first address can be a client IP address, the first port can be a client port, the second address can be a server IP address, the second port can be a server port, and the protocol type can be HTTP. When performing service screening, the HTTP request message and the corresponding HTTP response message can be used as service messages for the service rules to be extracted. The source address of the HTTP request message can be the client IP address, the source port can be the client port, the destination address can be the server IP address, the destination port can be the server port, and the protocol type can be HTTP. The source address of the HTTP response message can be the server IP address, the source port can be the server port, the destination address can be the client IP address, the destination port can be the client port, and the protocol type can be HTTP.

[0040] In some embodiments, multiple messages that meet the preset filtering rules can also be further screened, such as identifying the first preset number of messages that meet the preset filtering rules, or controlling the total number of intercepted bytes not to exceed a preset threshold (for example, 15,000 bytes). This allows performance requirements to be taken into account when extracting message business rules, thereby further improving the efficiency of extracting message business rules.

[0041] By adopting the above-mentioned manner, a plurality of messages satisfying the preset screening rule can be screened out, and the business messages having the same or similar business characteristics can be used as the business messages for which the business rule is to be extracted.

[0042] In step S102, at least one service message is encoded to obtain an encoded message.

[0043] In some embodiments, the number of values ​​allowed for coding units in the coding space of the coded message is greater than the number of values ​​allowed for each byte in the service message.

[0044] Business messages are transmitted in hexadecimal encoding, and the smallest transmission unit is a byte. The byte value space of business messages is "0x00-0xFF", that is, the number of byte values ​​allowed in the byte value space of business messages is 256. The above magnitude cannot be compared with the vocabulary of natural language, so complex semantics cannot be presented through the byte values ​​of business messages, and direct byte recognition of business messages has poor performance and consumes a lot of computing resources, which is difficult to meet the performance requirements of the business.

[0045] In order to solve the above technical problems, the business message can be encoded, and the number of coding unit values ​​allowed by the encoding space of the encoded message can be greater than the number of values ​​allowed for each byte in the business message through predetermined encoding rules, so that the semantics of the business message can be better obtained.

[0046] In step S103, the encoded message is input into the trained business rule extraction model to obtain a target business rule corresponding to at least one business message.

[0047] The target service rule includes a feature value for identifying a fingerprint feature of a service message, and the feature value is determined from the service message based on a semantic analysis of the service message.

[0048] In some possible implementations, the business rule extraction model can be any one of BERT (Bidirectional Encoder Representation from Transformers), GPT (Generative Pre-Trained Transformer), ALBERT (ALite BERT), or BART (Bidirectional and Auto-Regressive Transformers). Those skilled in the art can understand that variant models of Transformers including a combination of an encoder and a decoder or only a decoder can also be used for the above-mentioned business rule extraction model, and this application does not limit this.

[0049] For example, the above-mentioned business rule extraction model can recognize the semantics of the encoded message, and based on the semantics of the encoded message, determine the feature field that identifies the fingerprint feature of the business message, identify the feature value of the fingerprint feature that identifies the business message, and then obtain the business rules of the business message, thereby realizing the automatic extraction of the target business rules of the business message. Compared with the business rule extraction method based on expert experience, the business rule extraction model that recognizes the semantics of the encoded message can also automatically adapt to the messages of new businesses.

[0050] By adopting the above method, the target business rules corresponding to the business messages can be automatically extracted through the trained business rule extraction model, and the trained business rule extraction model can automatically adapt to the messages of new businesses, so as to analyze the business messages based on the extracted business message rules and improve the efficiency of network operation and maintenance.

[0051] Figure 2A flow chart of a method for extracting message service rules provided in an embodiment of the present application is shown as follows: Figure 2 As shown, step S102 may include the following steps:

[0052] In step S1021, for each service message, at least one byte unit in the service message is encoded to obtain an intermediate message corresponding to the service message.

[0053] The intermediate message includes at least one target encoding unit corresponding to each byte unit, and the byte unit includes multiple bytes.

[0054] In some embodiments, at least one byte unit in a service message may be encoded respectively through the following steps to obtain an intermediate message corresponding to the service message.

[0055] Step 20: When the byte unit includes a preset number of identical bytes or preset text symbols, obtain the target encoding unit corresponding to the preset number of identical bytes or preset text symbols from the preset encoding correspondence; or, when the byte unit does not include a preset number of identical bytes and preset text symbols, encode the byte unit according to the preset encoding rules to obtain the corresponding target encoding unit.

[0056] Figure 3 A schematic diagram of a method for encoding a service message provided in an embodiment of the present application is shown, Figure 3 As shown, the preset encoding rule may be a 2-Gram encoding method, the preset encoding correspondence may include a correspondence between a preset number of identical bytes and a target encoding unit, and a target encoding unit corresponding to a preset text symbol, and the specific encoding rule may include:

[0057] A1. Take two bytes as one byte unit, convert each byte unit into the corresponding decimal through 2-Gram encoding, and use the converted result as the target encoding unit corresponding to the byte unit, that is, (byte unit 0x0000~0xFFFF corresponds to target encoding unit 0-65535), for example Figure 3 In the example, the byte unit “0x1601” corresponds to the target coding unit 5633, and the byte unit “0x0103” corresponds to the target coding unit 259.

[0058] A2. For consecutive identical bytes, the corresponding target coding unit can be obtained from the correspondence between the preset number of identical bytes and the target coding unit. The correspondence between the preset number of identical bytes and the target coding unit can be, for example, "65535+number of consecutive identical bytes-(first consecutive number-1)+(n+1)*(second consecutive number-first consecutive number+1)", where the number of consecutive identical bytes can be greater than or equal to the first consecutive number (for example, 3) and less than or equal to the second consecutive number (for example, 10), and n is the value of the identical byte, which can range from 0 to 255. For example, in Figure 3 In the example, the byte unit is 6 consecutive 0x25 (37), which can be recorded as "6-0x25", and its corresponding target coding unit is 65535+6-2+38*8=65639. When the first consecutive number is 3 and the second consecutive number is 10, the encoding range of consecutive identical bytes, such as "3-0x00"-"10-0xFF", corresponds to a target coding unit range of 65536 to 67576.

[0059] A3. For commonly used characters appearing in protocol messages, the value of the target coding unit can be determined from the correspondence between the target coding units corresponding to the preset text symbols. The preset text symbols include but are not limited to "HTTP", "SMB", "GET", "POST", "USER", "PASS", "PWD", "CNAME", "User-Agent" and "Host", etc. Those skilled in the art can define the correspondence between the preset text symbols and the target coding units based on the common characters of the business messages. In some possible implementations, the range of the target coding unit corresponding to the preset text symbols can be, for example, 67577 to 68576. Those skilled in the art can understand that in order to increase the scalability of the correspondence between the preset text symbols and the target coding units, a certain number of reserved target coding unit values ​​can be reserved for possible extended text characters.

[0060] A4. In order to perform splicing processing on the coded messages respectively corresponding to a plurality of business messages, the preset coding correspondence may further include a correspondence between a preset functional coding unit and a target coding unit.

[0061] The preset function coding unit may include a start coding unit, an end coding unit and a separation coding unit.

[0062] For example: the start coding unit [START] (the value can be, for example, 68577) can identify the beginning of a business flow, the separation coding unit [SEP] (the value can be, for example, 68578) can identify the concatenation of two business messages, the end coding unit (the value can be, for example, 68579) can identify the end of a business flow, the preset function coding unit can also include a null coding unit [NOP] (the value can be, for example, 68580) for length padding, and the downstream task indication unit [PMT] (the value can be, for example, 68581) can be used to generate training message samples. Those skilled in the art can understand that, in order to expand the definition of the target coding unit corresponding to the function coding unit, a certain number of reserved target coding unit values ​​can be reserved for possible extended function characters.

[0063] The present application does not limit the specific encoding method. Those skilled in the art can adaptively adjust the encoding method under the inspiration of the encoding methods shown in A1 to A4.

[0064] Step 21: splice the obtained target coding units in sequence to obtain an intermediate message.

[0065] In some possible implementations, multiple target coding units in a service message are concatenated according to the byte order of the service message to obtain an intermediate message corresponding to the service message.

[0066] In step S1022, at least one intermediate message is connected through a preset functional encoding unit to obtain an encoded message.

[0067] Among them, the functional coding unit is a preset target coding unit that represents the beginning of a message, the end of a message or a message interval.

[0068] In some embodiments, at least one intermediate message may be connected through a preset functional coding unit in the following manner to obtain a coded message:

[0069] B1. When the number of service messages is one, connect the start coding unit and the end coding unit of the intermediate message corresponding to the service message to obtain a coded message.

[0070] B2. When there are multiple business messages, the intermediate messages corresponding to the multiple business messages are connected through the separation coding unit to obtain a connected intermediate message, and the starting coding unit, the connected intermediate message and the ending coding unit are connected to obtain a coded message.

[0071] Figure 4 A schematic diagram showing the structure of a coded message provided in an embodiment of the present application is shown as follows: Figure 4As shown, a plurality of intermediate messages are connected through separation coding units to form a connection intermediate message, and a start coding unit, a connection intermediate message and an end coding unit are connected to obtain a coded message.

[0072] By adopting the above technical solution, the byte units in the business message are encoded, so that the number of values ​​allowed by the encoding space of the encoded message is greater than the number of values ​​allowed for each byte in the business message (256). Compared with the number of values ​​allowed for each byte in the business message, it is closer to the number of values ​​allowed by the vocabulary space of natural language, so that the encoded message can better represent the semantics of the message and facilitate the processing of the business rule extraction model.

[0073] Since business messages contain a large number of protocols and their syntax is more complex than natural language, the training of business rule extraction models is more difficult. This part consumes more computing power than natural language model training. In some possible implementations, a distributed training framework can be used in the training process, for example, it can be completed in a GPU cluster.

[0074] Figure 5 A flow chart of a method for training a message service rule model provided in an embodiment of the present application is shown as follows: Figure 5 As shown, before step S101, it also includes:

[0075] In step S104, according to a preset message service rule, a first training message sample is obtained, and the first training message sample is encoded to obtain an encoded first training message sample.

[0076] The encoded first training message sample includes a plurality of training coding units.

[0077] In some possible implementations, based on existing message business rules, a first training message sample can be obtained from a preset business message database or from a business link, and the first training message sample can be encoded using steps S1021 to S1022 to obtain an encoded first training message sample.

[0078] In some embodiments, business messages may be encrypted messages. These encrypted messages have high information entropy and great randomness. When acquiring business messages, for the identified encrypted streams, only the unencrypted part may be selected. For example, for messages of the TLS (Transport Layer Security) protocol, only the handshake negotiation part of the messages may be selected as training message samples to improve the quality of the training message samples.

[0079] In step S105, the training coding units in the first training message sample are sequentially input into the basic large model to be trained, and the basic large model to be trained is trained by unsupervised training to obtain a pre-trained basic large model.

[0080] The basic large model to be trained can be any one of BERT, GPT, ALBERT or BART. Those skilled in the art will understand that a variant model of Transformer including a combination of an encoder and a decoder or only a decoder can also be used as the basic large model to be trained.

[0081] The goal of pre-training is to enable the basic large model to extract the business features of business messages and have the ability to understand the semantics of messages. In some possible implementation methods, the basic large model to be trained can be trained through the following steps:

[0082] Step 30: input the training coding units in the first training message sample into the basic large model to be trained in sequence, and obtain a preset number of prediction coding units output by the basic large model to be trained.

[0083] Step 31: when the first loss value between the prediction coding unit and the preset number of actual training coding units converges, the basic large model to be trained is determined as the basic large model after pre-training, wherein the actual training coding unit is the training coding unit to be input in the first training message sample.

[0084] The first loss value may be a negative log-likelihood loss, which is used to characterize the difference between the predicted coding unit and the actual training coding unit. When the first loss value converges, it indicates that the pre-trained basic large model has the ability to understand the semantics of the message.

[0085] For more technical details on unsupervised training of the basic large model, please refer to the description in the relevant technology, which will not be elaborated here.

[0086] In step S106, the encoded first training message sample and the corresponding message service rule are concatenated to obtain an encoded second training message sample.

[0087] Figure 6 A schematic diagram showing the structure of a second training message sample after encoding provided by an embodiment of the present application is shown as follows: Figure 6 As shown, the encoded first training message sample is spliced ​​with the corresponding message service rule, and then a downstream task indicator unit [PMT] can be added for easy identification to form an encoded second training message sample.

[0088] In step S107, the encoded second training message sample is input into the pre-trained basic large model, and the pre-trained basic large model is adjusted by supervised training to obtain a pre-trained business rule extraction model.

[0089] In some embodiments, the pre-trained basic large model can be adjusted through the following steps.

[0090] Step 40: Input the encoded second training message sample into the pre-trained basic large model to obtain the predicted message business rule corresponding to the encoded second training message sample.

[0091] Step 41: When the second loss values ​​of the predicted message business rules and the real message business rules converge, a pre-trained business rule extraction model is obtained.

[0092] The pre-trained basic large model is adjusted by calculating the loss values ​​of the predicted message business rules and the real message business rules (i.e., the message business rules added to the second training message sample), and optimizing the model parameters through the loss values, thereby obtaining the pre-trained business rule extraction model.

[0093] In some possible implementations, the second loss value may be determined by the following formula 1.

[0094] loss new =loss base +γ*loss extra (Formula 1)

[0095] Among them, loss new is the second loss value, loss base is the first loss value, loss extra is the additional loss value of the predicted message business rules and the real message business rules (i.e., the message business rules added to the second training message sample), loss extra =∑ (x,y) logP(y|x 1 ,…,x m ), x is the real message business rule, y is the predicted message business rule, and γ is the preset loss coefficient.

[0096] The second loss value takes into account the first loss value of the base large model, which can ensure faster convergence when adjusting the pre-trained base large model.

[0097] For specific technical details on the supervised training of the pre-trained basic large model, please refer to the technical description in the relevant technology, which will not be elaborated here.

[0098] In step S108, a trained business rule extraction model is obtained according to the pre-trained business rule extraction model.

[0099] In some embodiments, the pre-trained business rule extraction model may be used as a trained business rule extraction model.

[0100] By adopting the above method, the target business rules corresponding to the business messages can be automatically extracted through the trained business rule extraction model, and the trained business rule extraction model can automatically adapt to the messages of new businesses, so as to analyze the business messages based on the extracted business message rules and improve the efficiency of network operation and maintenance.

[0101] In another embodiment, reinforcement learning training is performed on the pre-trained business rule extraction model to obtain a trained business rule extraction model.

[0102] In the reinforcement learning system, the agent perceives the environment S t Then make the corresponding action i Get the corresponding reward or punishment reward i Finally, by optimizing the model with a specific strategy, the model can be evolved in the direction of obtaining higher rewards, thereby obtaining the model of the desired target. In the message business rule extraction system, the environment perceived by the agent is the input business message, the action performed is the output message business rule, and the reward obtained is the evaluation given by the message rule verification system.

[0103] Figure 7 Another flow chart of the training method of the message service rule model provided in the embodiment of the present application is shown as follows: Figure 7 As shown, step S108 may include the following steps:

[0104] In step S1081, a fully connected layer is added to the pre-trained business rule extraction model to obtain a Critic evaluation model.

[0105] In step S1082, the pre-trained business rule extraction model is used as the Actor business rule extraction model, the Actor business rule extraction model and the Critic evaluation model are optimized through the reinforcement learning algorithm of the Actor-Critic architecture, and the optimized Actor business rule extraction model is used as the trained business rule extraction model.

[0106] In some embodiments, PPO (Proximal Policy Optimization) can be used to optimize the Actor business rule extraction model and the Critic evaluation model, and the optimized Actor business rule extraction model is used as the trained business rule extraction model. Those skilled in the art can understand that other reinforcement learning algorithms of the Actor-Critic architecture can also be used to complete the reinforcement learning optimization of the business rule extraction model, and the present disclosure does not limit this.

[0107] In some possible implementations, the reinforcement learning optimization of the business rule extraction model can be completed through the following steps.

[0108] Step 50: According to a preset training service rule, obtain a third training message sample, encode the third training message sample, and obtain an encoded third training message sample.

[0109] The specific steps of obtaining the third training message sample and encoding the third training message sample may refer to the similar step S104, which will not be described in detail here.

[0110] Step 51: Input the encoded third training message sample into the Actor business rule extraction model to obtain the intermediate business rule, and input the third training message sample and the intermediate business rule into a preset message rule verification system to obtain the target reward value.

[0111] Among them, the target reward value represents the recall rate and precision rate of the intermediate business rule matching the third training message. In some possible implementation methods, the target reward value can be obtained by weighted averaging the recall rate and the precision rate using preset weights. This application does not limit the specific calculation method of the target reward value.

[0112] Figure 8 A schematic diagram of a message rule verification system provided by an embodiment of the present application is shown as follows: Figure 8 As shown, the third training message sample can be used as the message of the message business rule to be verified, the intermediate business rule can be used as the message business rule to be verified, and the existing business sampling message and the existing message business rule can be input into the message rule verification system together, so as to obtain the recall rate and precision rate of the message business rule to be verified.

[0113] In some embodiments, the recall rate is used to evaluate the ability of the service rule of the message to be verified to match the message of the current service. The recall rate of the service rule of the message to be verified can be determined by the following formula 2.

[0114]

[0115] Among them, recall is the recall rate, match is thecorrect Match the correct message entry to the business rule of the message to be verified, total num The number of all messages corresponding to the current service.

[0116] The accuracy rate is used to evaluate the ability of the business rule of the message to be verified to accurately identify the business message and avoid misidentifying messages of other services. The accuracy rate of the business rule of the message to be verified can be determined by the following formula 3.

[0117]

[0118] Among them, acc is the accuracy, match is correct Match the correct message entry for the message business rule to be verified. total The number of all business messages that match the business rules of the message to be verified.

[0119] The message rule verification system can adopt the implementation scheme in the relevant technology. Compared with the field of natural language processing, the model training process relies heavily on manual feedback optimization. The reinforcement learning stage of this application uses the results of the message rule verification system in the relevant technology for the reinforcement learning training of the model, which can further improve the efficiency of reinforcement learning training.

[0120] Step 52: Concatenate the encoded third training message sample and the intermediate business rule to obtain an encoded fourth training message sample.

[0121] For the specific splicing method, please refer to the similar step S106 and the attached Figure 6 , I will not elaborate on it here.

[0122] Step 53: Input the fourth training message sample into the Critic evaluation model to obtain a target evaluation value.

[0123] The Critic evaluation model is obtained by adding a fully connected layer to the pre-trained business rule extraction model. The quality of the Actor business rule extraction model can be evaluated by comparing the differences between the message business rules output by the current business rule extraction model and the intermediate business rules.

[0124] Step 54: Obtain a target deviation value according to the target reward value and the target evaluation value.

[0125] In some possible implementations, the difference between the target reward value and the target evaluation value can be used as the target deviation value, the ratio of the target reward value to the target evaluation value can be used as the target deviation value, or the absolute value of the difference between the target reward value and the target evaluation value can be used as the target deviation value. This application does not limit the specific method for obtaining the target deviation value.

[0126] Fig. 9 A schematic diagram showing a system architecture of reinforcement learning provided by an embodiment of the present application is shown as follows: Fig. 9 As shown, the message rule verification system evaluates the intermediate business rules output by the Actor business rule extraction model to obtain the target reward value, and the encoded third training message sample is spliced ​​with the intermediate business rules and input into the Critic evaluation model to obtain the target evaluation value, and then the target deviation value is obtained according to the target reward value and the target evaluation value.

[0127] Step 55: Obtain a third loss value of the Actor business rule extraction model according to the rule deviation value and the target deviation value of the intermediate business rule, obtain a fourth loss value of the Critic evaluation model according to the evaluation deviation value and the target reward value of the target evaluation value, optimize the Actor business rule extraction model according to the third loss value, and optimize the Critic evaluation model according to the fourth loss value.

[0128] The rule deviation value represents the deviation between the intermediate business rules and the historical business rules, and the evaluation deviation value represents the deviation between the target evaluation value and the historical evaluation value.

[0129] Fig.10 A schematic diagram of a process of training optimization of reinforcement learning provided by an embodiment of the present application is shown, such as Fig.10As shown, after the Actor business rule extraction model outputs the intermediate business rule according to the input third training message sample, it is compared with the historical business rule to obtain the rule deviation value, wherein the initial value of the historical business rule can be set to a preset default value or the first intermediate business rule. In some possible implementations, the rule deviation value can be determined by the cosine similarity of the semantic vectors corresponding to the historical business rule and the intermediate business rule, and then the third loss value can be obtained according to the target deviation value and the rule deviation value, which is used to iteratively optimize the Actor business rule extraction model. In some possible implementations, the rule deviation value and the target deviation value can be weighted averaged using a preset weight value to obtain the third loss value, so that the Actor business rule extraction model can be gradient optimized using the third loss value. After splicing the third training message sample and the intermediate business rule output by the Actor business rule extraction model, the target evaluation value can be obtained by inputting the Critic evaluation model, and the corresponding evaluation deviation value can be obtained by the target evaluation value and the historical evaluation value, wherein the initial value of the historical evaluation value can be a preset default value or the first target evaluation value. In some possible implementations, the evaluation deviation value may be the difference between the target evaluation value and the historical evaluation value, or the ratio between the target evaluation value and the historical evaluation value. This application does not limit the calculation method of the evaluation deviation value. In the proximal policy optimization algorithm PPO, the evaluation deviation value may also be obtained by clipping the deviation between the target evaluation value and the historical evaluation value through a preset upper threshold and a lower threshold, thereby limiting the upper and lower limits of the evaluation deviation value and preventing excessive adjustment during the training optimization process. After obtaining the evaluation deviation value, the target reward value and the evaluation deviation value may be weighted averaged according to the preset weights to obtain a fourth loss value, and the Critic evaluation model may be gradient optimized through the fourth loss value.

[0130] After completing a round of optimization, the intermediate business rules and target evaluation values ​​in this optimization can be used as historical business rules and historical evaluation values, respectively, for the next iterative optimization.

[0131] Step 56: When the third loss value converges, the obtained Actor business rule extraction model is used as the trained business rule extraction model.

[0132] By adopting the above technical solution, the business rule extraction model is trained and optimized through reinforcement learning, which realizes the continuous optimization of the message business model and further improves the adaptability of the message extraction model. The target reward value obtained by the message rule verification system is more deterministic than the manual feedback method in the natural language field, and the efficiency of obtaining the target reward value is also higher, which further improves the effect and training efficiency of the reinforcement learning training of the message business model.

[0133] Fig.11 A block diagram of a device for extracting message service rules provided in an embodiment of the present application is shown as follows: Fig.11 As shown, the message service rule extraction device 400 includes:

[0134] An acquisition module 410, configured to acquire at least one business message of a business rule to be extracted;

[0135] The encoding module 420 is used to encode at least one service message;

[0136] Determination module 430 is used to input the encoded message into the trained business rule extraction model to obtain a target business rule corresponding to at least one business message, wherein the target business rule includes a feature value for identifying a fingerprint feature of the business message, and the feature value is determined from the business message based on semantic analysis of the business message.

[0137] Optionally, the encoding module 420 is further configured to:

[0138] For each service message, encode at least one byte unit in the service message to obtain an intermediate message corresponding to the service message, wherein the intermediate message includes a target encoding unit corresponding to at least one byte unit, and the byte unit includes multiple bytes;

[0139] At least one intermediate message is connected through a preset functional coding unit to obtain a coded message, and the functional coding unit is a preset target coding unit that represents a message start, a message end or a message interval.

[0140] Optionally, the encoding module 420 is further configured to:

[0141] In the case where the byte unit includes a preset number of identical bytes or preset text symbols, obtaining target encoding units corresponding to the preset number of identical bytes or preset text symbols from a preset encoding correspondence; or,

[0142] When the byte unit does not include a preset number of identical bytes and a preset text symbol, the byte unit is encoded according to a preset encoding rule to obtain a corresponding target encoding unit;

[0143] The obtained target coding units are spliced ​​in sequence to obtain an intermediate message.

[0144] Optionally, the encoding module 420 is further configured to:

[0145] When the number of service messages is one, the start coding unit, the middle message corresponding to the service message, and the end coding unit are connected to obtain a coded message; or,

[0146] When there are multiple business messages, the intermediate messages corresponding to the multiple business messages are connected through the separation coding unit to obtain a connected intermediate message, and the starting coding unit, the connected intermediate message and the ending coding unit are connected to obtain a coded message.

[0147] Fig.12 A block diagram of another device for extracting message service rules provided in an embodiment of the present application is shown. Fig.12 As shown, the message service rule extraction device 400 further includes a training module 440, which is used to:

[0148] According to a preset message service rule, a first training message sample is obtained, and the first training message sample is encoded to obtain an encoded first training message sample, wherein the encoded first training message sample includes a plurality of training encoding units;

[0149] The training coding units in the first training message sample are sequentially input into the basic large model to be trained, and the basic large model to be trained is trained by unsupervised training to obtain a pre-trained basic large model;

[0150] splicing the encoded first training message sample and the corresponding message business rule to obtain an encoded second training message sample;

[0151] Input the encoded second training message sample into the pre-trained basic large model, adjust the pre-trained basic large model through supervised training, and obtain the pre-trained business rule extraction model;

[0152] Obtain a trained business rule extraction model based on the pre-trained business rule extraction model.

[0153] Optionally, the training module 440 is further configured to:

[0154] Use the pre-trained business rule extraction model as the trained business rule extraction model; or,

[0155] The pre-trained business rule extraction model is subjected to reinforcement learning training to obtain a trained business rule extraction model.

[0156] Optionally, the training module 440 is further configured to:

[0157] Based on the pre-trained business rule extraction model, a fully connected layer is added to obtain the Critic evaluation model;

[0158] The pre-trained business rule extraction model is used as the Actor business rule extraction model, the Actor business rule extraction model and the Critic evaluation model are optimized through the reinforcement learning algorithm of the Actor-Critic architecture, and the optimized Actor business rule extraction model is used as the trained business rule extraction model.

[0159] Optionally, the training module 440 is further configured to:

[0160] According to a preset training service rule, a third training message sample is obtained, and the third training message sample is encoded to obtain an encoded third training message sample;

[0161] Input the encoded third training message sample into the Actor business rule extraction model to obtain the intermediate business rule, input the third training message sample and the intermediate business rule into a preset message rule verification system to obtain a target reward value, where the target reward value represents the recall rate and precision rate of the intermediate business rule matching the third training message;

[0162] splicing the encoded third training message sample and the intermediate business rule to obtain an encoded fourth training message sample;

[0163] Input the fourth training message sample into the Critic evaluation model to obtain a target evaluation value;

[0164] Obtain a target deviation value according to the target reward value and the target evaluation value;

[0165] Obtain a third loss value of the Actor business rule extraction model according to the rule deviation value and the target deviation value of the intermediate business rule, obtain a fourth loss value of the Critic evaluation model according to the evaluation deviation value and the target reward value of the target evaluation value, optimize the Actor business rule extraction model according to the third loss value, and optimize the Critic evaluation model according to the fourth loss value, the rule deviation value represents the deviation value between the intermediate business rule and the historical business rule, and the evaluation deviation value represents the deviation between the target evaluation value and the historical evaluation value;

[0166] When the third loss value converges, the obtained Actor business rule extraction model is used as the trained business rule extraction model.

[0167] The device 400 provided in the embodiment of the present application can execute each method in the foregoing method embodiment and realize the functions and beneficial effects of each method in the foregoing method embodiment, which will not be repeated here.

[0168] Fig.13A schematic diagram of the hardware structure of an electronic device that implements the embodiment of the present application is shown. Referring to the figure, at the hardware level, the electronic device includes a processor, and optionally, an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage, etc. Of course, the electronic device may also include hardware required for other services.

[0169] The processor, the network interface and the memory may be interconnected through an internal bus, which may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0170] The memory stores the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0171] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a device for locating the target user at the logical level. The processor executes the program stored in the memory and specifically performs: Figure 1-2 , Figure 5 and Figure 7 The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0172] The above application Figure 1-2 , Figure 5 and Figure 7The method disclosed in the illustrated embodiment may be implemented in a processor or by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor or by instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0173] The electronic device can also execute the methods described in the foregoing method embodiments, and realize the functions and beneficial effects of the methods described in the foregoing method embodiments, which will not be repeated here.

[0174] Of course, in addition to software implementation methods, the electronic device of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0175] The embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable medium stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes Figure 1-2 , Figure 5 and Figure 7 The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0176] The computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0177] Furthermore, an embodiment of the present application also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, and when the program instructions are executed by a computer, the following process is implemented: Figure 1-2 , Figure 5 and Figure 7 The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0178] In short, the above is only a preferred embodiment of the present application and does not limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0179] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0180] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0181] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0182] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A method for extracting message service rules, characterized in that: The method comprises: Obtain at least one business message of the business rule to be extracted; Encoding the at least one service message to obtain an encoded message; The encoded message is input into a trained business rule extraction model to obtain a target business rule corresponding to the at least one business message, wherein the target business rule includes a feature value for identifying a fingerprint feature of the business message, and the feature value is determined from the business message based on a semantic analysis of the business message.

2. The method according to claim 1, characterized in that The number of values ​​allowed for the coding unit in the coding space of the coded message is greater than the number of values ​​allowed for each byte in the service message.

3. The method according to claim 1, characterized in that The encoding of the at least one service message to obtain an encoded message includes: For each of the service messages, respectively encode at least one byte unit in the service message to obtain an intermediate message corresponding to the service message, wherein the intermediate message includes at least one target encoding unit corresponding to each of the byte units, and the byte unit includes multiple bytes; The encoded message is obtained by connecting at least one of the intermediate messages through a preset functional encoding unit, and the functional encoding unit is a preset target encoding unit that represents the beginning of a message, the end of a message, or the interval between messages.

4. The method according to claim 3, characterized in that Encoding at least one byte unit in the service message to obtain an intermediate message corresponding to the service message includes: In the case where the byte unit includes a preset number of identical bytes or preset text symbols, obtaining target encoding units corresponding to the preset number of identical bytes or the preset text symbols from a preset encoding correspondence; or, When the byte unit does not include a preset number of identical bytes and a preset text symbol, the byte unit is encoded using a preset encoding rule to obtain a corresponding target encoding unit; The obtained target coding units are spliced ​​in sequence to obtain the intermediate message.

5. The method according to claim 3, characterized in that: The preset functional coding unit includes a start coding unit, an end coding unit and a separation coding unit, and the at least one intermediate message is connected by the preset functional coding unit to obtain the coded message, including: When the number of the service message is one, the start coding unit, the intermediate message corresponding to the service message, and the end coding unit are connected to obtain the coded message; or, In the case where there are multiple business messages, the intermediate messages corresponding to the multiple business messages are connected through the separation coding unit to obtain a connected intermediate message, and the starting coding unit, the connected intermediate message and the ending coding unit are connected to obtain the encoded message.

6. The method according to any one of claims 1 to 5, characterized in that Before obtaining at least one business message of the business rule to be extracted, the method further includes: According to a preset message service rule, a first training message sample is obtained, and the first training message sample is encoded to obtain an encoded first training message sample, wherein the encoded first training message sample includes a plurality of training encoding units; Inputting the training coding units in the first training message sample into the basic large model to be trained in sequence, training the basic large model to be trained by unsupervised training to obtain a pre-trained basic large model; splicing the encoded first training message sample and the corresponding message service rule to obtain an encoded second training message sample; Inputting the encoded second training message sample into the pre-trained basic large model, adjusting the pre-trained basic large model through supervised training to obtain a pre-trained business rule extraction model; The trained business rule extraction model is obtained according to the pre-trained business rule extraction model.

7. The method according to claim 6, characterized in that The step of obtaining the trained business rule extraction model according to the pre-trained business rule extraction model includes: The pre-trained business rule extraction model is used as the trained business rule extraction model; or, Reinforcement learning training is performed on the pre-trained business rule extraction model to obtain the trained business rule extraction model.

8. The method according to claim 7, characterized in that The performing reinforcement learning training on the pre-trained business rule extraction model to obtain the trained business rule extraction model includes: Adding a fully connected layer to the pre-trained business rule extraction model to obtain a Critic evaluation model; The pre-trained business rule extraction model is used as the Actor business rule extraction model, the Actor business rule extraction model and the Critic evaluation model are optimized through the reinforcement learning algorithm of the Actor-Critic architecture, and the optimized Actor business rule extraction model is used as the trained business rule extraction model.

9. The method according to claim 8, characterized in that The pre-trained business rule extraction model is used as the Actor business rule extraction model, the Actor business rule extraction model and the Critic evaluation model are optimized by the reinforcement learning algorithm of the Actor-Critic architecture, and the optimized Actor business rule extraction model is used as the trained business rule extraction model, including: According to a preset training service rule, a third training message sample is obtained, and the third training message sample is encoded to obtain an encoded third training message sample; Input the encoded third training message sample into the Actor business rule extraction model to obtain an intermediate business rule, input the third training message sample and the intermediate business rule into a preset message rule verification system to obtain a target reward value, wherein the target reward value represents the recall rate and precision rate of the intermediate business rule matching the third training message; splicing the encoded third training message sample and the intermediate business rule to obtain an encoded fourth training message sample; Inputting the fourth training message sample into the Critic evaluation model to obtain a target evaluation value; Obtaining a target deviation value according to the target reward value and the target evaluation value; Obtaining a third loss value of the Actor business rule extraction model according to the rule deviation value of the intermediate business rule and the target deviation value, obtaining a fourth loss value of the Critic evaluation model according to the evaluation deviation value of the target evaluation value and the target reward value, optimizing the Actor business rule extraction model according to the third loss value, and optimizing the Critic evaluation model according to the fourth loss value, the rule deviation value representing the deviation value between the intermediate business rule and the historical business rule, and the evaluation deviation value representing the deviation between the target evaluation value and the historical evaluation value; When the third loss value converges, the obtained Actor business rule extraction model is used as the trained business rule extraction model.

10. A device for extracting message service rules, characterized in that: The device comprises: An acquisition module, used to acquire at least one business message of the business rule to be extracted; An encoding module, used to encode the at least one service message; A determination module is used to input the encoded message into a trained business rule extraction model to obtain a target business rule corresponding to at least one business message, wherein the target business rule includes a feature value for identifying a fingerprint feature of the business message, and the feature value is determined from the business message based on a semantic analysis of the business message.

11. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor is used to perform the steps of the method for extracting message service rules according to any one of claims 1 to 9.

12. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device comprising a plurality of application programs, enables the electronic device to perform the steps of the method for extracting message business rules according to any one of claims 1 to 9.

Images