Protocol message transmission method and device, storage medium and electronic device

Abstract

The embodiment of the present application provides a method and device for transmitting a protocol message, a storage medium and an electronic device, wherein the method includes: obtaining an initial message to be transmitted to a terminal device through a target transmission protocol; searching for data in the initial message from a compressed data set; when the target data in the initial message is found from the compressed data set, replacing the target data in the initial message with the corresponding target index in the compressed data set to obtain a compressed message; transmitting the compressed message to the terminal device. Through this application, the problem of low transmission efficiency of protocol messages is solved, thereby achieving the effect of improving the transmission efficiency of protocol messages.

Description

Technical Field

[0001] The embodiments of the present application relate to the computer field, and specifically, to a method and device for transmitting a protocol message, a storage medium, and an electronic device. Background Art

[0002] In the related art, when compressing the protocol message to be transmitted, only the repeated content in a single protocol message is compressed. It can be understood that in the related art, the protocol message is compressed only based on the self-repetitiveness of the protocol message.

[0003] In actual situations, the more recently accessed data is, the easier it is to access next, that is, there may be similar data between data in different protocol messages. In the related art, there is less repeated content in such a single protocol message, and the compression efficiency is low. Summary of the invention

[0004] The embodiments of the present application provide a method and device for transmitting a protocol message, a storage medium, and an electronic device, so as to at least solve the problem of low transmission efficiency of protocol messages in the related art.

[0005] According to one embodiment of the present application, a method for transmitting a protocol message is provided, wherein a compressed data set corresponding to a target transmission protocol is stored in a server, and an index and reference data with a corresponding relationship are recorded in the compressed data set, and the reference data is data with an occurrence frequency greater than or equal to a frequency threshold that is screened out from a protocol log of the target transmission protocol recorded by the server. The method is applied to the server, and the method includes: obtaining an initial message to be transmitted to a terminal device via the target transmission protocol; searching for data in the initial message from the compressed data set; when the target data in the initial message is found from the compressed data set, replacing the target data in the initial message with the corresponding target index in the compressed data set to obtain a compressed message; and transmitting the compressed message to the terminal device.

[0006] In an exemplary embodiment, a log collector is deployed in the server. Before obtaining the initial message to be transmitted to the terminal device via the target transmission protocol, the method further includes: collecting the protocol log of the target transmission protocol recorded by the server by the log collector to obtain an initial log set, and parsing each protocol log in the initial log set by the log collector to obtain an initial log data set; filtering data with an occurrence frequency greater than or equal to the frequency threshold from the initial log data set to obtain the reference data; generating indexes corresponding to the reference data in order of the occurrence frequency from high to low, and recording the corresponding indexes and the reference data in order of the occurrence frequency from high to low.

[0007] In an exemplary embodiment, the filtering of data having an occurrence frequency greater than or equal to the frequency threshold from the initial log data set to obtain the reference data includes: extracting candidate data carried in the log data corresponding to each protocol log in the initial log data set, wherein the log data corresponding to at least two protocol logs in the initial log data set include the candidate data; determining a first occurrence frequency of the candidate data based on the candidate data and the initial log data set, and determining a second occurrence frequency of the candidate data based on the candidate data and the initial log data set; performing a product operation on the first occurrence frequency and the second occurrence frequency to obtain a target occurrence frequency of the candidate data; and filtering data having the target occurrence frequency greater than or equal to the frequency threshold from the candidate data to obtain the reference data.

[0008] In an exemplary embodiment, determining the first occurrence frequency of the candidate data according to the candidate data and the initial log data set includes: when the initial log data set includes N data and the candidate data includes M data, determining the i1th occurrence frequency of the i-th data in the M data by performing the following steps, wherein N is a positive integer, M is a positive integer, and i is a positive integer less than or equal to M: when the i-th data appears K times in the N data, determining the i1th occurrence frequency as a value obtained by dividing K by S, wherein S is the number of times the data with the largest number of occurrences in the N data appears in the N data, and S is a positive integer; The determining, according to the candidate data and the initial log set, the second occurrence frequency of the candidate data comprises: when the candidate data comprises M data, the initial log set comprises Q protocol logs, and P protocol logs among the Q protocol logs comprise the i-th data among the M data, determining the i2-th occurrence frequency of the i-th data by performing the following steps, wherein M is a positive integer, Q is a positive integer, P is a positive integer less than or equal to Q, and i is a positive integer less than or equal to M: performing a sum operation on P and a reference value to obtain a target sum value; dividing Q by the target sum value to obtain a target quotient value; performing a logarithm operation on the target quotient value to obtain the i2-th occurrence frequency.

[0009] In an exemplary embodiment, after generating the indexes corresponding to the reference data in the order of the occurrence frequency from high to low, and recording the corresponding indexes and the reference data in the order of the occurrence frequency from high to low, the method further includes: setting the validity period of the compressed data set to a target duration, wherein the target duration is equal to the duration difference between the target time and the generation time of the compressed data set, the generation time is earlier than the target time, and the target time is the latest time that the server and the terminal device are allowed to use the compressed data set to transmit protocol messages through the target transmission protocol; recording the generation time, validity period and version identifier of the compressed data set.

[0010] In an exemplary embodiment, after recording the generation time, validity period and version identifier of the compressed data set, the method further includes: when it is detected that the compressed data set is not within the validity period, collecting the update protocol log of the target transmission protocol recorded by the server through the log collector to obtain an update log set, and parsing each protocol log in the update log set through the log collector to obtain an update log data set; filtering data with an occurrence frequency greater than or equal to the frequency threshold from the update log data set to obtain update data; generating an update index corresponding to the update data, and recording the update index and the update data with a corresponding relationship in order of the occurrence frequency from high to low, wherein the update data set includes the update index and the update data with a corresponding relationship.

[0011] In an exemplary embodiment, a first transmission device is deployed in the server, a first decoder is deployed in the first transmission device, and the first decoder is connected to the terminal device. The acquisition of the initial message to be transmitted to the terminal device through the target transmission protocol includes: receiving a compression request message initiated by the terminal device; extracting the initial version identifier carried in the compression request message through the first decoder, and searching for a reference index in the compression request message in the compressed data set corresponding to the initial version identifier through the first decoder; when the reference index in the compression request message is found in the compressed data set through the first decoder, the reference index is replaced by the corresponding data to be compressed in the compressed data set through the first decoder to obtain the initial request message; and responding to the initial request message to obtain the initial message.

[0012] In an exemplary embodiment, searching for data in the initial message from the compressed data set includes: when the initial message includes R data and the compressed data set includes T indexes and T data having a corresponding relationship, searching for the sth data in the R data from the compressed data set by performing the following steps, wherein R is a positive integer, T is a positive integer, and s is a positive integer less than or equal to R: extracting the sth data set from the R data, wherein the sth data set includes the sth data and the sth data corresponding to the sth data L adjacent data, where L is a positive integer; searching for the sth data set from the T data; when each data in the sth data set is found from the T data, marking the sth data set as index data, and continuing to search for the s+L+1th data in the R data from the T data; when each data in the sth data set is not found from the T data, marking the sth data set as non-index data, and continuing to search for the s+L+1th data in the R data from the T data.

[0013] In an exemplary embodiment, a first transmission device is deployed in the server, a first encoder is deployed in the first transmission device, the first encoder is connected to the terminal device, and the transmission of the compressed message to the terminal device includes: performing a corresponding first encoding operation on the target index in the compressed message by the first encoder to obtain a first compressed encoded message, and performing a corresponding second encoding operation on other data in the compressed message by the first encoder to obtain a second compressed encoded message, wherein the other data includes data other than the target data in the compressed message, the first compressed encoded message includes data obtained by encoding the target index, or data obtained by encoding the number of bytes occupied by the target index, the second compressed encoded message includes data obtained by encoding the data volume of the other data and data obtained by encoding the other data, or the second compressed encoded message includes data obtained by encoding the number of bytes occupied by the data volume of the other data and data obtained by encoding the other data; carrying the initial version identifier of the compressed data set in the target compressed message, and transmitting the target compressed message to the terminal device, wherein the target compressed message includes the first compressed encoded message and the second compressed encoded message.

[0014] In an exemplary embodiment, before obtaining the initial message to be transmitted to the terminal device via the target transmission protocol, the method further includes: receiving a target data request initiated by the terminal device, wherein the target data request is used to request the acquisition of the compressed data set; responding to the target data request, extracting the compressed data set, and transmitting the compressed data set, the generation time of the compressed data set, the validity period of the compressed data set, and the initial version identifier of the compressed data set to the terminal device; or, receiving an update data request initiated by the terminal device, wherein the update data request is used to request the acquisition of an update data set; responding to the update data request, extracting the update data set, and transmitting the update data set, the generation time of the update data set, the validity period of the update data set, and the update version identifier of the update data set to the terminal device.

[0015] According to another embodiment of the present application, a method for transmitting a protocol message is provided, wherein a compressed data set corresponding to a target transmission protocol is stored in a terminal device, wherein the compressed data set records an index and reference data with a corresponding relationship, wherein the reference data is data whose occurrence frequency is greater than or equal to a frequency threshold and is screened out from a protocol log of the target transmission protocol recorded by a server, and the method is applied to the terminal device, and the method includes: receiving a compressed message transmitted by the server, and extracting an initial version identifier carried in the compressed message; searching for data in the compressed message from the compressed data set corresponding to the initial version identifier; and when the target compressed data in the compressed message is found in the compressed data set, replacing the target compressed data in the compressed message with the corresponding target data in the compressed data set to obtain a target message.

[0016] In an exemplary embodiment, before receiving the compressed message transmitted by the server, the method further includes: detecting whether the terminal device stores the compressed data set; in case of detecting that the compressed data set is not stored in the terminal device, initiating a target data request to the server, wherein the target data request is used to request to obtain the compressed data set, and receiving the compressed data set returned by the server in response to the target data request, the generation time of the compressed data set, the validity period of the compressed data set, and the initial version identifier of the compressed data set; in case of detecting that the compressed data set is stored in the terminal device, detecting whether the compressed data set is within the validity period, and in case of detecting that the compressed data set is not within the validity period, initiating an update data request to the server, wherein the update data request is used to request to obtain an updated data set, and receiving the updated compressed data set returned by the server in response to the update data request, the generation time of the updated data set, the validity period of the updated data set, and the updated version identifier of the updated data set.

[0017] In an exemplary embodiment, detecting whether the compressed data set is within the validity period includes: detecting whether a target duration difference between a current time and a target time is less than or equal to a difference threshold, wherein the current time is the time for detecting whether the compressed data set is within the validity period, and the current time is earlier than the target time; when it is detected that the target duration difference is less than or equal to the difference threshold, determining that the compressed data set is not within the validity period.

[0018] In an exemplary embodiment, a second transmission device is deployed in the terminal equipment, a second decoder is deployed in the second transmission device, and the second decoder is connected to the server. The searching for data in the compressed message from the compressed data set corresponding to the initial version identifier includes: performing a first decoding operation corresponding to a first encoding operation on a first compressed coded message by the second decoder to obtain a target index, and performing a second decoding operation corresponding to a second encoding operation on a second compressed coded message by the second decoder to obtain other data in the initial message except the target data; searching for the target data corresponding to the target index from the compressed data set corresponding to the initial version identifier, wherein the target message includes the target data and the other data.

[0019] In an exemplary embodiment, searching for the target data corresponding to the target index from the compressed data set corresponding to the initial version identifier includes: when the target index includes W indexes and the compressed data set includes T indexes and T data with corresponding relationships, searching for the t-th data corresponding to the t-th index among the W indexes from the T indexes and T data with corresponding relationships by performing the following steps, wherein the target data includes K data, the K data include the t-th data, W is a positive integer, T is a positive integer, and t is a positive integer equal to T: searching for the t-th index from the T indexes; when the t-th index is found from the T indexes, determining the data corresponding to the t-th index among the T indexes and T data with corresponding relationships as the t-th data.

[0020] In an exemplary embodiment, a second transmission device is deployed in the terminal device, a second encoder is deployed in the second transmission device, and the second encoder is connected to the server. Before receiving the compressed message transmitted by the server, the method also includes: generating an initial request message, wherein the initial request message is used to request to obtain the initial message; searching for the data in the initial request message in the compressed data set by the second encoder; when the data to be compressed in the initial request message is found from the compressed data set by the second encoder, the data to be compressed in the initial request message is replaced by the corresponding reference index in the compressed data set by the second encoder to obtain a compressed request message; initiating the compressed request message to the server, wherein the compressed request message carries the initial version identifier of the compressed data set, and the server is configured to obtain the initial message in response to the received initial request message, and replace the target data in the initial message with the compressed message obtained by the corresponding target index in the compressed data set and transmit it to the terminal device.

[0021] According to another embodiment of the present application, a device for transmitting a protocol message is provided, comprising: a compressed data set corresponding to a target transmission protocol is stored in a server, the compressed data set records indexes and reference data with corresponding relationships, the reference data is data with an appearance frequency greater than or equal to a frequency threshold screened out from a protocol log of the target transmission protocol recorded by the server, the device is applied to the server, the device comprises: an acquisition module, for acquiring an initial message to be transmitted to a terminal device via the target transmission protocol; a first search module, for searching for data in the initial message from the compressed data set; a first replacement module, for replacing the target data in the initial message with a corresponding target index in the compressed data set to obtain a compressed message when the target data in the initial message is found from the compressed data set; and a transmission module, for transmitting the compressed message to a terminal device.

[0022] According to another embodiment of the present application, another device for transmitting a protocol message is provided, comprising: a compressed data set corresponding to a target transmission protocol is stored in a terminal device, the compressed data set records an index and reference data with a corresponding relationship, the reference data is data with an occurrence frequency greater than or equal to a frequency threshold screened out from a protocol log of the target transmission protocol recorded by a server, the device is applied to the terminal device, the device comprises: a third receiving module, for receiving a compressed message transmitted by the server, and extracting an initial version identifier carried in the compressed message; a second search module, for searching for data in the compressed message from the compressed data set corresponding to the initial version identifier; a second replacement module, for replacing the target compressed data in the compressed message with the corresponding target data in the compressed data set, when the target compressed data in the compressed message is found in the compressed data set, to obtain a target message.

[0023] According to another embodiment of the present application, a server system is further provided, the server system comprising a connected server and a terminal device, wherein the server and the terminal device are configured to execute the steps of any of the above method embodiments during operation.

[0024] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0025] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0026] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0027] Through the present application, the server creates corresponding compressed data sets for different transmission protocols. The compressed data sets record the data that appear frequently in the protocol log of the transmission protocol. When the server transmits a protocol message to a terminal device through the transmission protocol, the server replaces the data in the protocol message with the data in the compressed data set. In this way, the data that appears frequently in the protocol messages transmitted through the transmission protocol between different servers and terminal devices are replaced with corresponding indexes, thereby reducing the amount of data in the protocol messages transmitted between the server and the terminal device. Therefore, the problem of low transmission efficiency of the protocol messages can be solved, thereby achieving the effect of improving the transmission efficiency of the protocol messages. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a hardware structure block diagram of a server device of a method for transmitting a protocol message in an embodiment of the present application;

[0029] Figure 2 is a flowchart of a method for transmitting a protocol message according to an embodiment of the present application;

[0030] Figure 3 is a schematic diagram of an optional network architecture according to an embodiment of the present application;

[0031] Figure 4 is a schematic diagram of an optional compressed data set according to an embodiment of the present application;

[0032] Figure 5 is a schematic diagram of an optional compressed bytecode according to an embodiment of the present application;

[0033] Figure 6 is a flowchart of a method for transmitting a protocol message according to an embodiment of the present application;

[0034] Figure 7 is a schematic diagram of transmission of an optional protocol message according to an embodiment of the present application;

[0035] Figure 8 is a schematic diagram of an optional index dictionary table according to an embodiment of the present application;

[0036] Fig. 9 It is a structural block diagram of an optional protocol message transmission device according to an embodiment of the present application;

[0037] Fig.10 It is a structural block diagram of another optional protocol message transmission device according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0039] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0040] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a method for transmitting a protocol message in an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a microprocessor MCU (Microcontroller Unit) or a programmable logic device FPGA (Field Programmable Gate Array) and a memory 104 for storing data, wherein the server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0041] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the transmission method of the protocol message in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0042] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0043] First, the terms involved in the embodiments of the present application are explained as follows:

[0044] Szip: Statistical HTTP Compression, a method based on statistical analysis of HTTP messages and building a dynamic index dictionary to compress communication data.

[0045] Sdict: System Dictionary, a dynamic index dictionary obtained based on statistical analysis of message logs in the application system dimension.

[0046] Self-repetitiveness: The higher the frequency and quantity of the same characters or strings in the text content, the higher the self-repetitiveness.

[0047] In this embodiment, a method for transmitting a protocol message is provided. A compressed data set corresponding to a target transmission protocol is stored in a server. The compressed data set records an index and reference data with a corresponding relationship. The reference data is data with an appearance frequency greater than or equal to a frequency threshold that is screened out from a protocol log of the target transmission protocol recorded by the server. The method is applied to the server. Figure 2 is a flow chart of a method for transmitting a protocol message according to an embodiment of the present application, such as Figure 2As shown, the process includes the following steps:

[0048] Step S202, obtaining an initial message to be transmitted to a terminal device via the target transmission protocol;

[0049] Step S204, searching the compressed data set for data in the initial message;

[0050] Step S206, when the target data in the initial message is found in the compressed data set, the target data in the initial message is replaced with the corresponding target index in the compressed data set to obtain a compressed message;

[0051] Step S208: Transmit the compressed message to the terminal device.

[0052] Through the above steps, the server creates corresponding compressed data sets for different transmission protocols. The compressed data sets record the data that appear frequently in the protocol log of the transmission protocol. When the server transmits a protocol message to the terminal device through the transmission protocol, the server replaces the data in the protocol message with the data in the compressed data set. In this way, the data that appears frequently in the protocol messages transmitted through the transmission protocol between different servers and terminal devices is replaced with the corresponding index, thereby reducing the data volume of the protocol messages transmitted between the server and the terminal device. Therefore, the problem of low transmission efficiency of the protocol messages can be solved, thereby achieving the effect of improving the transmission efficiency of the protocol messages.

[0053] In the technical solution provided in the above step S202, the target transmission protocol can be but is not limited to being used for communication between the server and the terminal device. For example, the target transmission protocol can be but is not limited to including the http (Hypertext Transfer Protocol) protocol, and can be but is not limited to storing corresponding compressed data sets for different transmission protocols. It can be understood that the compressed data sets corresponding to different transmission protocols can be but are not limited to being different. It should be noted that the reference data recorded in the compressed data set may change with the different data communicated between the server and the terminal device through the target transmission protocol. It can be understood that the reference data changes dynamically.

[0054] Optionally, in this embodiment, the protocol log of the target transmission protocol may be generated in the following manner, but is not limited to: the protocol log may be generated by the protocol message transmitted between the server and the terminal device through the target transmission protocol. In this way, the content related to the target transmission protocol on the server may generate a protocol log, which improves the matching degree between the generated compressed data set and the content to be transmitted through the target transmission protocol.

[0055] In an exemplary embodiment, a log collector can be deployed in the server in the following manner but is not limited to the following manner, before obtaining the initial message to be transmitted to the terminal device via the target transmission protocol: the protocol log of the target transmission protocol recorded by the server is collected by the log collector to obtain an initial log set, and each protocol log in the initial log set is parsed by the log collector to obtain an initial log data set; data with an occurrence frequency greater than or equal to the frequency threshold is filtered from the initial log data set to obtain the reference data; indexes corresponding to the reference data are generated in order of the occurrence frequency from high to low, and the indexes and the reference data with corresponding relationships are recorded in order of the occurrence frequency from high to low.

[0056] Optionally, in this embodiment, the arrangement order of the indexes in the compressed data set may be, but is not limited to, positively correlated with the frequency of occurrence of the reference data. For example, when the indexes include 01 to 10, the frequency of occurrence of the data corresponding to the index value of 01 is higher than the frequency of occurrence of the data corresponding to the index value of 8.

[0057] Optionally, in this embodiment, a higher frequency of occurrence of data indicates that a higher frequency of occurrence of the data in the protocol message transmitted between the server and the terminal device through the target transmission protocol in the current time period is obtained. It can be understood that the data is frequently accessed by the terminal device and the server.

[0058] In an exemplary embodiment, the reference data can be obtained by filtering data whose occurrence frequency is greater than or equal to the frequency threshold from the initial log data set, but is not limited to the following method: extracting candidate data carried in the log data corresponding to each protocol log in the initial log data set, wherein the log data corresponding to at least two protocol logs in the initial log data set include the candidate data; determining a first occurrence frequency of the candidate data based on the candidate data and the initial log data set, and determining a second occurrence frequency of the candidate data based on the candidate data and the initial log data set; performing a product operation on the first occurrence frequency and the second occurrence frequency to obtain a target occurrence frequency of the candidate data; and filtering data whose target occurrence frequency is greater than or equal to the frequency threshold from the candidate data to obtain the reference data.

[0059] Optionally, in this embodiment, the protocol log may include, but is not limited to, log data, and the candidate data may include, but is not limited to, words in the log data, such as phrases or words, etc. For example, the protocol log may include, but is not limited to: on A year B month C day, the usage rate of the host storage pool exceeds the minor alarm threshold, and the candidate data may include, but is not limited to: host, storage pool, usage rate, exceed, and minor alarm threshold.

[0060] In an exemplary embodiment, the first occurrence frequency of the candidate data can be determined based on the candidate data and the initial log data set in the following manner, but is not limited to: when the initial log data set includes N data and the candidate data includes M data, the i1th occurrence frequency of the i-th data in the M data is determined by performing the following steps, wherein N is a positive integer, M is a positive integer, and i is a positive integer less than or equal to M: when the i-th data appears K times in the N data, the i1th occurrence frequency is determined as a value obtained by dividing K by S, wherein S is the number of times the data with the largest number of occurrences in the N data appears in the N data, and S is a positive integer. integer; the determining, according to the candidate data and the initial log set, the second occurrence frequency of the candidate data comprises: when the candidate data comprises M data, the initial log set comprises Q protocol logs, and P protocol logs among the Q protocol logs include the i-th data among the M data, determining the i2-th occurrence frequency of the i-th data by performing the following steps, wherein M is a positive integer, Q is a positive integer, P is a positive integer less than or equal to Q, and i is a positive integer less than or equal to M: performing a sum operation on P and a reference value to obtain a target sum value; dividing Q by the target sum value to obtain a target quotient value; performing a logarithm operation on the target quotient value to obtain the i2-th occurrence frequency.

[0061] Optionally, in this embodiment, the reference value may include but is not limited to a value set based on experience or a value screened through multiple experiments. As an optional example, the reference value may include but is not limited to 1.

[0062] Optionally, in this embodiment, the i-th occurrence frequency of the i-th data may be determined in the following manner but is not limited to: performing a product operation on the i1-th occurrence frequency and the i2-th occurrence frequency to obtain the i-th occurrence frequency of the i-th data.

[0063] In an exemplary embodiment, the index corresponding to the reference data can be generated in the order of the occurrence frequency from high to low, and the index and the reference data with a corresponding relationship are recorded in the order of the occurrence frequency from high to low in the following manner, but is not limited to: the validity period of the compressed data set is set to a target duration, wherein the target duration is equal to the duration difference between the target time and the generation time of the compressed data set, the generation time is earlier than the target time, and the target time is the latest time that the server and the terminal device are allowed to use the compressed data set to transmit protocol messages through the target transmission protocol; the generation time, validity period and version identifier of the compressed data set are recorded.

[0064] Optionally, in this embodiment, within the validity period of the compressed data set, the server and the terminal device are configured to allow the use of the compressed data set to transmit protocol messages through the target transmission protocol, and when the compressed data set exceeds the validity period, the server and the terminal device are configured not to allow the use of the compressed data set to transmit protocol messages through the target transmission protocol.

[0065] Optionally, in this embodiment, the version identifiers of the compressed data sets generated by the protocol logs of the target transmission protocol recorded by the server in different time periods may be different but are not limited to being different. For example, the version identifier may include but is not limited to a version number.

[0066] Figure 3 is a schematic diagram of an optional network architecture according to an embodiment of the present application, such as Figure 3 As shown, both the terminal and the server are deployed with a message compression device, in which an encoder and a decoder are deployed. The message compression device of the terminal can be connected to the message compression device of the server through a WAN (Wide Area Network, wide area network) but is not limited to it. A message log collector (equivalent to a log collector) is set on the server side to obtain and parse the original HTTP request and response messages. The message log collector intercepts the request messages received by the server and the response messages returned, and obtains their original content in a natural language format.

[0067] The message content can be divided into two parts, but not limited to, the original first part is the message line and message header of the HTTP message. In the HTTP / 2 protocol, it can be compressed using the HPACK (Header Compression for HTTP / 2) algorithm, but not limited to. Under normal circumstances, the variable data in the message line and the message header generally does not account for a high proportion, and the relative change rate of its content and format is low, and the correlation with the business data in the message body is not strong. The matching degree with the compressed data set (for example, the Sdict dictionary) created by the embodiment of the present application is average. Therefore, the embodiment of the present application does not perform additional processing on the message header; the original second part is the message body of the HTTP message, and its content is written into the cache database as a time series log. Unless otherwise specified, the message logs described subsequently refer to the message body time series logs of the HTTP message.

[0068] It is possible but not limited to setting a message storage device on the server side to store the original log data collected by the collector. It should be noted that the embodiments of the present application do not limit the collection and storage methods of message logs. At present, the common solution in the industry for log data storage of message log-related types is the ELK architecture. For the first transmission device, the only logs that need to be collected are message logs, and there is no demand for aggregated search, near real-time search, etc. Therefore, for the first transmission device, a custom lightweight log collector can interact directly with the message parser, and storage can choose HBase (Hadoop Database) or Elasticsearch (Elastic Search), etc.

[0069] Extract all log documents in the recent period from massive time series logs as corpus. You can use, but are not limited to, the TF-IDF algorithm to extract keywords (equivalent to candidate data) for each log, and store the results in a hash table, where the key is the keyword and the value is the TF-IDF value (the value obtained from each analysis is accumulated). Sort the final hash table key-value pairs by value from large to small to obtain the software dynamic index dictionary table Sdict (equivalent to a compressed data set). Figure 4 is a schematic diagram of an optional compressed data set according to an embodiment of the present application, such as Figure 4 As shown, Index indicates the index, Name indicates the name of the candidate data, and the index can be, but is not limited to, 0 to 10. For example, index 0 corresponds to data 1, and index 1 corresponds to data 2. It should be noted that the range and quantity of extracted logs are not fixed, and need to be based on the actual application scenario and log magnitude. Considering many periodic application scenarios, it is recommended to take a date range of one day, one week, one month, etc.

[0070] After generating the compressed data set, the validity period of the software dynamic index dictionary table Sdict (equivalent to the compressed data set) can also be set, but not limited to, starting from the time when the dynamic index dictionary table Sdict is generated (equivalent to the generation time), which can be set to one day, one week, etc. The embodiment of the present application does not limit the size of the validity period, but it should be noted that the larger the validity period, the slower the iteration speed of the dynamic index dictionary table Sdict, and the adaptability of the old version to the latest communication data will be reduced, thereby affecting the compression effect; in addition, a version identifier (for example, version number) can be set for each dynamic index dictionary table Sdict, but not limited to, and the generation date, validity period, and version number of the dynamic index dictionary table Sdict are combined and regarded as the signature of the dynamic index dictionary table Sdict.

[0071] In this way, due to the periodic update mechanism of the dynamic index dictionary, the index dictionary is version-managed and lifecycle-managed. The communicating parties align the compression dictionary version used by specifying the message header key-value pair, and by setting the validity period and version number for the dictionary, the validity and high matching degree of the dictionary used are ensured, thereby ensuring the stability and security of the compression process.

[0072] In an exemplary embodiment, after recording the generation time, validity period and version identifier of the compressed data set, the following method may be used, but is not limited to: when it is detected that the compressed data set is not within the validity period, the update protocol log of the target transmission protocol recorded by the server is collected by the log collector to obtain an update log set, and each protocol log in the update log set is parsed by the log collector to obtain an update log data set; data with an occurrence frequency greater than or equal to the frequency threshold is filtered from the update log data set to obtain update data; an update index corresponding to the update data is generated, and the update index and the update data with a corresponding relationship are recorded in order from high to low according to the occurrence frequency, wherein the update data set includes the update index and the update data with a corresponding relationship.

[0073] Optionally, in this embodiment, after recording the generation time, validity period and version identifier of the compressed data set, the above method also includes: when it is detected that the compressed data set is within the validity period and has reached the set update cycle, collecting the update protocol log of the target transmission protocol recorded by the server through the log collector to obtain an update log set, and parsing each protocol log in the update log set through the log collector to obtain an update log data set; filtering data with an occurrence frequency greater than or equal to the frequency threshold from the update log data set to obtain update data; generating an update index corresponding to the update data, and recording the update index and update data with a corresponding relationship in order of occurrence frequency from high to low, wherein the update data set includes update indexes and update data with a corresponding relationship.

[0074] Optionally, in this embodiment, the updated data may be, but is not limited to, completely different from the reference data, or partially different, and the index corresponding to the updated data may be, but is not limited to, generated in descending order of frequency of occurrence.

[0075] In an exemplary embodiment, a first transmission device may be deployed in the server in the following manner but is not limited to the following manner, wherein a first decoder is deployed in the first transmission device, and the first decoder is connected to the terminal device, and the initial message to be transmitted to the terminal device through the target transmission protocol is obtained: a compression request message initiated by the terminal device is received; an initial version identifier carried in the compression request message is extracted by the first decoder, and a reference index in the compression request message is searched for in the compressed data set corresponding to the initial version identifier by the first decoder; when the reference index in the compression request message is found in the compressed data set by the first decoder, the reference index is replaced by the corresponding data to be compressed in the compressed data set by the first decoder to obtain an initial request message; and the initial request message is responded to to obtain the initial message.

[0076] Optionally, in this embodiment, when the compression request message initiated by the terminal device uses a compressed data set, the first decoder may carry, from the message header of the request message, but is not limited to, an initial version identifier of the compressed data set used to compress the request message.

[0077] Optionally, in this embodiment, when a compression request message initiated by a terminal device is received, the above method also includes: extracting an initial version identifier carried in the compression request message through the first decoder, but not limited to; when the initial version identifier is extracted from the compression request message through the first decoder, the received compression request message can be decompressed by the first decoder using the compressed data set corresponding to the initial version identifier, but not limited to, to obtain the initial request message before compression.

[0078] In the technical solution provided in the above step S204, the data in the initial message can be searched from the compressed data set in the following manners, but not limited to: searching for the data in the initial message from the reference data recorded in the compressed data set, or comparing the data in the initial message with the reference data.

[0079] In an exemplary embodiment, the data in the initial message can be searched from the compressed data set in the following manner, but is not limited to: when the initial message includes R data and the compressed data set includes T indexes and T data with a corresponding relationship, the sth data among the R data is searched from the compressed data set by performing the following steps, wherein R is a positive integer, T is a positive integer, and s is a positive integer less than or equal to R: extracting the sth data set from the R data, wherein the sth data set includes the sth data and the data corresponding to the sth data set; L data adjacent to s data, where L is a positive integer; searching for the s-th data set from the T data; when each data in the s-th data set is found from the T data, marking the s-th data set as index data, and continuing to search for the s+L+1-th data in the R data from the T data; when each data in the s-th data set is not found from the T data, marking the s-th data set as non-index data, and continuing to search for the s+L+1-th data in the R data from the T data.

[0080] Optionally, in this embodiment, the sth data set may include but is not limited to including L data starting from the sth data, or including the sth data and sLth data, or the sth data and s+Lth data, or the sath data, the sth data and s+(La)th data, etc.

[0081] For example, it is possible but not limited to compare the first character of the message content of the initial message with the index words in the index dictionary Sdict (equivalent to the compressed data set). If the string consisting of N adjacent characters starting with the character can match the corresponding index word, then mark the string consisting of N characters as the index word, and the next match starts from the N+1th character; if the index word cannot be matched, the next match starts from the next character adjacent to the character, and if there are adjacent unmatched characters, they are merged into one string for processing. According to this method, the original content is marked as a non-index word string and an index word string in sequence, and processed according to the first processing method and the second processing method respectively.

[0082] As an optional example, matching can be performed using, but is not limited to, a forward maximum matching method (or other algorithms, which are not limited in this application). In terms of algorithm efficiency, the forward maximum matching method of the example in this application has a time complexity of ○(n) and a space complexity of ○(0), and has very high compression and decompression efficiency.

[0083] In the technical solution provided in the above step S206, when the alternative data in the initial message is not found from the compressed data set, the alternative data in the initial message can be but is not limited to being retained. It can be understood that the compressed message can include but is not limited to including the target index and the alternative data.

[0084] Optionally, in this embodiment, the initial message may include, but is not limited to, A year B month C day, the usage rate of the host storage pool exceeds the secondary alarm threshold, the reference data may include, but is not limited to: host, storage pool, usage rate, exceed and secondary alarm threshold, the indexes corresponding to the host, storage pool, usage rate, exceed and secondary alarm threshold are 01, 02, 03, 04 and 05 respectively, in this case, the compressed message may include, but is not limited to: A year B month C day, 0102 030405. In this way, the data that often appears in multiple protocol logs is replaced with the corresponding index, so that the compression of the initial message is realized, and the amount of data transmitted between the terminal device and the server is greatly reduced.

[0085] As an optional example, a message compression device (equivalent to the first transmission device and the second transmission device) is provided at both the sender and the receiver, responsible for compressing and decompressing the message body content. The server and the terminal device can be, but are not limited to, both the sender and the receiver.

[0086] When the sender sends data, the system dictionary (equivalent to a compressed data set) is used to compress the message body of the request message. For example, the key-value pair Szip={index dictionary version number} is added to the message header. The client and server determine whether the received communication data is Szip compressed based on this attribute. If so, it needs to be decompressed accordingly according to the compression logic, otherwise no processing is done.

[0087] Specifically, the method for performing Szip compression on the message body is: based on the index dictionary table Sdict, the forward maximum matching method can be used but is not limited to scan the message body content of the message, and the first processing method is used for the unmatched character string, and the second processing method is used for the matched keyword character string, and finally a compressed binary byte array is obtained.

[0088] In the technical solution provided in the above step S208, the compressed message may, but is not limited to, carry a version identifier of the compressed data set. It can be understood that when the terminal device receives the compressed message, it can decompress the compressed message using the compressed data set corresponding to the version identifier. In this way, the consistency of the compressed data set used for data transmission between the terminal device and the server through the target transmission protocol is ensured.

[0089] In an exemplary embodiment, a first transmission device may be deployed in the server in the following manner, wherein a first encoder is deployed in the first transmission device, and the first encoder is connected to the terminal device, and the compressed message is transmitted to the terminal device by: performing a corresponding first encoding operation on the target index in the compressed message by the first encoder to obtain a first compressed encoded message, and performing a corresponding second encoding operation on other data in the compressed message by the first encoder to obtain a second compressed encoded message, wherein the other data includes data in the compressed message other than the target data, and the first compressed encoded message includes the data on the target index. The data obtained by encoding the target index, or the data obtained by encoding the number of bytes occupied by the target index, the second compressed encoded message includes the data obtained by encoding the data volume of the other data and the data obtained by encoding the other data, or the second compressed encoded message includes the data obtained by encoding the number of bytes occupied by the data volume of the other data and the data obtained by encoding the other data; the initial version identifier of the compressed data set is carried in the target compressed message, and the target compressed message is transmitted to the terminal device, wherein the target compressed message includes the first compressed encoded message and the second compressed encoded message.

[0090] Optionally, in this embodiment, the index data and non-index data may, but are not limited to, have corresponding encoding operations, may, but are not limited to, performing corresponding encoding operations on the index data and non-index data, and may, but are not limited to, carrying the initial version identifier in at least one of the first compressed message and the second compressed message.

[0091] For example, a non-index word string (equivalent to non-index data) can be processed by, but not limited to, the first processing method (equivalent to the second encoding operation), and the encoded byte array includes two parts. The first part can include, but is not limited to, two situations: starting with 00, and the following 6 bits are the length value of the word's byte array, occupying 1 byte; starting with 01, and the following 6 bits are the number of bytes occupied by the length value, occupying N bytes. The second part is the original byte array of the word, and the array length is represented by referring to the length value in the first part. Figure 5 is a schematic diagram of an optional compressed bytecode according to an embodiment of the present application, such as Figure 5 As shown, when the length of String Data is 6, it starts with 00, and the following 6 bits (for example, 000110) are the byte array length value 6 of the word, or, when the length of String Data is 64, it starts with 01, and the following is the byte array length value 64 of the word.

[0092] The index word string (equivalent to index data) can be processed by the second processing method (equivalent to the first encoding operation) but is not limited to it, and the encoded byte array represents the index value of the index word in the Sdict dictionary table. There are two cases: starting with 10, the following 6 bits are the index value of the index word in the Sdict dictionary table, occupying 1 byte; starting with 11, the following 14 bits are the number of bytes occupied by the index value, occupying 2 bytes (the corresponding index dictionary can support but is not limited to 16384 index words).

[0093] It should be noted that in the embodiment of the present application, the number of index terms in the index dictionary Sdict may be limited to no more than 16384, but is not limited thereto. The larger the number of index terms, the higher the compression ratio will be, but the corresponding retrieval efficiency of the index dictionary will be reduced, resulting in reduced compression efficiency. The index terms in the index dictionary Sdict are sorted from high to low by score. For indexes with index values ​​of 64 and later, their index values ​​occupy 2 bytes, and the compression ratio is reduced by half relative to index values ​​before 64. In practical applications, different optional compression levels can also be designed by limiting the number of terms in the index dictionary. The more index terms are used, the higher the compression ratio and the slightly increased compression time. Conversely, the compression ratio is reduced and the compression time is reduced.

[0094] In an exemplary embodiment, before obtaining the initial message to be transmitted to the terminal device via the target transmission protocol, the following methods may be used, but are not limited to: receiving a target data request initiated by the terminal device, wherein the target data request is used to request obtaining the compressed data set; responding to the target data request, extracting the compressed data set, and transmitting the compressed data set, the generation time of the compressed data set, the validity period of the compressed data set, and the initial version identifier of the compressed data set to the terminal device; or, receiving an update data request initiated by the terminal device, wherein the update data request is used to request obtaining an updated data set; responding to the update data request, extracting the updated data set, and transmitting the updated data set, the generation time of the updated data set, the validity period of the updated data set, and the updated version identifier of the updated data set to the terminal device.

[0095] Optionally, in this embodiment, the terminal device may not store the compressed data set. In such a case, the terminal device may, but is not limited to, initiate a target data request to the server, and the server may, but is not limited to, respond to the target data request and transmit the compressed data set as well as the generation time of the compressed data set, the validity period of the compressed data set, and the initial version identifier of the compressed data set to the terminal device.

[0096] Optionally, in this embodiment, the compressed data set stored in the terminal device may have expired. In such a case, the terminal device may, but is not limited to, initiate a data update request to the server, and the server may, but is not limited to, respond to the data update request and transmit the updated data set as well as the generation time of the updated data set, the validity period of the updated data set, and the version identifier of the updated data set to the terminal device.

[0097] As an optional example, the interaction mechanism between the client and server of Sdict may be designed, but is not limited to, and the request may be triggered by default on the software Index page, or it may be designed as a manual option, and the client may choose to trigger the request by itself.

[0098] When the server receives the Sdict acquisition request, it sends the latest dynamic index dictionary table to the client. The sent data here includes the dynamic encoding dictionary table (equivalent to the compressed data set) and its signature.

[0099] The server will periodically update the index dictionary table (equivalent to a compressed data set). The client and server update and use the index dictionary table based on the following principles:

[0100] 1) If there is no index dictionary (equivalent to a compressed data set), the client will request the index dictionary from the server (only one request will be triggered within a certain period of time); the server will try to generate an index dictionary. The client checks the validity period of the index dictionary it holds. If it is about to expire, it will request the server to update the dictionary. If the latest version of the dictionary is successfully obtained, the new version of the dictionary will be used in the next request;

[0101] 2) The version of the encoding index dictionary of the server's response message must be consistent with the client's request encoding version;

[0102] 3) Expired dictionaries are not allowed to be used.

[0103] It should be noted that for dictionary version switching, adjacent versions of dictionaries are allowed to exist at the same time. The new version of the dictionary is generated some time before the old version of the dictionary becomes invalid. When the new and old dictionaries exist and are valid at the same time, the server determines the dictionary version used for this communication based on the version number passed in the message header (equivalent to the initial version identifier).

[0104] In this embodiment, another method for transmitting a protocol message is provided. A compressed data set corresponding to a target transmission protocol is stored in a terminal device. The compressed data set records an index and reference data with a corresponding relationship. The reference data is data with an appearance frequency greater than or equal to a frequency threshold that is screened out from a protocol log of the target transmission protocol recorded by a server. The method is applied to the terminal device. Figure 6 is a flow chart of a method for transmitting a protocol message according to an embodiment of the present application, such as Figure 6 As shown, the process includes the following steps:

[0105] Step S602, receiving a compressed message transmitted by the server, and extracting an initial version identifier carried in the compressed message;

[0106] Step S604, searching for data in the compressed message from the compressed data set corresponding to the initial version identifier;

[0107] Step S606: When the target compressed data in the compressed message is found in the compressed data set, the target compressed data in the compressed message is replaced with the corresponding target data in the compressed data set to obtain the target message.

[0108] Through the above steps, when the terminal device receives the compressed message sent by the server, it can, but is not limited to, use the compressed data set corresponding to the initial version identifier to decompress the compressed message to obtain the target message before compression. In this way, the protocol message transmitted between the terminal device and the server is compressed, which reduces the data volume of the transmitted protocol message. Therefore, the problem of low transmission efficiency of the protocol message can be solved, thereby achieving the effect of improving the transmission efficiency of the protocol message.

[0109] In the technical solution provided in the above step S302, the message header of the compressed message may, but is not limited to, carry an initial version identifier. A second transmission device is deployed in the terminal device, a second decoder is deployed in the second transmission device, and the second decoder is connected to the server. The initial version identifier carried in the compressed message may, but is not limited to, be extracted in the following manner: extracting the initial version identifier from the compressed message by the second decoder.

[0110] In an exemplary embodiment, before receiving the compressed message transmitted by the server, it is possible but not limited to the following methods: detecting whether the terminal device stores the compressed data set; in the case where it is detected that the compressed data set is not stored in the terminal device, initiating a target data request to the server, wherein the target data request is used to request to obtain the compressed data set, and receiving the compressed data set returned by the server in response to the target data request, the generation time of the compressed data set, the validity period of the compressed data set, and the initial version identifier of the compressed data set; in the case where it is detected that the compressed data set is stored in the terminal device, detecting whether the compressed data set is within the validity period, and in the case where it is detected that the compressed data set is not within the validity period, initiating an update data request to the server, wherein the update data request is used to request to obtain an updated data set, and receiving the updated compressed data set returned by the server in response to the update data request, the generation time of the updated data set, the validity period of the updated data set, and the updated version identifier of the updated data set.

[0111] Optionally, in this embodiment, when it is detected that the compressed data set is within the validity period but is about to expire, a request for updating data is initiated to the server, wherein the terminal device and the server are configured to use the updated data set to compress the protocol message to be transmitted when the protocol message is transmitted via the target transmission protocol next time.

[0112] In an exemplary embodiment, whether the compressed data set is within the validity period can be detected in the following manner but is not limited to: detecting whether the target duration difference between the current time and the target time is less than or equal to a difference threshold, wherein the current time is the time for detecting whether the compressed data set is within the validity period, and the current time is earlier than the target time; when it is detected that the target duration difference is less than or equal to the difference threshold, it is determined that the compressed data set is not within the validity period.

[0113] Optionally, in this embodiment, whether the compressed data set is within the validity period may also be detected in the following manner but is not limited to: detecting whether the current time is later than the target time, and if it is detected that the current time is later than the target time, determining that the compressed data set is not within the validity period.

[0114] In the technical solution provided in the above step S304, the terminal device may store, but is not limited to, compressed data sets corresponding to multiple version identifiers, for example, compressed data sets corresponding to different transmission protocols, or compressed data sets of different versions corresponding to the same transmission protocol.

[0115] In an exemplary embodiment, a second transmission device may be deployed in the terminal device in the following manner but is not limited to the following manner, a second decoder is deployed in the second transmission device, the second decoder is connected to the server, and the data in the compressed message is searched from the compressed data set corresponding to the initial version identifier: a first decoding operation corresponding to the first encoding operation is performed on the first compressed encoded message by the second decoder to obtain a target index, and a second decoding operation corresponding to the second encoding operation is performed on the second compressed encoded message by the second decoder to obtain other data in the initial message except the target data; the target data corresponding to the target index is searched from the compressed data set corresponding to the initial version identifier, wherein the target message includes the target data and the other data.

[0116] Optionally, in this embodiment, the first compressed encoded message may include but is not limited to data obtained after performing corresponding encoding operations on index data, and the second compressed encoded message may include but is not limited to data obtained after performing corresponding encoding operations on non-index data. It may include but is not limited to performing corresponding decoding operations on the first compressed encoded message through a second decoder to obtain a target index, and performing corresponding decoding operations on the second compressed encoded message through a second decoder to obtain other data.

[0117] Optionally, in this embodiment, the target data corresponding to the target index can be searched from the compressed data set corresponding to the initial version identifier in the following manner but is not limited to: searching for the target data corresponding to the target index from the reference data included in the compressed data set, or comparing the target index with the index included in the compressed data set.

[0118] In an exemplary embodiment, the target data corresponding to the target index can be searched from the compressed data set corresponding to the initial version identifier in the following manner, but not limited to: when the target index includes W indexes and the compressed data set includes T indexes and T data with corresponding relationships, the t-th data corresponding to the t-th index among the W indexes are searched from the T indexes and T data with corresponding relationships by performing the following steps, wherein the target data includes K data, the K data include the t-th data, W is a positive integer, T is a positive integer, and t is or is equal to T. Search the t-th index from the T indexes; when the t-th index is found from the T indexes, determine the data corresponding to the t-th index among the T indexes and T data with corresponding relationships as the t-th data.

[0119] Optionally, in this embodiment, if the tth index is not found from T indexes, it is determined that the tth data is not found, and the search for the tth index from T indexes is retried a target number of times. If the tth data is still not found, the compressed data set is obtained from the server again.

[0120] In an exemplary embodiment, a second transmission device may be deployed in the terminal device in the following manner but is not limited to the following manner, wherein a second encoder is deployed in the second transmission device, and the second encoder is connected to the server, before receiving the compressed message transmitted by the server: an initial request message is generated, wherein the initial request message is used to request to obtain the initial message; the data in the initial request message is searched for in the compressed data set by the second encoder; when the data to be compressed in the initial request message is found from the compressed data set by the second encoder, the data to be compressed in the initial request message is replaced by the corresponding reference index in the compressed data set by the second encoder to obtain a compressed request message; the compressed request message is initiated to the server, wherein the compressed request message carries the initial version identifier of the compressed data set, and the server is configured to obtain the initial message in response to the received initial request message, and replace the target data in the initial message with the compressed message obtained by the corresponding target index in the compressed data set and transmit it to the terminal device.

[0121] Optionally, in this embodiment, the sender of data may include but is not limited to a terminal device or a server. When a compressed data set is stored in the terminal device, the terminal device may include but is not limited to using the compressed data set to compress the initial request message to be initiated to the server. In this way, the amount of data in the request message initiated by the terminal device to the server is reduced, thereby improving the efficiency of communication between the terminal device and the server through the target transmission protocol.

[0122] In the technical solution provided in the above-mentioned step S306, when the target compressed data in the compressed message is not found from the compressed data set, the target compressed data in the compressed message may be searched from the compressed data set for a number of references but is not limited to. When the target compressed data is still not found, it may indicate that the compressed data set stored in the terminal device is inconsistent with the compressed data set used when the server sends the message. In this case, the terminal device may, but is not limited to, re-obtain the compressed data set from the server.

[0123] In order to better understand the transmission process of the protocol message in the embodiment of the present application, the transmission process of the protocol message in the embodiment of the present application is explained and illustrated in combination with optional embodiments below, which can be applicable to but not limited to the embodiment of the present application.

[0124] The sender transmits the compressed byte array to the receiver through the network. The receiver decompresses it according to the same logic to obtain the original message body content, completing a complete communication process. Figure 7 is a schematic diagram of transmission of an optional protocol message according to an embodiment of the present application, such as Figure 7 As shown, a complete request and response process may include but is not limited to the following steps:

[0125] Step S701: The client sends service request data.

[0126] Step S702: The client compression device determines whether the data is compressible; if it is determined that the data is compressible, step S703 is executed; if it is determined that the data is not compressible, step S704 is executed.

[0127] Step S703: Use the dynamic index dictionary to compress the message body content and set the message header compression flag, and then execute step S704.

[0128] Step S704: Send the content through the network.

[0129] Step S705: The server-side compression device determines whether decompression is required. If it is determined that decompression is required, step S706 is executed; if it is determined that decompression is not required, step S707 is executed.

[0130] Step S706: Use the dynamic index dictionary to decompress the message body content to obtain the original message.

[0131] For example, in an application scenario of a data center server status monitoring system, the server status monitoring system continuously obtains the operation status logs and error information logs of all devices in the data center server cluster, and can obtain a large number of logs every day and report them to the data platform. Through the embodiment of the present application, the data platform collects request logs and extracts index dictionary tables. Figure 8 is a schematic diagram of an optional index dictionary table according to an embodiment of the present application, such as Figure 8 As shown in the figure, Index represents the index, Name represents the name of the candidate data, and the index dictionary table records indexes 0 to 13 and the corresponding data. For example, indexes 0 to 13 correspond to server, disk, usage, alarm, hard disk, memory, server, host, storage pool, storage, thick waist, main, threshold and exceed respectively.

[0132] In such a case, for a server status log, such as "host storage pool usage exceeds the secondary alarm threshold", the compression rate of the existing compression algorithm in the related technology is far less than 50%. Using the compression algorithm of the example of this application, "host", "storage pool", "usage rate", "exceed", "secondary", "alarm", and "threshold" are all high-frequency words and appear in the index dictionary table. They can be replaced by index values ​​for encoding, and the compression rate can exceed 80%, and the length of the text content is greatly shortened.

[0133] Through the embodiment of the present application, in view of the problem of low compression efficiency of HTTP protocol message body, the embodiment of the present application proposes a solution for efficient compression based on the analysis and extraction of index dictionaries based on massive message logs combined with a certain compression algorithm. The index dictionary proposed in the embodiment of the present application can fully discover the similarities between different HTTP connection messages within a period of time, and can fully compress similar message bodies. Compared with the existing compression algorithm, both the compression ratio and the compression efficiency are greatly improved, thereby effectively reducing the length of the HTTP message body and improving network transmission efficiency.

[0134] Step S707: Execute business processing, send response data, and record message logs.

[0135] Step S708: The server determines whether compression is required. If compression is determined to be required, step S709 is executed. If compression is determined not to be required, step S710 is executed.

[0136] Step S709: Use the dynamic index dictionary to compress the message body content and set the message header compression flag.

[0137] Step S710: Send the content through the network.

[0138] Step S711: The client compression device determines whether decompression is required. If it is determined that decompression is required, step S712 is executed; if it is determined that decompression is not required, step S713 is executed.

[0139] Step S712: Use the dynamic index dictionary to decompress the message body content to obtain the original message.

[0140] Step S713: The client uses the response data to perform business processing, and the communication ends.

[0141] Through the embodiments of the present application, in the related art, data compression in the transmission protocol (for example, HTTP communication protocol) belongs to stateless compression, and the compression algorithm is for the content of single communication data, and cannot utilize the similarity of system dimension data. When the character string of the content of single communication data is very scattered and the self-repetitiveness is very low, the compression efficiency is very poor. The present invention proposes a solution to construct a dynamic index dictionary based on HTTP message statistical analysis, which can fully discover the similarities between multiple communication data, extract hotspot character string units to construct a system index dictionary, and the communication data of the client and the server can use the dictionary to compress and decompress according to a certain compression algorithm, thereby achieving a higher compression ratio and compression efficiency, improving network performance, reducing bandwidth consumption, and improving response speed.

[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0143] In the present embodiment, a transmission device for a protocol message is also provided, and the device is used to implement the above-mentioned embodiment and preferred implementation mode, and the descriptions that have been made are not repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0144] Fig. 9 is a structural block diagram of an optional protocol message transmission device according to an embodiment of the present application, such as Fig. 9 As shown, the device includes a compressed data set corresponding to the target transmission protocol stored in the server, the compressed data set records the index and reference data with a corresponding relationship, the reference data is the data with an appearance frequency greater than or equal to the frequency threshold screened from the protocol log of the target transmission protocol recorded by the server, the device is applied to the server, and the device includes:

[0145] An acquisition module 902 is used to acquire an initial message to be transmitted to a terminal device via the target transmission protocol;

[0146] A first search module 904, configured to search for data in the initial message from the compressed data set;

[0147] A first replacement module 906 is used to replace the target data in the initial message with the corresponding target index in the compressed data set to obtain a compressed message when the target data in the initial message is found in the compressed data set;

[0148] The transmission module 908 is used to transmit the compressed message to the terminal device.

[0149] Through the above-mentioned device, the server creates corresponding compressed data sets for different transmission protocols. The compressed data sets record the data that appear frequently in the protocol log of the transmission protocol. When the server transmits a protocol message to the terminal device through the transmission protocol, the server replaces the data in the protocol message with the data in the compressed data set. In this way, the data that appears frequently in the protocol messages transmitted through the transmission protocol between different servers and terminal devices is replaced with the corresponding index, thereby reducing the data volume of the protocol messages transmitted between the server and the terminal device. Therefore, the problem of low transmission efficiency of the protocol messages can be solved, thereby achieving the effect of improving the transmission efficiency of the protocol messages.

[0150] In an exemplary embodiment, the apparatus further comprises:

[0151] A first collection and analysis module, for a log collector deployed in the server, before acquiring the initial message to be transmitted to the terminal device through the target transmission protocol, collecting the protocol log of the target transmission protocol recorded by the server through the log collector to obtain an initial log set, and parsing each protocol log in the initial log set through the log collector to obtain an initial log data set;

[0152] A first screening module, configured to screen data having an appearance frequency greater than or equal to the frequency threshold from the initial log data set to obtain the reference data;

[0153] The first generating record module is used to generate the index corresponding to the reference data in the order of the occurrence frequency from high to low, and record the index and the reference data having a corresponding relationship in the order of the occurrence frequency from high to low.

[0154] In an exemplary embodiment, the first screening module includes:

[0155] an extracting unit, configured to extract candidate data carried in log data corresponding to each protocol log in the initial log data set, wherein log data corresponding to at least two protocol logs in the initial log data set all include the candidate data;

[0156] a first determining unit, configured to determine a first occurrence frequency of the candidate data according to the candidate data and the initial log data set, and to determine a second occurrence frequency of the candidate data according to the candidate data and the initial log data set;

[0157] a product unit, configured to perform a product operation on the first occurrence frequency and the second occurrence frequency to obtain a target occurrence frequency of the candidate data;

[0158] A screening unit is used to screen data whose target occurrence frequency is greater than or equal to the frequency threshold from the candidate data to obtain the reference data.

[0159] In an exemplary embodiment, the first determining unit is configured to:

[0160] In the case where the initial log data set includes N data and the candidate data includes M data, the i1th occurrence frequency of the i-th data in the M data is determined by performing the following steps, wherein N is a positive integer, M is a positive integer, and i is a positive integer less than or equal to M: in the case where the i-th data appears K times in the N data, the i1th occurrence frequency is determined as a value obtained by dividing K by S, wherein S is the number of times the data with the largest number of occurrences in the N data appears in the N data, and S is a positive integer;

[0161] The determining, according to the candidate data and the initial log set, the second occurrence frequency of the candidate data includes: when the candidate data includes M data, the initial log set includes Q protocol logs, and P protocol logs among the Q protocol logs include the i-th data among the M data, determining the i2-th occurrence frequency of the i-th data by performing the following steps, wherein M is a positive integer, Q is a positive integer, P is a positive integer less than or equal to Q, and i is a positive integer less than or equal to M: performing a sum operation on P and a reference value to obtain a target sum value; dividing Q by the target sum value to obtain a target quotient value; and performing a logarithm operation on the target quotient value to obtain the i2-th occurrence frequency.

[0162] In an exemplary embodiment, the apparatus further comprises:

[0163] A setting module, configured to set the validity period of the compressed data set to a target duration after generating the indexes corresponding to the reference data in the order of the occurrence frequency from high to low, and recording the indexes and the reference data having a corresponding relationship in the order of the occurrence frequency from high to low, wherein the target duration is equal to the duration difference between the target time and the generation time of the compressed data set, the generation time is earlier than the target time, and the target time is the latest time that the server and the terminal device are allowed to use the compressed data set to transmit a protocol message through the target transmission protocol;

[0164] The recording module is used to record the generation time, validity period and version identification of the compressed data set.

[0165] In an exemplary embodiment, the apparatus further comprises:

[0166] A second collection and analysis module is used for, after recording the generation time, validity period and version identifier of the compressed data set, collecting the update protocol log of the target transmission protocol recorded by the server through the log collector to obtain an update log set when it is detected that the compressed data set is not within the validity period, and parsing each protocol log in the update log set through the log collector to obtain an update log data set;

[0167] A second screening module, screening data whose occurrence frequency is greater than or equal to the frequency threshold from the update log data set, to obtain update data;

[0168] The second generation record module is used to generate an update index corresponding to the update data, and record the update index and the update data with a corresponding relationship in order of the occurrence frequency from high to low, wherein the update data set includes the update index and the update data with a corresponding relationship.

[0169] In an exemplary embodiment, the acquisition module includes:

[0170] A receiving unit, configured to receive a compression request message initiated by the terminal device;

[0171] A first search unit, configured to extract an initial version identifier carried in the compression request message through the first decoder, and search for a reference index in the compression request message in the compressed data set corresponding to the initial version identifier through the first decoder;

[0172] a replacement unit, configured to, when the reference index in the compression request message is found in the compressed data set by the first decoder, replace the reference index with the corresponding data to be compressed in the compressed data set by the first decoder to obtain an initial request message;

[0173] A responding unit is used to respond to the initial request message and obtain the initial message.

[0174] In an exemplary embodiment, the second searching unit is used to:

[0175] In the case where the initial message includes R data and the compressed data set includes T indexes and T data having a corresponding relationship, the sth data among the R data is searched from the compressed data set by performing the following steps, wherein R is a positive integer, T is a positive integer, and s is a positive integer less than or equal to R:

[0176] Extracting an sth data set from the R data, wherein the sth data set includes the sth data and L data adjacent to the sth data, wherein L is a positive integer;

[0177] Find the sth data set from the T data;

[0178] When all the data in the sth data set are found from the T data, the sth data set is marked as index data, and the search for the s+L+1th data in the R data continues from the T data; when all the data in the sth data set are not found from the T data, the sth data set is marked as non-index data, and the search for the s+L+1th data in the R data continues from the T data.

[0179] In an exemplary embodiment, the transmission module includes:

[0180] a first execution unit, configured to perform a corresponding first encoding operation on the target index in the compressed message through the first encoder to obtain a first compressed encoded message, and perform a corresponding second encoding operation on other data in the compressed message through the first encoder to obtain a second compressed encoded message, wherein the other data includes data other than the target data in the compressed message, the first compressed encoded message includes data obtained by encoding the target index, or data obtained by encoding the number of bytes occupied by the target index, the second compressed encoded message includes data obtained by encoding the data volume of the other data and data obtained by encoding the other data, or the second compressed encoded message includes data obtained by encoding the number of bytes occupied by the data volume of the other data and data obtained by encoding the other data;

[0181] A transmission unit carries the initial version identifier of the compressed data set in a target compressed message, and transmits the target compressed message to the terminal device, wherein the target compressed message includes the first compressed encoding message and the second compressed encoding message.

[0182] In an exemplary embodiment, the apparatus further comprises:

[0183] The first receiving module is used to receive a target data request initiated by the terminal device before obtaining the initial message to be transmitted to the terminal device through the target transmission protocol, wherein the target data request is used to request to obtain the compressed data set; in response to the target data request, extract the compressed data set, and transmit the compressed data set, the generation time of the compressed data set, the validity period of the compressed data set and the initial version identifier of the compressed data set to the terminal device.

[0184] Alternatively, a second receiving module is used to receive an update data request initiated by the terminal device, wherein the update data request is used to request an update data set; in response to the update data request, the update data set is extracted, and the update data set, the generation time of the update data set, the validity period of the update data set and the update version identifier of the update data set are transmitted to the terminal device.

[0185] Fig.10 is a structural block diagram of another optional protocol message transmission device according to an embodiment of the present application, such as Fig.10As shown, the device includes a compressed data set corresponding to a target transmission protocol stored in a terminal device, the compressed data set records an index and reference data with a corresponding relationship, the reference data is data with an appearance frequency greater than or equal to a frequency threshold screened from a protocol log of the target transmission protocol recorded by a server, the device is applied to the terminal device, and the device includes:

[0186] The third receiving module 1002 is used to receive the compressed message transmitted by the server and extract the initial version identifier carried in the compressed message;

[0187] A second search module 1004 is used to search for data in the compressed message from the compressed data set corresponding to the initial version identifier;

[0188] The second replacement module 1006 is used to replace the target compressed data in the compressed message with the corresponding target data in the compressed data set to obtain the target message when the target compressed data in the compressed message is found in the compressed data set.

[0189] Through the above-mentioned device, when the terminal device receives the compressed message sent by the server, it can, but is not limited to, use the compressed data set corresponding to the initial version identifier to decompress the compressed message to obtain the target message before compression. In this way, the protocol message transmitted between the terminal device and the server is compressed, which reduces the data volume of the transmitted protocol message. Therefore, the problem of low transmission efficiency of the protocol message can be solved, thereby achieving the effect of improving the transmission efficiency of the protocol message.

[0190] In an exemplary embodiment, the apparatus further comprises:

[0191] A first detection module, used for detecting whether the terminal device stores the compressed data set before receiving the compressed message transmitted by the server;

[0192] A first initiating module is configured to initiate a target data request to the server when it is detected that the compressed data set is not stored in the terminal device, wherein the target data request is used to request to obtain the compressed data set, and receive the compressed data set, the generation time of the compressed data set, the validity period of the compressed data set, and the initial version identifier of the compressed data set returned by the server in response to the target data request;

[0193] The second detection module is used to detect whether the compressed data set is within the validity period when it is detected that the compressed data set is stored in the terminal device, and to initiate an update data request to the server when it is detected that the compressed data set is not within the validity period, wherein the update data request is used to request to obtain an updated data set, and receive the updated compressed data set returned by the server in response to the update data request, the generation time of the updated data set, the validity period of the updated data set and the updated version identifier of the updated data set.

[0194] In an exemplary embodiment, the second detection module includes:

[0195] a detection unit, configured to detect whether a target duration difference between a current time and a target time is less than or equal to a difference threshold, wherein the current time is a time for detecting whether the compressed data set is within a validity period, and the current time is earlier than the target time;

[0196] The second determining unit is configured to determine that the compressed data set is not within a validity period when it is detected that the target duration difference is less than or equal to the difference threshold.

[0197] In an exemplary embodiment, the second search module includes:

[0198] A second execution unit is used to perform a first decoding operation corresponding to the first encoding operation on the first compressed coded message through the second decoder to obtain a target index, and perform a second decoding operation corresponding to the second encoding operation on the second compressed coded message through the second decoder to obtain other data in the initial message except the target data;

[0199] The third searching unit is used to search the target data corresponding to the target index from the compressed data set corresponding to the initial version identifier, wherein the target message includes the target data and the other data.

[0200] In an exemplary embodiment, the third searching unit is used to:

[0201] In the case where the target index includes W indexes and the compressed data set includes T indexes and T data having a corresponding relationship, the t-th data corresponding to the t-th index in the W indexes is searched from the T indexes and T data having a corresponding relationship by performing the following steps, wherein the target data includes K data, the K data include the t-th data, W is a positive integer, T is a positive integer, and t is a positive integer equal to T:

[0202] Find the t-th index from the T indexes;

[0203] When the t-th index is found from the T indexes, the data corresponding to the t-th index among the T indexes and T data having a corresponding relationship is determined as the t-th data.

[0204] In an exemplary embodiment, the apparatus further comprises:

[0205] A generating module, configured to: a second transmission device is deployed in the terminal device, a second encoder is deployed in the second transmission device, the second encoder is connected to the server, and before receiving the compressed message transmitted by the server, an initial request message is generated, wherein the initial request message is used to request to obtain an initial message;

[0206] A third search module, configured to search the data in the initial request message in the compressed data set through the second encoder;

[0207] a third replacement module, configured to, when the data to be compressed in the initial request message is found from the compressed data set by the second encoder, replace the data to be compressed in the initial request message with a corresponding reference index in the compressed data set by the second encoder to obtain a compression request message;

[0208] The second initiating module is used to initiate the compression request message to the server, wherein the compression request message carries the initial version identifier of the compressed data set, and the server is configured to obtain the initial message in response to the received initial request message, and replace the target data in the initial message with the compressed message obtained by the corresponding target index in the compressed data set and transmit it to the terminal device.

[0209] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0210] According to another embodiment of the present application, a server system is further provided, the server system comprising a connected server and a terminal device, wherein the server and the terminal device are configured to execute the steps of any of the above method embodiments during operation.

[0211] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0212] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0213] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0214] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0215] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0216] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0217] An embodiment of the present application also provides a computer program, which includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any one of the above method embodiments.

[0218] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0219] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0220] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for transmitting a protocol message, characterized in that: A compressed data set corresponding to a target transmission protocol is stored in the server, wherein the compressed data set records an index and reference data having a corresponding relationship, wherein the reference data is data with an appearance frequency greater than or equal to a frequency threshold that is screened out from a protocol log of the target transmission protocol recorded by the server, and the method is applied to the server, and the method includes: Acquire an initial message to be transmitted to a terminal device via the target transmission protocol; Searching for data in the initial message from the compressed data set; When the target data in the initial message is found in the compressed data set, the target data in the initial message is replaced with the corresponding target index in the compressed data set to obtain a compressed message; Transmitting the compressed message to a terminal device; The server is provided with a first transmission device, the first transmission device is provided with a first decoder, the first decoder is connected to the terminal device, and the obtaining of the initial message to be transmitted to the terminal device through the target transmission protocol includes: Receiving a compression request message initiated by the terminal device; Extracting the initial version identifier carried in the compression request message through the first decoder, and searching the reference index in the compression request message in the compressed data set corresponding to the initial version identifier through the first decoder; In the case where the reference index in the compression request message is found in the compressed data set by the first decoder, the reference index is replaced by the corresponding data to be compressed in the compressed data set by the first decoder to obtain an initial request message; Respond to the initial request message and obtain the initial message.

2. The method according to claim 1, characterized in that The server is deployed with a log collector. Before obtaining the initial message to be transmitted to the terminal device through the target transmission protocol, the method further includes: Collecting the protocol log of the target transmission protocol recorded by the server by the log collector to obtain an initial log set, and parsing each protocol log in the initial log set by the log collector to obtain an initial log data set; Filtering data whose occurrence frequency is greater than or equal to the frequency threshold from the initial log data set to obtain the reference data; The indexes corresponding to the reference data are generated in the order of the occurrence frequencies from high to low, and the indexes and the reference data having a corresponding relationship are recorded in the order of the occurrence frequencies from high to low.

3. The method according to claim 2, characterized in that The step of filtering data having an appearance frequency greater than or equal to the frequency threshold from the initial log data set to obtain the reference data includes: Extracting candidate data carried in log data corresponding to each protocol log in the initial log data set, wherein log data corresponding to at least two protocol logs in the initial log data set all include the candidate data; Determine a first occurrence frequency of the candidate data according to the candidate data and the initial log data set, and determine a second occurrence frequency of the candidate data according to the candidate data and the initial log data set; Performing a product operation on the first occurrence frequency and the second occurrence frequency to obtain a target occurrence frequency of the candidate data; The data whose target occurrence frequency is greater than or equal to the frequency threshold are screened from the candidate data to obtain the reference data.

4. The method according to claim 3, characterized in that The determining, according to the candidate data and the initial log data set, the first occurrence frequency of the candidate data comprises: when the initial log data set comprises N data and the candidate data comprises M data, determining the i1th occurrence frequency of the i-th data in the M data by performing the following steps, wherein N is a positive integer, M is a positive integer, and i is a positive integer less than or equal to M: when the i-th data appears K times in the N data, determining the i1th occurrence frequency as a value obtained by dividing K by S, wherein S is the number of times the data with the largest number of occurrences in the N data appears in the N data, and S is a positive integer; The determining, according to the candidate data and the initial log set, the second occurrence frequency of the candidate data includes: when the candidate data includes M data, the initial log set includes Q protocol logs, and P protocol logs among the Q protocol logs include the i-th data among the M data, determining the i2-th occurrence frequency of the i-th data by performing the following steps, wherein M is a positive integer, Q is a positive integer, P is a positive integer less than or equal to Q, and i is a positive integer less than or equal to M: performing a sum operation on P and a reference value to obtain a target sum value; dividing Q by the target sum value to obtain a target quotient value; and performing a logarithm operation on the target quotient value to obtain the i2-th occurrence frequency.

5. The method according to claim 2, characterized in that: After generating the indexes corresponding to the reference data in the order of the occurrence frequencies from high to low, and recording the indexes and the reference data having a corresponding relationship in the order of the occurrence frequencies from high to low, the method further includes: Setting the validity period of the compressed data set to a target duration, wherein the target duration is equal to the duration difference between the target time and the generation time of the compressed data set, the generation time is earlier than the target time, and the target time is the latest time that the server and the terminal device are allowed to use the compressed data set to transmit a protocol message through the target transmission protocol; The generation time, validity period and version identification of the compressed data set are recorded.

6. The method according to claim 5, characterized in that After recording the generation time, validity period and version identification of the compressed data set, the method further includes: In the case where it is detected that the compressed data set is not within the validity period, collecting the update protocol log of the target transmission protocol recorded by the server through the log collector to obtain an update log set, and parsing each protocol log in the update log set through the log collector to obtain an update log data set; Filtering data whose occurrence frequency is greater than or equal to the frequency threshold from the update log data set to obtain update data; Generate an update index corresponding to the update data, and record the update index and the update data having a corresponding relationship in order of the occurrence frequency from high to low, wherein the update data set includes the update index and the update data having a corresponding relationship.

7. The method according to claim 1, characterized in that The searching the data in the initial message from the compressed data set includes: In the case where the initial message includes R data and the compressed data set includes T indexes and T data having a corresponding relationship, the sth data among the R data is searched from the compressed data set by performing the following steps, wherein R is a positive integer, T is a positive integer, and s is a positive integer less than or equal to R: Extracting an sth data set from the R data, wherein the sth data set includes the sth data and L data adjacent to the sth data, wherein L is a positive integer; Find the sth data set from the T data; When all the data in the sth data set are found from the T data, the sth data set is marked as index data, and the search for the s+L+1th data in the R data continues from the T data; when all the data in the sth data set are not found from the T data, the sth data set is marked as non-index data, and the search for the s+L+1th data in the R data continues from the T data.

8. The method according to claim 1, characterized in that: The server is provided with a first transmission device, the first transmission device is provided with a first encoder, the first encoder is connected to the terminal device, and the transmitting the compressed message to the terminal device includes: The first encoder performs a corresponding first encoding operation on the target index in the compressed message to obtain a first compressed encoded message, and the first encoder performs a corresponding second encoding operation on other data in the compressed message to obtain a second compressed encoded message, wherein the other data includes data in the compressed message other than the target data, the first compressed encoded message includes data obtained by encoding the target index, or data obtained by encoding the number of bytes occupied by the target index, the second compressed encoded message includes data obtained by encoding the data volume of the other data and data obtained by encoding the other data, or the second compressed encoded message includes data obtained by encoding the number of bytes occupied by the data volume of the other data and data obtained by encoding the other data; The initial version identifier of the compressed data set is carried in a target compressed message, and the target compressed message is transmitted to the terminal device, wherein the target compressed message includes the first compressed encoding message and the second compressed encoding message.

9. The method according to claim 1, characterized in that: Before acquiring the initial message to be transmitted to the terminal device through the target transmission protocol, the method further includes: receiving a target data request initiated by the terminal device, wherein the target data request is used to request to obtain the compressed data set; responding to the target data request, extracting the compressed data set, and transmitting the compressed data set, the generation time of the compressed data set, the validity period of the compressed data set, and the initial version identifier of the compressed data set to the terminal device; or, Receive an update data request initiated by the terminal device, wherein the update data request is used to request an update data set; respond to the update data request, extract the update data set, and transmit the update data set, the generation time of the update data set, the validity period of the update data set, and the update version identifier of the update data set to the terminal device.

10. A method for transmitting a protocol message, characterized in that: A compressed data set corresponding to a target transmission protocol is stored in a terminal device, wherein the compressed data set records an index and reference data having a corresponding relationship, wherein the reference data is data with an appearance frequency greater than or equal to a frequency threshold screened from a protocol log of the target transmission protocol recorded by a server, and the method is applied to the terminal device, and the method includes: Receiving a compressed message transmitted by the server, and extracting an initial version identifier carried in the compressed message; Searching for data in the compressed message from the compressed data set corresponding to the initial version identifier; When the target compressed data in the compressed message is found in the compressed data set, the target compressed data in the compressed message is replaced with the corresponding target data in the compressed data set to obtain the target message; The terminal device is provided with a second transmission device, the second transmission device is provided with a second decoder, the second decoder is connected to the server, and the step of searching for data in the compressed message from the compressed data set corresponding to the initial version identifier includes: Performing a first decoding operation corresponding to the first encoding operation on the first compressed coded message by the second decoder to obtain a target index, and performing a second decoding operation corresponding to the second encoding operation on the second compressed coded message by the second decoder to obtain other data in the initial message except the target data; The target data corresponding to the target index is searched from the compressed data set corresponding to the initial version identifier, wherein the target message includes the target data and the other data.

11. The method according to claim 10, characterized in that Before receiving the compressed message transmitted by the server, the method further includes: Detecting whether the terminal device stores the compressed data set; In the case where it is detected that the compressed data set is not stored in the terminal device, a target data request is initiated to the server, wherein the target data request is used to request to obtain the compressed data set, and the compressed data set, the generation time of the compressed data set, the validity period of the compressed data set, and the initial version identifier of the compressed data set are returned by the server in response to the target data request; When it is detected that the compressed data set is stored in the terminal device, whether the compressed data set is within the validity period is detected. When it is detected that the compressed data set is not within the validity period, an update data request is initiated to the server, wherein the update data request is used to request to obtain an updated data set, and receive the updated compressed data set returned by the server in response to the update data request, the generation time of the updated data set, the validity period of the updated data set and the updated version identifier of the updated data set.

12. The method according to claim 11, characterized in that The detecting whether the compressed data set is within the validity period includes: Detecting whether a target duration difference between a current time and a target time is less than or equal to a difference threshold, wherein the current time is a time for detecting whether the compressed data set is within a validity period, and the current time is earlier than the target time; When it is detected that the target duration difference is less than or equal to the difference threshold, it is determined that the compressed data set is not within the validity period.

13. The method according to claim 10, characterized in that The step of searching the compressed data set corresponding to the initial version identifier for the target data corresponding to the target index includes: In the case where the target index includes W indexes and the compressed data set includes T indexes and T data having a corresponding relationship, the t-th data corresponding to the t-th index in the W indexes is searched from the T indexes and T data having a corresponding relationship by performing the following steps, wherein the target data includes K data, the K data include the t-th data, W is a positive integer, T is a positive integer, and t is a positive integer equal to T: Find the t-th index from the T indexes; When the t-th index is found from the T indexes, the data corresponding to the t-th index among the T indexes and T data having a corresponding relationship is determined as the t-th data.

14. The method according to claim 10, characterized in that The terminal device is provided with a second transmission device, the second transmission device is provided with a second encoder, the second encoder is connected to the server, and before receiving the compressed message transmitted by the server, the method further includes: Generate an initial request message, wherein the initial request message is used to request to obtain an initial message; Searching the compressed data set for data in the initial request message by the second encoder; In the case where the data to be compressed in the initial request message is found from the compressed data set by the second encoder, the data to be compressed in the initial request message is replaced by a corresponding reference index in the compressed data set by the second encoder to obtain a compression request message; The compression request message is initiated to the server, wherein the compression request message carries the initial version identifier of the compressed data set, and the server is configured to obtain the initial message in response to the received initial request message, and replace the target data in the initial message with the compressed message obtained by the corresponding target index in the compressed data set and transmit it to the terminal device.

15. A transmission device for a protocol message, characterized in that: A compressed data set corresponding to a target transmission protocol is stored in the server, and an index and reference data having a corresponding relationship are recorded in the compressed data set, and the reference data is data with an appearance frequency greater than or equal to a frequency threshold that is screened from a protocol log of the target transmission protocol recorded by the server, and the device is applied to the server, and includes: An acquisition module, used for acquiring an initial message to be transmitted to a terminal device via the target transmission protocol; A first search module, used for searching the data in the initial message from the compressed data set; A first replacement module, configured to replace the target data in the initial message with a corresponding target index in the compressed data set to obtain a compressed message when the target data in the initial message is found in the compressed data set; A transmission module, used to transmit the compressed message to a terminal device, wherein the terminal device is used to decompress the received compressed message using the compressed data set to obtain a target message corresponding to the initial message; The server is provided with a first transmission device, the first transmission device is provided with a first decoder, the first decoder is connected to the terminal device, and the acquisition module includes: A receiving unit, configured to receive a compression request message initiated by the terminal device; A first search unit, configured to extract an initial version identifier carried in the compression request message through the first decoder, and search for a reference index in the compression request message in the compressed data set corresponding to the initial version identifier through the first decoder; a replacement unit, configured to, when the reference index in the compression request message is found in the compressed data set by the first decoder, replace the reference index with the corresponding data to be compressed in the compressed data set by the first decoder to obtain an initial request message; A responding unit is used to respond to the initial request message and obtain the initial message.

16. A transmission device for a protocol message, characterized in that: A compressed data set corresponding to a target transmission protocol is stored in a terminal device, wherein the compressed data set records an index and reference data having a corresponding relationship, wherein the reference data is data with an appearance frequency greater than or equal to a frequency threshold screened from a protocol log of the target transmission protocol recorded by a server, and the apparatus is applied to the terminal device, and comprises: A third receiving module is used to receive the compressed message transmitted by the server and extract the initial version identifier carried in the compressed message; A second search module, used to search for data in the compressed message from the compressed data set corresponding to the initial version identifier; A second replacement module is used to replace the target compressed data in the compressed message with the corresponding target data in the compressed data set to obtain the target message when the target compressed data in the compressed message is found in the compressed data set; The terminal device is provided with a second transmission device, the second transmission device is provided with a second decoder, the second decoder is connected to the server, and the second search module includes: A second execution unit is used to perform a first decoding operation corresponding to the first encoding operation on the first compressed coded message through the second decoder to obtain a target index, and perform a second decoding operation corresponding to the second encoding operation on the second compressed coded message through the second decoder to obtain other data in the initial message except the target data; The third searching unit is used to search the target data corresponding to the target index from the compressed data set corresponding to the initial version identifier, wherein the target message includes the target data and the other data.

17. A server system, characterized in that: The server system comprises a connected server and a terminal device, wherein the server is configured to implement the steps of the method described in any one of claims 1 to 9, and the terminal device is configured to implement the steps of the method described in any one of claims 10-14.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 9 are implemented, or the steps of the method described in any one of claims 10 to 14 are implemented.

19. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the processor implements the steps of the method described in any one of claims 1 to 9, or implements the steps of the method described in any one of claims 10 to 14.

20. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 9, or implements the steps of the method described in any one of claims 10 to 14.

Images