Communication protocol data serialization method and device, computer equipment and storage medium

Abstract

The invention relates to a communication protocol data serialization method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring service protocol field data corresponding to a service communication request; determining the data length of the business protocol field data; obtaining a communication protocol packet header based on the data length of the service protocol data, and determining the length of a communication protocol packet; distributing the memory with the length of the communication protocol packet to a communication thread of the service communication request; and copying the communication protocol header and the business protocol field data to a memory in a serialization form to obtain serialized communication protocol data. In the embodiment of the invention, the serialization processing of the communication protocol data can be directly carried out on one memory, that is, the serialization processing of the communication protocol data can be completed only through one-time copying, the serialization efficiency of the communication protocol data can be effectively improved, and thus the communication performance is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a communication protocol data serialization method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of computer technology and network technology, communication protocol technology has emerged. Communication protocol refers to the rules and agreements that two entities must follow to complete communication or service. In order for multiple data communication systems in different geographical locations to be able to work together to achieve information exchange and resource sharing, they must have a common language.

[0003] However, in the current communication process, for example, in the communication process of the content distribution network, it is necessary to first serialize the business protocol at the offset position of the memory block, and serialize the communication protocol header in another memory block, and copy the serialized communication protocol header to the previous memory block, and then copy the complete resource package to the memory block sent and received by the network. This process requires copying resources at least twice, which affects the communication performance of the server. Summary of the invention

[0004] Based on this, it is necessary to provide a communication protocol data serialization method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve communication performance in response to the above technical problems.

[0005] In a first aspect, the present application provides a communication protocol data serialization method. The method comprises:

[0006] Acquire the service protocol field data corresponding to the service communication request, wherein the service communication request is used to send the service protocol field data or request the service protocol field data;

[0007] Determine the data length of the service protocol field data;

[0008] Obtaining a communication protocol packet header based on the data length of the service protocol data, and determining the communication protocol packet length;

[0009] Allocating memory of the communication protocol packet length to the communication thread of the service communication request;

[0010] The communication protocol header and the service protocol field data are copied to the memory in a serialized form to obtain serialized communication protocol data.

[0011] In a second aspect, the present application also provides a communication protocol data serialization device. The device comprises:

[0012] A request receiving module, used to obtain the business protocol field data corresponding to the business communication request, wherein the business communication request is used to send the business protocol field data or request the business protocol field data;

[0013] A protocol length determination module, used to determine the data length of the service protocol field data;

[0014] A communication protocol determination module, used to obtain a communication protocol packet header based on the data length of the service protocol data, and determine the communication protocol packet length;

[0015] A memory allocation module, used to allocate memory of the communication protocol packet length to the communication thread of the business communication request;

[0016] The protocol serialization module is used to copy the communication protocol header and the business protocol field data to the memory in a serialized form to obtain serialized communication protocol data.

[0017] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0018] Acquire the service protocol field data corresponding to the service communication request, wherein the service communication request is used to send the service protocol field data or request the service protocol field data;

[0019] Determine the data length of the service protocol field data;

[0020] Obtaining a communication protocol packet header based on the data length of the service protocol data, and determining the communication protocol packet length;

[0021] Allocating memory of the communication protocol packet length to the communication thread of the service communication request;

[0022] The communication protocol header and the service protocol field data are copied to the memory in a serialized form to obtain serialized communication protocol data.

[0023] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0024] Acquire the service protocol field data corresponding to the service communication request, wherein the service communication request is used to send the service protocol field data or request the service protocol field data;

[0025] Determine the data length of the service protocol field data;

[0026] Obtaining a communication protocol packet header based on the data length of the service protocol data, and determining the communication protocol packet length;

[0027] Allocating memory of the communication protocol packet length to the communication thread of the service communication request;

[0028] The communication protocol header and the service protocol field data are copied to the memory in a serialized form to obtain serialized communication protocol data.

[0029] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0030] Acquire the service protocol field data corresponding to the service communication request, wherein the service communication request is used to send the service protocol field data or request the service protocol field data;

[0031] Determine the data length of the service protocol field data;

[0032] Obtaining a communication protocol packet header based on the data length of the service protocol data, and determining the communication protocol packet length;

[0033] Allocating memory of the communication protocol packet length to the communication thread of the service communication request;

[0034] The communication protocol header and the service protocol field data are copied to the memory in a serialized form to obtain serialized communication protocol data.

[0035] The above-mentioned communication protocol data serialization method, device, computer equipment, storage medium and computer program product extract the business protocol field data of the business communication request after receiving the business communication request, obtain the basic data of the communication process, and then determine the data length of the business protocol field data; determine the length of the business protocol part that needs to be serialized, and obtain the communication protocol packet header based on the data length of the business protocol data, and determine the communication protocol packet length. After obtaining the data length of the business protocol field data, the communication protocol header can be obtained, and the length of the entire communication protocol packet can be determined. Finally, the memory of the communication protocol packet length can be directly allocated to the communication thread of the business communication request, and the communication protocol header and the business protocol field data are copied to the memory, and the serialization processing of the communication protocol data of the communication process is completed to obtain the serialized communication protocol data. In this embodiment, by pre-processing the length of the business protocol field data after serialization, and the communication protocol packet length that depends on the length of the business protocol field data, the serialization processing of the communication protocol data can be directly performed on a piece of memory, that is, only one copy is required to complete the serialization processing of the communication protocol data, which can effectively improve the efficiency of the communication protocol data serialization, thereby improving the communication performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 An application environment diagram of a communication protocol data serialization method in one embodiment;

[0037] Figure 2 A schematic diagram of a flow chart of a communication protocol data serialization method in one embodiment;

[0038] Figure 3 A schematic diagram of the composition of the business protocol field data in one embodiment;

[0039] Figure 4 A schematic diagram of the structure and length of a service protocol field in one embodiment;

[0040] Figure 5 A schematic diagram of an interval table lookup in one embodiment;

[0041] Figure 6 A schematic diagram of calculating the average value of field values ​​in one embodiment;

[0042] Figure 7 A schematic diagram of an interval table lookup upward search process in an embodiment;

[0043] Figure 8 A schematic diagram of a downward search process of an interval table lookup in an embodiment;

[0044] Fig. 9 It is a schematic diagram of the overall process of a communication protocol data serialization method in one embodiment;

[0045] Fig.10 It is a flowchart of a communication protocol data serialization method in another embodiment;

[0046] Fig.11 It is a structural block diagram of a communication protocol data serialization device in one embodiment;

[0047] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] The communication protocol data serialization method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the access server 104 through the network, and the access server 104 communicates with the external server 106 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the access server 104, or it can be placed on the cloud or other servers. When the terminal 102 wants to communicate with the external server 106, send data to the external server 106 or obtain data from the external server 106, it can submit a business communication request to the access server 104 to request the access server 104 to implement communication processing, such as sending data to the external server 106 or obtaining data from the external server 106. First, the access server 104 receives the business communication request and obtains the business protocol field data corresponding to the business communication request; determines the data length of the business protocol field data; obtains the communication protocol packet header based on the data length of the business protocol data, and determines the communication protocol packet length; allocates the memory of the communication protocol packet length to the communication thread of the business communication request; copies the communication protocol header and the business protocol field data to the memory in a serialized form to obtain serialized communication protocol data. The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0050] In one embodiment, Figure 2 As shown, a communication protocol data serialization method is provided, and the method is applied to Figure 1 The access server 104 in the example is used as an example to illustrate, including the following steps:

[0051] Step 201: Acquire business protocol field data corresponding to a business communication request, where the business communication request is used to send business protocol field data or request business protocol field data.

[0052] Among them, the service communication request refers to a request submitted by the terminal 102 to the access server 104 to implement communication-related functions through the access server. For example, the access server 104 can be requested to send data through the service communication request, or the access server 104 can be requested to obtain data through the service communication request. The service protocol field data refers to the service data that needs to be processed during the communication process, that is, the specific content that needs to be sent or obtained. The service protocol data can be specifically composed of multiple field data, each of which includes a field name and a field value.

[0053] Specifically, when the terminal 102 wants to communicate with the external server 106, it can submit a business communication request to the access server 104 to request the access server 104 to implement communication processing, such as sending data to the external server 106 or obtaining data from the external server 106. When the access server 104 receives the business communication request, it will first obtain the business protocol field data of the business communication request, which may be the business protocol data that needs to be sent or returned to the source, and then communicate the communication protocol data based on these business protocol data, and perform serialization processing, and finally transmit the serialized communication protocol data. In one embodiment, the present application can be specifically applied to the content delivery network (CDN) of the instant messaging software, and the access server 104 is the access server of the content delivery network. The user can request the content distribution network of the instant messaging software to download pictures and videos through the client of the instant messaging software.

[0054] Step 203: determine the data length of the service protocol field data.

[0055] The data length of the service protocol field data refers to the number of bytes occupied by the service protocol data in the memory.

[0056] Specifically, the solution of the present application determines the length of the communication protocol packet by pre-processing the length of the serialized business protocol packet, thereby accurately allocating memory and reducing the memory loss in the serialization process of the communication protocol data. Therefore, after obtaining the business protocol field data of the business communication request, it is first necessary to determine the data length of the business protocol field data. For the business protocol field data, it can be divided into different fields, and the length of each field can be calculated separately, and finally all the lengths are added to obtain the final data length of the business protocol field data. For each business protocol field, the length of the field name and the field value can be calculated separately, and then the length of the two can be combined to obtain the length of the current business protocol field.

[0057] Step 205, obtaining a communication protocol packet header based on the data length of the service protocol data, and determining the communication protocol packet length, where the communication protocol packet length is the sum of the data length and the communication protocol packet header length.

[0058] Among them, the communication protocol refers to the rules and agreements that both entities must follow to complete communication or service. In order for multiple data communication systems in different geographical locations to be interconnected through communication channels and devices to work together to achieve information exchange and resource sharing, they must have a common language. What to communicate, how to communicate, and when to communicate must follow certain mutually acceptable rules. For example, the Hyper Text Transfer Protocol (HTTP) is a request-response communication protocol that specifies what kind of messages the client may send to the server and what kind of response it will get. The headers of the request and response messages are given in ASCII form. The communication protocol header message consists of a request from the client to the server and a response from the server to the client. It generally includes a header and a subject part. For example, for the Hypertext Transfer Protocol, its request message format is: request line-general information header-request header-entity header-message body. The communication protocol header is before the message body.

[0059] Specifically, when the communication protocol header is composed of a fixed part and the length data of the business protocol field data, after obtaining the data length of the business protocol field data, the data length of the business protocol field data can be combined with the fixed part of these communication protocols to obtain the communication protocol header of the communication process, and then the communication protocol header is combined with the business protocol field data to obtain the complete communication protocol data, and the data length of the communication protocol header is combined with the data length of the business protocol field data to determine the communication protocol packet length. For example, in one embodiment, the present application is applied to the content distribution network communication process of instant messaging software. The business protocol adopts the KV protocol, and the communication protocol adopts the HTTP protocol. At this time, the business protocol field contains three parts of content, namely, wenxinnum=12345678, token=f0131dfd, data=...(128K), at this time, the lengths of these three fields can be calculated separately, and then these lengths are combined to obtain the data length of the business protocol field data is 131130 bytes. Then combine the fixed length of the HTTP header and the data length 131130 of the business protocol field data to obtain the communication protocol header, that is, the fixed length of the HTTP header plus 6 bytes. Then combine the communication protocol header with the business protocol field data to obtain the final communication protocol packet length of the fixed length of the HTTP header + 6 + 131130 bytes.

[0060] Step 207: Allocate memory of the communication protocol packet length to the communication thread of the service communication request.

[0061] Among them, memory means that during the communication process, the access server 104 will allocate the corresponding part of the physical memory to the communication process for use according to the needs of the communication process, and then the access server 104 will serialize the communication protocol header and business protocol field data into the memory to obtain complete communication protocol data, and complete the communication processing through the communication thread.

[0062] Specifically, after determining the complete communication protocol packet length, the corresponding communication-related processing can be performed on the communication protocol packet, and in order to realize the serialization of the communication protocol data, the memory of the communication protocol packet length can be first allocated to the communication thread of the business communication request. For example, in one of the embodiments, the present application is applied to the content distribution network communication process of the instant messaging software. After determining that the final communication protocol packet length is 131180 bytes, the content distribution network communication process of the access server 104 will apply for a memory block of 131180 bytes, and the access server 104 will allocate a memory block of 131180 bytes for the communication process. The serialization of the communication protocol data is completed on this memory block to obtain the communication protocol data that can be used for transmission.

[0063] Step 209, copy the communication protocol header and the service protocol field data to the memory in a serialized form to obtain serialized communication protocol data.

[0064] Among them, serialization refers to serializing an object into a binary form (byte array). Serialization is also called encoding, and is mainly used in the fields of network transmission and data persistence. By copying the communication protocol header and business protocol field data in binary form to the memory, the serialization of the communication protocol data can be completed, and the communication protocol data that can be used for network transmission can be obtained.

[0065] Specifically, after applying for memory for the communication thread, the communication protocol header and business protocol field data constructed previously can be copied to the memory in binary form to complete the serialization process of the communication protocol data. In one embodiment, the present application is applied to the content distribution network communication process of the instant messaging software. When the final communication protocol packet length is determined and the memory block is allocated to the communication process of the content distribution network, the communication protocol header and the business protocol field data can be serialized separately. For example, for the HTTP communication protocol, the HTTP protocol header can be serialized from POST, and then the business protocol field can be serialized. Finally, all the complete communication protocol data can be serialized to the memory block to complete the complete processing flow of the communication protocol data. Since the size of the memory to be allocated has been calculated in advance before serialization, the communication protocol header and the business protocol field data can be directly copied to the allocated memory block in sequence without the need to serialize the business protocol at the offset position of the memory block, and serialize the communication protocol header in another memory block, and copy the serialized communication protocol header to the previous memory block. It can effectively reduce the number of copies in the serialization process of the communication protocol data and improve communication efficiency.

[0066] The above communication protocol data serialization method extracts the business protocol field data of the business communication request after receiving the business communication request, obtains the basic data of the communication process, and then determines the data length of the business protocol field data; determines the length of the business protocol part that needs to be serialized, and obtains the communication protocol packet header based on the data length of the business protocol data, and determines the communication protocol packet length. After obtaining the data length of the business protocol field data, the communication protocol header can be obtained, and the length of the entire communication protocol packet can be determined. Finally, the memory of the communication protocol packet length can be directly allocated to the communication thread of the business communication request, and the communication protocol header and the business protocol field data are copied to the memory, and the serialization processing of the communication protocol data of the communication process is completed to obtain the serialized communication protocol data. In this embodiment, by pre-processing the length of the business protocol field data after serialization, and the communication protocol packet length that depends on the length of the business protocol field data, the serialization processing of the communication protocol data can be directly performed on a piece of memory, that is, only one copy is required to complete the serialization processing of the communication protocol data, which can effectively improve the efficiency of the communication protocol data serialization, thereby improving the communication performance.

[0067] In one embodiment, determining the data length of the business protocol field data includes: determining each business protocol field in the business protocol field data; performing field length identification processing on each business protocol field to obtain the data length of different business protocol fields; and determining the data length of the business protocol field data based on the sum of the data lengths of different business protocol fields.

[0068] Specifically, the business protocol field data is composed of multiple different business protocol fields, so the serialization of the business protocol field data is the serialization of each business protocol field in the business protocol field data. Therefore, when calculating the data length of the serialized business protocol field data, you can first determine the business protocol fields contained in the business protocol field data, that is, by performing field length identification processing on each business protocol field respectively, to obtain the data length corresponding to the serialized different business protocol fields. And because the complete business protocol field data can be obtained by splicing different business protocol fields together, the total length of the business protocol field data is equal to the sum of the data lengths of different business protocol fields. For example, in one of the embodiments, Figure 3As shown, the business protocol field data includes the field "wenxinnum", the field "token" and the field "data", wherein for the field wenxinnum=12345678, the data length=13+12=25 bytes, for the field token=f0131dfd, the length=(4+strlen("token"))+(4+strlen("f0131dfd"))=21 bytes, for the field data=...(128KB), the length=(4+strlen("data"))+(4+128K)=131084 bytes, and the total business protocol data length is 131130 bytes. In this embodiment, by splitting the business protocol field data into different business protocol fields and performing field length identification processing respectively, and by specifically identifying the length of each business protocol field, the overall length of the business protocol field data can be effectively determined, thereby ensuring the accuracy and efficiency of the communication protocol data serialization process.

[0069] In one embodiment, field length identification processing is performed on each business protocol field respectively to obtain the data length of different business protocol fields, including: determining the field name and field value of each business protocol field, and obtaining the field type of the field value; performing field length identification processing on the field name based on a string counting function to obtain the field name length, and performing field length identification processing on the field value based on the field type of the field value to obtain the field value length; adding the field name length and field value length of each business protocol field to obtain the data length of different business protocol fields.

[0070] The business protocol field specifically includes two parts of data: field name and field value. The field name is the name of the business protocol field, and the field value is the specific parameter value of the field. String counting function refers to a type of function used to identify the length of a string. For example, for C / C++ or JAVA, the strlen() function can be used to identify strings. It starts scanning from a certain location in the memory (it can be the beginning of the string, a certain location in the middle, or even an uncertain memory area) until it encounters the first string terminator '\0', and then returns the counter value (the length does not include '\0').

[0071] Specifically, for the data length calculation process of each business protocol field, it can be divided into two parts, field name and field value, and calculated separately. For the field name, since the field name is generally a string structure, it can be directly identified by the string counting function. For the field value, it can be identified based on the specific type of the field value. That is, the field name is identified based on the string counting function to obtain the field name length, and the field value is identified based on the field type of the field value to obtain the field value length. After obtaining the field name length and field value length, the field name length and field value length of each business protocol field can be added to obtain the data length of different business protocol fields. For example, in one of the embodiments, the business protocol of the present application is specifically a k / v protocol. At this time, the field length of each business protocol field can be identified based on the data type of the k / v protocol, and the k / v protocol uses length-value encoding for the field name and field value, where the length is fixed to 4 bytes, so the identification of the field length focuses on the value length, for example, for a business protocol field of wenxinnum=12345678, where wenxinnum is the field name and 12345678 is the field value. First, the length is identified through the string counting function: strlen("wenxinnum") = 9, the length of value is 9 bytes, plus the fixed 4 bytes of length, that is, the field name is a total of 13 bytes. For the field value, the field length can be identified through the field type of the field value. For string field value data, it can be identified by using the string counting function method like the field name. For other types of field value data, the length can be identified based on the specific data type of the field value data. Finally, the field name length and field value length of each business protocol field are added to obtain the data length of different business protocol fields. In this embodiment, the length of the field name is identified through the string counting function, and then the data length of the business protocol field is identified in combination with the field value length, which can effectively ensure the calculation accuracy and efficiency of the data length of the business protocol field.

[0072] In one of the embodiments, the field type of the field value includes an integer type; based on the field type of the field value, the field value is identified by field length, and obtaining the field value length includes: serializing the integer field value into string data; determining the data length of the string data through a string length identification function, and using the data length of the string data as the field value length of the integer field value.

[0073] Specifically, for integer data, field recognition can be achieved by serialization. By serializing the integer field value into string data, the integer field value data can be recognized by using a string counting function in a string manner. In a specific embodiment, Figure 4 As shown, for the integer data "12345678" after the sequence, it can be calculated in the way of a string, strlen ("12345678") = 8, the length of value is 8 bytes, plus the fixed 4 bytes of length, that is, the field name has a total of 12 bytes. In this embodiment, by converting the integer field value into string data, and then combining the string length identification function to realize the identification of the field value length, the calculation accuracy and efficiency of the service protocol field data length can be effectively guaranteed.

[0074] In one embodiment, the field type of the field value includes an integer type. Performing field length identification processing on the field value based on the field type of the field value to obtain the field value length includes: determining a numerical interval of the integer field value; performing interval table lookup processing based on the numerical interval to obtain length information corresponding to the numerical interval, and obtaining the field value length of the integer field value.

[0075] Among them, the interval table lookup processing only identifies the numerical interval where the integer field value data is located, and then searches the interval length corresponding table based on the numerical interval to identify the field value length of the integer field value.

[0076] Specifically, since integer data with the same number of digits can be divided into corresponding intervals, for example, the 9 integers from 1 to 9 are all in the interval [10^0, 10^1). Therefore, in addition to serializing the integer data, the field length identification processing for the integer field value can also directly determine the numerical interval of the integer field value; that is, based on the numerical interval, an interval table lookup is performed to obtain the length information corresponding to the numerical interval, and the field value length of the integer field value is obtained. For example, for the integer data 12345678, which is between the interval [10^7, 10^8), an interval table lookup is performed based on the numerical interval, and the field value length of the integer field value can be obtained. It can be determined that the length of the integer data is 8. The process of calculating the length directly by interval table lookup can be referred to. Figure 5As shown, the data length of integer data within the range can be determined by table lookup. In a specific embodiment, for the process of determining the numerical interval of an integer field value, the average field value of the integer field value can be first determined; the field value average is processed by taking the logarithm of the average; the numerical interval of the logarithm of the average is determined; and the numerical interval of the integer field value is searched based on the numerical interval of the logarithm of the average. Specifically, since the distribution of field value data under different field names is completely different, such as the average size of thumbnails is 10KB, the average size of original images is 1MB, and the average size of videos is 10MB. Therefore, in order to perform a more efficient numerical interval search, the field value average of the integer field value can be determined; the field value average is processed by taking the logarithm of the average to obtain the logarithm of the average. Then determine the numerical interval of the logarithm of the average; and search for the numerical interval of the integer field value based on the numerical interval of the logarithm of the average. In a specific embodiment, as Figure 6 As shown, the average value avg_val and the corresponding avg=log(avg_val) can be calculated for each field value. For example, avg_val=1234, avg=3, and the corresponding numerical interval of avg3 is in the interval [10^3, 10^4). That is, the numerical interval of the integer field value can be found based on the numerical interval of the logarithm of the average. Then, for the process of determining the numerical interval of the integer field value and calculating the length of the field value, the logarithm of the integer field value and the logarithm of the average can be compared to obtain the field value comparison result; the interval search direction is determined based on the field value comparison result; based on the interval search direction, the numerical interval of the logarithm of the average is used as a reference to find the numerical interval of the integer field value. Taking the numerical interval avg of the logarithm of the average as a reference, if the logarithm of the current field value is greater than avg, refer to Figure 7 Search upward for the value interval corresponding to the corresponding length. If the logarithm of the current field value is less than avg, refer to Figure 8 Look down for the numerical interval corresponding to the corresponding length, thereby effectively improving the search efficiency of the numerical interval. In particular, for floating-point numerical data, it can be converted into integer data before length recognition. For example, for the floating-point numerical data 3.1415926, retain 2 digits, 314, and directly calculate the serialized length of 314. For negative numerical data, the negative number can be converted to a positive number before recognition, such as the length of -1234 is 1234 length + 1. In this embodiment, the field length recognition of integer field value data is realized by interval table lookup, which can effectively improve the recognition efficiency while ensuring the recognition accuracy.

[0077] In one embodiment, the field type of the field value includes an enumeration type. Performing field length identification processing on the enumeration type field value to obtain the field value length includes: determining an enumeration type data length corresponding table corresponding to the enumeration type field value; performing field search in the enumeration type data length corresponding table based on the enumeration type field value to obtain the field value length.

[0078] Specifically, for enumerated variable value data, since the value of the enumerated variable is limited to the range of the listed values, the length of the enumerated field value data and its corresponding length are also within the listed range. Therefore, the length of each enumerated variable can be calculated in advance to construct an enumerated data length correspondence table, in which the enumerated variables and variable lengths correspond one to one. When it is necessary to determine the length of the enumerated variable, the enumerated data length correspondence table corresponding to the enumerated field value can be directly determined, and then the field is searched in the enumerated data length correspondence table based on the enumerated field value, and the found value is used as the field value length. For example, if the value of the enumerated field type is {1, 10, 100, 1000}, you can create a table:

[0079] switch(type){

[0080] case 1:return 1;

[0081] case 10:return 2;

[0082] case 100:return 3;

[0083] case 1000:return 4;

[0084] In this embodiment, by pre-building an enumeration data length corresponding table to realize the length calculation of enumeration data, the field value length of enumeration field value data can be efficiently determined, thereby ensuring the recognition efficiency and accuracy of the data length of the service protocol field data.

[0085] In one embodiment, a communication protocol packet header is obtained based on the data length of the business protocol data, and determining the communication protocol packet length includes: determining the length information of the data length of the business protocol data; obtaining the communication protocol header based on the fixed parameters of the communication protocol header and the length information of the data length; determining the communication protocol packet length based on the length of the fixed parameters of the communication protocol header, the length information of the data length and the data length of the business protocol data.

[0086] Specifically, the fixed parameters of the communication protocol header refer to the fixed content of the communication protocol part. For example, for the HTTP communication protocol, the fixed parameters of its header can be POST / sayhello HTTP / 1.1\r\n, and the length information of the data length is also one of the components of the header. Therefore, the fixed parameters of the communication protocol header and the length information of the data length can be combined to obtain the communication protocol header. After obtaining the communication protocol header, the length of the fixed parameters of the communication protocol header, the length information of the data length, and the data length of the service protocol data can be added together to determine the final communication protocol packet length. In a specific embodiment, if the length of the fixed parameters of the communication protocol header is 44 bytes, and the length information of the data length is 131130 bytes, then the length information of the data length is 6 bytes, then the length of the fixed parameters of the communication protocol header, the length information of the data length, and the data length of the service protocol data are finally added together, and the final communication protocol packet length obtained is 44+6+131130=131180 bytes, that is, a memory block of 131180 bytes needs to be allocated to the current communication process.

[0087] In one embodiment, step 201 includes: receiving a service communication request from a client, and obtaining service protocol field data corresponding to the service communication request from the service communication request. The method also includes: generating a content sending message based on the serialized communication protocol data; and pushing the content sending message to the target end of the service communication request.

[0088] Specifically, the scheme of the present application can be applied to the process of data transmission. When the user needs to send business data to the outside world through the method of the present application, the business communication request containing the business data to be sent can be sent to the access server equipped with the method of the present application. The access server obtains the business protocol field data corresponding to the business communication request from the business communication request, and then completes the initialization processing process of the network communication protocol, and sends the data through the network, that is, generates a content sending message based on the serialized communication protocol data; pushes the content sending message to the target end of the business communication request. In addition, the method of the present application can also be specifically applied to the source return process of the content distribution network. At this time, the business communication request submitted by the client is mainly used to request business data from the specified business data source station, and the server can obtain the business protocol field data corresponding to the business communication request from the target source station, and then generate a content feedback message based on the serialized communication protocol data; and then send the content feedback message back to the client of the business communication request. The client can obtain the requested business data according to the content returned by the server. In this embodiment, the data is sent or returned to the source by obtaining the business protocol field data corresponding to the business communication request, thereby ensuring the effectiveness of the communication process.

[0089] The present application also provides an application scenario, which applies the above-mentioned communication protocol data serialization method. Specifically, the application of the communication protocol data serialization method in this application scenario is as follows:

[0090] When a user wants to transmit business data through instant messaging software, the communication protocol data serialization method of this application can be used to serialize the communication protocol data generated based on the business data, where the business protocol adopts the k / v protocol, the communication protocol adopts the HTTP protocol, and the data is transmitted through the content distribution network. The overall serialization process of this application can be referred to Fig. 9 As shown, first, the user submits a business communication request on the instant messaging software, which contains three fields, one of which is the user number "wenxinnum" on the instant messaging software, the second is the token field "token", and the third is the data "data" field. The field value of the "wenxinnum" field is 12345678, the field value of the "token" field is f0131dfd, and the "data" field is the business data, a total of 128k, and then it is necessary to determine the data length of the three fields of the business protocol field data, that is, to perform field length identification processing on each business protocol field respectively, and obtain the data length of different business protocol fields; based on the sum of the data lengths of different business protocol fields, determine the data length of the business protocol field data. Among them, for the "wenxinnum" field name, through the length calculation method of the k / v protocol, it is determined that the length of the string field is "wenxinnum"=>4+strlen("wenxinnum")=>4+9=13. For the integer field name "12345678", by serializing it into a string, we get "12345678"=>4+strlen("12345678")=>4+8=12, so for wenxinnum=12345678, its length=13+12=25 bytes. In addition, instead of using the additional serialization method, we can use the interval table lookup method. Since 12345678 is between [10^7,10^8), the length of the field value in its interval is 8. Similarly, for "token=f0131dfd", the calculated total length=(4+strlen("token"))+(4+strlen("f0131dfd"))=21 bytes, and for data=...(128KB), its length=(4+strlen("data"))+(4+128K)=131084 bytes. Therefore, the data length of the service protocol field data is 131130 bytes. Based on the length of the service protocol field data, the communication protocol packet header can be obtained based on the data length of the service protocol data, such as Fig. 9As shown in the upper right corner, based on the length of the fixed parameters of the communication protocol header, the length information of the data length and the data length of the business protocol data, the total length of the communication protocol packet is determined in advance to be 131180 bytes, and then the corresponding 131180 bytes of memory are applied, and the communication protocol data is copied to the memory in a serialized form, and the following is obtained: Fig. 9 The serialized communication protocol data is shown in the lower middle part. Then, a content sending message can be generated based on the serialized communication protocol data; and the content sending message is pushed to the target end of the business communication request.

[0091] In one embodiment, the complete process of the communication protocol data serialization method of the present application can be referred to Fig.10 As shown, including:

[0092] Step 1001, obtain the business protocol field data corresponding to the business communication request. Step 1003, determine each business protocol field in the business protocol field data. Step 1005, determine the field name and field value of each business protocol field, and obtain the field type of the field value. Step 1007, perform field length identification processing on the field name based on the string counting function to obtain the field name length, and perform field length identification processing on the field value based on the field type of the field value to obtain the field value length. Step 1009, add the field name length and field value length of each business protocol field to obtain the data length of different business protocol fields. Step 1011, determine the data length of the business protocol field data based on the sum of the data lengths of different business protocol fields. Step 1013, determine the length information of the data length of the business protocol data. Step 1015, obtain the communication protocol header based on the fixed parameters of the communication protocol header and the length information of the data length. Step 1017, determine the communication protocol packet length based on the length of the fixed parameters of the communication protocol header, the length information of the data length, and the data length of the business protocol data. Step 1019, allocate memory of the communication protocol packet length to the communication thread of the service communication request. Step 1021, copy the communication protocol header and service protocol field data to the memory in a serialized form to obtain serialized communication protocol data.

[0093] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0094] Based on the same inventive concept, the embodiment of the present application also provides a communication protocol data serialization device for implementing the communication protocol data serialization method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more communication protocol data serialization device embodiments provided below can refer to the limitations of the communication protocol data serialization method above, and will not be repeated here.

[0095] In one embodiment, Fig.11 As shown, a communication protocol data serialization device is provided, comprising:

[0096] The request receiving module 1102 is used for the request receiving module to obtain the business protocol field data corresponding to the business communication request, and the business communication request is used to send the business protocol field data or request the business protocol field data.

[0097] The protocol length determination module 1104 is used to determine the data length of the service protocol field data.

[0098] The communication protocol determination module 1106 is used to obtain the communication protocol packet header based on the data length of the service protocol data and determine the communication protocol packet length.

[0099] The memory allocation module 1108 is used to allocate memory of the communication protocol packet length to the communication thread of the business communication request.

[0100] The protocol serialization module 1110 is used to copy the communication protocol header and the business protocol field data to the memory in a serialized form to obtain serialized communication protocol data.

[0101] In one embodiment, the protocol length determination module 1104 is specifically used to: determine each business protocol field in the business protocol field data; perform field length identification processing on each business protocol field to obtain the data length of different business protocol fields; and determine the data length of the business protocol field data based on the sum of the data lengths of different business protocol fields.

[0102] In one embodiment, the protocol length determination module 1104 is specifically used to: determine the field name and field value of each business protocol field, and obtain the field type of the field value; perform field length identification processing on the field name based on a string counting function to obtain the field name length, and perform field length identification processing on the field value based on the field type of the field value to obtain the field value length; add the field name length and field value length of each business protocol field to obtain the data length of different business protocol fields.

[0103] In one embodiment, the field type of the field value includes an integer type; the protocol length determination module 1104 is also used to: perform field length identification processing on the integer field value, and obtain the field value length including: serializing the integer field value into string data; determining the data length of the string data through a string length identification function, and using the data length of the string data as the field value length of the integer field value.

[0104] In one embodiment, the field type of the field value includes an integer type; the protocol length determination module 1104 is also used to: determine the numerical interval of the integer field value; perform interval table lookup processing based on the numerical interval to obtain the length information corresponding to the numerical interval, and obtain the field value length of the integer field value.

[0105] In one embodiment, the protocol length determination module 1104 is also used to: determine the field value average of the integer field value; perform logarithmic processing on the field value average to obtain the logarithm of the average; determine the numerical interval of the logarithm of the average; and search for the numerical interval of the integer field value based on the numerical interval of the logarithm of the average.

[0106] In one embodiment, the protocol length determination module 1104 is also used to: compare the logarithm of the integer field value with the logarithm of the mean to obtain a field value comparison result; determine the interval search direction based on the field value comparison result; based on the interval search direction, take the numerical interval of the logarithm of the mean as a reference to find the numerical interval of the integer field value.

[0107] In one embodiment, the field type of the field value includes an enumeration type; the protocol length determination module 1104 is also used to: determine the enumeration type data length corresponding table corresponding to the enumeration type field value; perform a field search in the enumeration type data length corresponding table based on the enumeration type field value to obtain the field value length.

[0108] In one embodiment, the communication protocol determination module 1106 is specifically used to: determine the length information of the data length of the business protocol data; obtain the communication protocol header based on the fixed parameters of the communication protocol header and the length information of the data length; determine the communication protocol packet length based on the length of the fixed parameters of the communication protocol header, the length information of the data length and the data length of the business protocol data.

[0109] In one embodiment, the request receiving module 1102 is specifically used to: receive a service communication request from a client, and obtain service protocol field data corresponding to the service communication request from the service communication request. The device also includes a content sending module, which is used to: generate a content sending message based on the serialized communication protocol data; and push the content sending message to the target end of the service communication request.

[0110] In one embodiment, the request receiving module 1102 is specifically used to: receive a business communication request from a client, and obtain business protocol field data corresponding to the business communication request from a target source station; the device also includes a content return source module, which is used to: generate a content feedback message based on serialized communication protocol data; and return the content feedback message to the client of the business communication request.

[0111] Each module in the above communication protocol data serialization device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0112] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store relevant data of the communication protocol data serialization process. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a communication protocol data serialization method is implemented.

[0113] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0114] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0115] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0116] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0117] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0118] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0119] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0120] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A communication protocol data serialization method, characterized in that: The method comprises: Acquire the service protocol field data corresponding to the service communication request, wherein the service communication request is used to send the service protocol field data or request the service protocol field data; Determine the data length of the service protocol field data; Obtaining a communication protocol packet header based on the data length of the service protocol data, and determining a communication protocol packet length, wherein the communication protocol packet length is the sum of the data length and the length of the communication protocol packet header; Allocating memory of the communication protocol packet length to the communication thread of the service communication request; The communication protocol header and the service protocol field data are copied to the memory in a serialized form to obtain serialized communication protocol data.

2. The method according to claim 1, characterized in that: Determining the data length of the service protocol field data includes: Determining each business protocol field in the business protocol field data; Performing field length identification processing on each of the service protocol fields respectively to obtain data lengths of different service protocol fields; The data length of the service protocol field data is determined based on the sum of the data lengths of the different service protocol fields.

3. The method according to claim 2, characterized in that The performing field length identification processing on each of the service protocol fields to obtain the data lengths of different service protocol fields comprises: Determine the field name and field value of each business protocol field, and obtain the field type of the field value; Performing field length identification processing on the field name based on a string counting function to obtain the field name length, and performing field length identification processing on the field value based on the field type of the field value to obtain the field value length; The field name length and field value length of each service protocol field are added together to obtain the data length of different service protocol fields.

4. The method according to claim 3, characterized in that: The field type of the field value includes integer type; The performing field length identification processing on the field value based on the field type of the field value to obtain the field value length includes: Serialize the integer field value into string data; The data length of the string type data is determined by a string length identification function, and the data length of the string type data is used as the field value length of the integer field value.

5. The method according to claim 3, characterized in that: The field type of the field value includes integer type; The performing field length identification processing on the field value based on the field type of the field value to obtain the field value length includes: Determine a numerical range of the integer field value; An interval table lookup is performed based on the numerical interval to obtain the length information corresponding to the numerical interval, and the field value length of the integer field value is obtained.

6. The method according to claim 5, characterized in that Determining the numerical range of the integer field value includes: Determine the average of the integer field values; Performing logarithm processing on the average of the field values ​​to obtain the logarithm of the average; Determining a numerical interval of the logarithm of the mean; The numerical interval of the integer field value is found based on the numerical interval of the logarithm of the mean.

7. The method according to claim 6, characterized in that The step of searching for the numerical interval of the integer field value based on the numerical interval of the mean logarithm includes: Compare the logarithm of the integer field value with the logarithm of the average to obtain a field value comparison result; Determine the interval search direction based on the field value comparison result; Based on the interval search direction, the numerical interval of the logarithm of the mean is used as a reference to find the numerical interval of the integer field value.

8. The method according to claim 3, characterized in that The field type of the field value includes an enumeration type; Perform field length identification processing on enumeration field values, and obtain the field value length including: Determine the enumeration type data length corresponding to the enumeration type field value; Based on the enumeration type field value, a field search is performed in the enumeration type data length corresponding table to obtain the field value length.

9. The method according to claim 1, characterized in that: The obtaining of the communication protocol packet header based on the data length of the service protocol data and determining the communication protocol packet length comprises: Determine the length information of the data length of the service protocol data; Obtaining a communication protocol header based on the fixed parameters of the communication protocol header and the length information of the data length; The communication protocol packet length is determined based on the length of the fixed parameter of the communication protocol header, the length information of the data length, and the data length of the service protocol data.

10. The method according to any one of claims 1 to 9, characterized in that The business protocol field data corresponding to the business communication request is obtained including: Receive a service communication request from a client, and obtain service protocol field data corresponding to the service communication request from the service communication request; The method further comprises: Generate a content sending message based on the serialized communication protocol data; Push the content to send a message to the target end of the business communication request.

11. The method according to any one of claims 1 to 9, characterized in that The obtaining of the business protocol field data corresponding to the business communication request also includes: Receive a service communication request from a client, and obtain service protocol field data corresponding to the service communication request from a target source station; The method further comprises: generating a content feedback message based on the serialized communication protocol data; The content feedback message is sent back to the client of the service communication request.

12. A communication protocol data serialization device, characterized in that: The device comprises: A request receiving module, used to obtain the business protocol field data corresponding to the business communication request, wherein the business communication request is used to send the business protocol field data or request the business protocol field data; A protocol length determination module, used to determine the data length of the service protocol field data; A communication protocol determination module, used to obtain a communication protocol packet header based on the data length of the service protocol data, and determine the communication protocol packet length; A memory allocation module, used to allocate memory of the communication protocol packet length to the communication thread of the business communication request; The protocol serialization module is used to copy the communication protocol header and the business protocol field data to the memory in a serialized form to obtain serialized communication protocol data.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images