Message data processing method and related equipment

Abstract

The present application discloses a method for processing message data and related equipment, which can parse the message header information in the message data to be transmitted to obtain the message header field of the message data; query the index information of the message header field in the local message header mapping information; when the index information of the message header field is not queried, pull the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information; re-query the index information of the message header field in the updated local message header mapping information; when the target index information corresponding to the message header field is queried, package the target index information and the message content to obtain the compressed message data, and transmit the compressed message data. The present application can reduce the amount of data transmitted, and can also better maintain the sender's message header mapping information by deploying the configuration update center of the message header mapping information on the server.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for processing message data and related equipment. Background Art

[0002] A message is a unit of data exchanged and transmitted in a network. In addition to the data information that needs to be transmitted, each message usually also needs to identify the network address, network protocol, message length and other descriptive information used to transmit the message. This descriptive information is also called the message header field. This descriptive information can generally be reused in different messages. In current related technologies, the message header field is usually directly transmitted in plain text with other message content. As network business functions become more and more complex, the header traffic is increasing, which leads to the problem of waste of network resources due to data redundancy. Summary of the invention

[0003] An embodiment of the present application provides a method for processing message data and related equipment. The related equipment may include a message data processing device, an electronic device, a computer-readable storage medium and a computer program product. It can reduce the amount of data transmitted and can also be used through a unified data configuration center to facilitate the maintenance of local message header mapping information of the sender and the receiver. The data consistency of the sender and the receiver can be more effectively guaranteed.

[0004] The present application embodiment provides a method for processing message data, which is applied to a sending device, including:

[0005] Acquire message data to be transmitted and local message header mapping information, wherein the message data includes message header information and message content, and the message header mapping information includes a mapping relationship between preset message header fields and preset index information;

[0006] Parsing the message header information in the message data to obtain the message header field of the message data;

[0007] Querying the index information of the message header field in the local message header mapping information;

[0008] When the index information of the message header field is not found in the local message header mapping information, the remote message header mapping information is pulled from the server to update the local message header mapping information based on the remote message header mapping information;

[0009] Re-querying the index information of the message header field in the updated local message header mapping information;

[0010] When the target index information corresponding to the message header field is found, the target index information and the message content are packaged to obtain compressed message data, and the compressed message data is transmitted.

[0011] The embodiment of the present application also provides a method for processing message data, which is applied to a receiving device, including:

[0012] Receiving compressed message data, wherein the compressed message data includes message header information and message content;

[0013] Extracting the message header information from the compressed message data, and detecting the message header information;

[0014] When it is detected that the message header information is the target index information after message header compression, query the message header field corresponding to the target index information in the local message header mapping information, wherein the message header mapping information includes a mapping relationship between a preset message header field and preset index information;

[0015] When the message header field corresponding to the target index information is not found in the local message header mapping information, the remote message header mapping information is pulled from the server to update the local message header mapping information based on the remote message header mapping information;

[0016] Re-querying the message header field corresponding to the target index information in the updated local message header mapping information;

[0017] When the target message header field corresponding to the target index information is found, the target message header field and the message content in the compressed message data are stored.

[0018] The embodiment of the present application also provides a method for processing message data, which is applied to a server, including:

[0019] Acquire target message header data stored in a receiving device, wherein the target message header data includes a plurality of message header fields;

[0020] Determine, from the target message header data, newly added message header fields and non-newly added message header fields based on remote message header mapping information, wherein the remote message header mapping information includes a mapping relationship between preset message header fields and preset index information;

[0021] Counting the frequency information of each non-newly added message header field in the target message header data;

[0022] Updating remote message header mapping information according to the frequency information and the newly added message header field;

[0023] The updated remote message header mapping information is sent to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information.

[0024] Accordingly, an embodiment of the present application provides a method and apparatus for processing message data, which is applied to a sending device, including:

[0025] An acquisition unit, used to acquire message data to be transmitted and local message header mapping information, wherein the message data includes message header information and message content, and the message header mapping information includes a mapping relationship between a preset message header field and preset index information;

[0026] A parsing unit, used to parse the message header information in the message data to obtain the message header field of the message data;

[0027] A first query unit, used to query the index information of the message header field in the local message header mapping information;

[0028] A first pulling unit, configured to pull remote header mapping information from a server when the index information of the header field is not found in the local header mapping information, so as to update the local header mapping information based on the remote header mapping information;

[0029] A second query unit, used to re-query the index information of the message header field in the updated local message header mapping information;

[0030] The transmission unit is used to package the target index information and the message content to obtain compressed message data and transmit the compressed message data when the target index information corresponding to the message header field is queried.

[0031] Accordingly, an embodiment of the present application further provides a method and apparatus for processing message data, which is applied to a receiving device, including:

[0032] A receiving unit, configured to receive compressed message data, wherein the compressed message data includes message header information and message content;

[0033] An extraction unit, used to extract the message header information from the compressed message data and detect the message header information;

[0034] A third query unit is used to query the message header field corresponding to the target index information in the local message header mapping information when it is detected that the message header information is the target index information after message header compression, wherein the message header mapping information includes a mapping relationship between a preset message header field and a preset index information;

[0035] A second pulling unit is used to pull remote message header mapping information from the server when the message header field corresponding to the target index information is not found in the local message header mapping information, so as to update the local message header mapping information based on the remote message header mapping information;

[0036] A fourth query unit, configured to re-query the message header field corresponding to the target index information in the updated local message header mapping information;

[0037] A storage unit is used to store the target message header field and the message content in the compressed message data when the target message header field corresponding to the target index information is queried.

[0038] Accordingly, the embodiment of the present application further provides a method and device for processing message data, which is applied to a server and includes:

[0039] A data acquisition unit, used to acquire target message header data stored in a receiving device, wherein the target message header data includes a plurality of message header fields;

[0040] A determining unit, configured to determine, from the target message header data, newly added message header fields and non-newly added message header fields based on remote message header mapping information, wherein the remote message header mapping information includes a mapping relationship between preset message header fields and preset index information;

[0041] A statistical unit, used to count the frequency information of each non-newly added message header field in the target message header data;

[0042] A first updating unit, configured to update remote message header mapping information according to the frequency information and the newly added message header field;

[0043] The second updating unit is used to trigger the sending of updated remote message header mapping information to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information.

[0044] Optionally, in some embodiments of the present application, the message data processing device further includes a data transmission unit, as follows:

[0045] The data transmission unit is used to package the message header field and the message content to obtain packaged message data when the target index information corresponding to the message header field cannot be found in the updated local message header mapping information; and transmit the packaged message data.

[0046] Optionally, in some embodiments of the present application, the message data processing device further includes a data storage unit, as follows:

[0047] The data storage unit is used to store the uncompressed message header field and the message content in the compressed message data when it is detected that the message header information is an uncompressed message header field.

[0048] Optionally, in some embodiments of the present application, the message header field includes a field name and a field value; the determination unit may be specifically configured to filter out the newly added field name and the non-newly added field name from the target message header data based on the field name in the current remote message header mapping information;

[0049] The statistical unit may be specifically configured to count each field value corresponding to the non-newly added field name in the target message header data to obtain the frequency information of occurrence of each field value corresponding to the non-newly added field name;

[0050] The first updating unit may include an adding subunit and an updating subunit, as follows:

[0051] The adding subunit is used to add the newly added field name and its corresponding index information to the remote message header mapping information;

[0052] The updating subunit is used to update the added remote message header mapping information according to the frequency information of each field value corresponding to the non-newly added field name.

[0053] Optionally, in some embodiments of the present application, the update sub-unit can be specifically used to update the field value corresponding to the non-newly added field name to the remote message header mapping information after being added when the frequency information of the field value corresponding to the non-newly added field name is greater than the first preset frequency information; when the frequency information of the field value corresponding to the non-newly added field name is not greater than the first preset frequency information, and when it is detected that the field value exists in the remote message header mapping information after being added, delete the field value from the remote message header mapping information after being added.

[0054] Optionally, in some embodiments of the present application, the remote message header mapping information includes first remote mapping information and second remote mapping information;

[0055] The adding subunit may be specifically used to add the newly added field name and its corresponding index information to the first remote mapping information;

[0056] The step of “updating the field value corresponding to the non-newly added field name to the added remote message header mapping information when the frequency information of the field value corresponding to the non-newly added field name is greater than the first preset frequency information” may include:

[0057] When the frequency information of the field value corresponding to the non-newly added field name is greater than the second preset frequency information, the non-newly added field name, the field value of the non-newly added field name and the corresponding index information are updated to the first remote mapping information after the addition, and the first preset frequency information is less than the second preset frequency information;

[0058] When the frequency information of the field value corresponding to the non-newly added field name is not greater than the second preset frequency information but greater than the first preset frequency information, the field value corresponding to the non-newly added field name and its corresponding index information are added to the second remote mapping information.

[0059] Optionally, in some embodiments of the present application, the message data processing method device may include a request update unit as follows:

[0060] The request update unit is used to receive pull request information sent by a target device, where the pull request information indicates pulling remote message header mapping information, and the target device is a sending device or a receiving device; based on the pull request information, the current remote message header mapping information is sent to the target device, so that the target device updates the local message header mapping information of the target device based on the current remote message header mapping information.

[0061] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0062] An electronic device provided in an embodiment of the present application includes a processor and a memory, wherein the memory stores a plurality of instructions, and the processor loads the instructions to execute the steps in the message data processing method provided in the embodiment of the present application.

[0063] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps in the method for processing message data provided in the embodiment of the present application are implemented.

[0064] In addition, an embodiment of the present application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps in the message data processing method provided in the embodiment of the present application.

[0065] The embodiment of the present application provides a method for processing message data and related equipment, which can obtain message data to be transmitted and local message header mapping information, wherein the message data includes message header information and message content, and the message header mapping information includes a mapping relationship between a preset message header field and a preset index information; the message header information in the message data is parsed to obtain the message header field of the message data; the index information of the message header field is queried in the local message header mapping information; when the index information of the message header field is not found in the local message header mapping information, the remote message header mapping information is pulled from the server to update the local message header mapping information based on the remote message header mapping information; the index information of the message header field is re-queried in the updated local message header mapping information; when the target index information corresponding to the message header field is queried, the target index information and the message content are packaged and processed to obtain compressed message data, and the compressed message data is transmitted. The present application can not only use the corresponding index information to compress the message header field through the message header mapping information to reduce the amount of transmitted data; but also the sender has its own message header mapping information locally, and in the compression process, the index information is first searched locally to improve the data compression efficiency. When the corresponding index information cannot be found locally on the sender, the remote message header mapping information saved on the server can be pulled to update the local message header mapping information of the sending device, so as to query the corresponding index information again to achieve compression of the message header field; wherein, this method of deploying the configuration update center of the message header mapping information on the server is conducive to better maintenance of the sender's message header mapping information;

[0066] Alternatively, compressed message data can be received, wherein the compressed message data includes message header information and message content; the message header information is extracted from the compressed message data, and the message header information is detected; when it is detected that the message header information is the target index information after the message header is compressed, the message header field corresponding to the target index information is queried in the local message header mapping information, and the message header mapping information includes a mapping relationship between a preset message header field and a preset index information; when the message header field corresponding to the target index information is not queried in the local message header mapping information, the remote message header mapping information is pulled from the server to update the local message header mapping information based on the remote message header mapping information; the message header field corresponding to the target index information is re-queried in the updated local message header mapping information; when the target message header field corresponding to the target index information is queried, the target message header field and the message content in the compressed message data are stored. The present application can compress the message header information through indexing to reduce the amount of transmitted data, and in the process of decompressing the message data, it can first be decompressed through the local message header mapping information of the receiver, which can improve the efficiency of decompression; when the corresponding message header field cannot be queried locally at the receiver, the remote message header mapping information saved in the server can be pulled to update the local message header mapping information of the receiving device, so as to query the corresponding message header field again to achieve decompression of the message header information; wherein, this method of deploying the configuration update center of the message header mapping information on the server is conducive to better maintenance of the message header mapping information of the receiver;

[0067] Alternatively, the target message header data stored in the receiving device can be obtained, the target message header data including multiple message header fields; based on the remote message header mapping information, the newly added message header fields and the non-newly added message header fields are determined from the target message header data, the remote message header mapping information including the mapping relationship between the preset message header fields and the preset index information; the frequency information of the corresponding occurrence of each non-newly added message header field in the target message header data is counted; according to the frequency information and the newly added message header field, the remote message header mapping information is updated; and the updated remote message header mapping information is triggered to be sent to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information. The present application can obtain the target message header data in the receiving device, and then perform statistical analysis on the target message header data to update the remote message header mapping information according to the statistical analysis results, thereby improving the integrity of the compression configuration data of the message header field stored in the remote message header mapping information; moreover, after the remote message header mapping information is updated, the updated data is actively pushed to the sender and the receiver to trigger the local data update of the sender and the receiver, so that through a unified data configuration center, it is beneficial to maintain the data of the sender and the receiver, and the data consistency of the sender and the receiver can be more effectively guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0069] Figure 1a It is a scenario diagram of a method for processing message data provided in an embodiment of the present application;

[0070] Figure 1b It is an illustration of a method for processing message data provided in an embodiment of the present application;

[0071] Figure 1c is another illustration of the method for processing message data provided by an embodiment of the present application;

[0072] Figure 1d is a flowchart of a method for processing message data provided by an embodiment of the present application;

[0073] Figure 1e is another illustration of the method for processing message data provided by an embodiment of the present application;

[0074] Figure 2a is another flow chart of the method for processing message data provided by an embodiment of the present application;

[0075] Figure 2b is another flow chart of the method for processing message data provided by an embodiment of the present application;

[0076] Figure 3a is another flow chart of the method for processing message data provided by an embodiment of the present application;

[0077] Figure 3b is another flow chart of the method for processing message data provided by an embodiment of the present application;

[0078] Figure 3c is another flow chart of the method for processing message data provided by an embodiment of the present application;

[0079] Figure 3d is another flow chart of the method for processing message data provided by an embodiment of the present application;

[0080] Figure 4a It is a structural schematic diagram of a message data processing device provided in an embodiment of the present application;

[0081] Figure 4b It is a structural schematic diagram of a message data processing device provided in an embodiment of the present application;

[0082] Figure 4c It is a structural schematic diagram of a message data processing device provided in an embodiment of the present application;

[0083] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0085] An embodiment of the present application provides a method for processing message data and related equipment. The related equipment may include a message data processing device, an electronic device, a computer-readable storage medium, and a computer program product.

[0086] The message data processing method provided in the embodiment of the present application can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and assisted driving. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data calculation, storage, processing, and sharing.

[0087] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The backend services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the backend system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, which can only be achieved through cloud computing.

[0088] Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, allowing various application systems to obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". From the user's perspective, the resources in the "cloud" are infinitely scalable and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0089] As a provider of basic cloud computing capabilities, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0090] The method for processing message data provided in the embodiment of the present application can be applied to Figure 1a In the application environment shown. The message data processing system provided in the embodiment of the present application includes a sending device 100, a receiving device 120 and a server 110. The sending device 100 and the receiving device 120 are connected via a network, and the sending device 100 and the receiving device 120 are also respectively connected to the server 110 for communication via a network, for example, via a wired or wireless network connection. The server 110 can be used to maintain and update the remote configuration center dictionary, and the remote configuration center dictionary can be integrated on the server 110 or placed on other servers.

[0091] In one embodiment, the sending device 100 can be used to: obtain message data to be transmitted and local message header mapping information, the message data including message header information and message content, the message header mapping information including a mapping relationship between a preset message header field and a preset index information; parse the message header information in the message data to obtain the message header field of the message data; query the index information of the message header field in the local message header mapping information; when the index information of the message header field is not found in the local message header mapping information, pull the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information; re-query the index information of the message header field in the updated local message header mapping information; when the target index information corresponding to the message header field is found, package the target index information and the message content to obtain compressed message data, and transmit the compressed message data. The sending device 100 may be a terminal device, and a client may be set on the terminal device, and the client may be an application client or a browser client, etc. The sending device 100 may specifically include various desktop computers, laptops, smart phones, tablet computers, IoT devices, portable wearable devices, vehicle terminals, aircraft, and network devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Network devices may be routers, switches, firewalls, load balancers, network storage, network adapters, etc.

[0092] The receiving device 120 can be used to: receive compressed message data, wherein the compressed message data includes message header information and message content; extract the message header information from the compressed message data, and detect the message header information; when it is detected that the message header information is the target index information after the message header is compressed, query the message header field corresponding to the target index information in the local message header mapping information, and the message header mapping information includes the mapping relationship between the preset message header field and the preset index information; when the message header field corresponding to the target index information is not found in the local message header mapping information, pull the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information; re-query the message header field corresponding to the target index information in the updated local message header mapping information; when the target message header field corresponding to the target index information is found, store the target message header field and the message content in the compressed message data. The receiving device 120 may be a terminal device, and a client may be set on the terminal device, and the client may be an application client or a browser client, etc. The receiving device 120 may specifically include various desktop computers, laptops, smart phones, tablet computers, IoT devices, portable wearable devices, vehicle terminals, aircraft, and network devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Network devices may be routers, switches, firewalls, load balancers, network storage, network adapters, etc.

[0093] The server 110 can be used to: obtain the target message header data stored in the receiving device, the target message header data includes multiple message header fields; based on the remote message header mapping information, determine the newly added message header field and the non-newly added message header field from the target message header data, the remote message header mapping information includes the mapping relationship between the preset message header field and the preset index information; count the frequency information of the corresponding occurrence of each non-newly added message header field in the target message header data; update the remote message header mapping information according to the frequency information and the newly added message header field; trigger to send the updated remote message header mapping information to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information. Among them, the server 110 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms.

[0094] The message data processing method provided in the present application can be applied to the transmission of mirrored traffic data, and specifically can be applied to the transmission of mirrored traffic data based on cloud service control traffic data. In real application scenarios, when some cloud service providers provide services to customers, in order to meet the user's data security requirements, they usually need to deploy their cloud service products in the user's local computer room. Since the cloud service control traffic is independent of the user's business traffic (to reduce the risk of mutual interference) and is transmitted between the cloud service center and the user's local device, in order to meet the user's needs for auditing the control traffic, it is necessary to mirror the control traffic to obtain the mirrored traffic, and transmit the mirrored traffic to the computer device used to audit the control traffic. In order to improve the transmission efficiency of the mirrored traffic, the present application can compress the header field of the message data of the mirrored traffic and transmit the compressed message data.

[0095] like Figure 1b In the distributed cloud service environment diagram shown in the figure, the deployment room and the public cloud are independent and connected to the customer's local area. Specifically, there are multiple availability zones in the public cloud area, such as Figure 1b Availability Zone 1 (AZ1, Availability Zone 1) and Availability Zone 2 (AZ2, Availability Zone 2) in the distributed cloud scenario; in the distributed cloud scenario, the CDC dedicated cluster deployed at the customer site and the public cloud area share a set of cloud center area management and control, and transmit the control traffic to the cloud center area management and control server in the public cloud area through VPN (VirtualPrivate Network). In addition, the CDC dedicated cluster also mirrors the control traffic to obtain mirrored traffic, and transmits the mirrored traffic to the user Internet Data Center (IDC) used to audit the control traffic to audit the control traffic. In the process of traffic transmission, header compression is achieved by adopting the method provided in this application, thereby saving bandwidth and optimizing transmission performance. After receiving the transmitted message data, the user Internet Data Center (IDC) decompresses the compressed header field in the received message data, thereby restoring the header field of the mirrored traffic.

[0096] CDC stands for Cloud Dedicated Cluster. CDC is a fully managed infrastructure cloud product in a distributed cloud scenario. It deploys the computing, storage, network and other resource facilities of the public cloud to the customer's local computer room in a hardware and software integrated manner to meet the customer's business needs such as data security and low latency. CDC needs to be deployed in the local computer room specified by the customer to meet the security requirement that data does not leave the local computer room. Since CDC's control traffic is independent of the customer's business traffic and shares a set with the central cloud control, and is transmitted between the cloud center and the customer's computer room, the control traffic needs to be mirrored and saved to meet the customer's requirements for local auditing of the control traffic.

[0097] IDC computer room is the Internet Data Center (IDC) computer room, which provides a full range of services such as server hosting, leasing and related value-added services to enterprises and organizations.

[0098] Among them, the control flow is Figure 1c The transmission is performed in the hardware environment shown. Figure 1c First, the control traffic is transmitted between the "VPN Gateway (on the cloud)" and the "VPN Gateway (inside CDC)", which belong to the two ends of the VPN gateway and are used for VPN communication. Among them, the "VPN Gateway (on the cloud)" is deployed in the cloud center area and connected to the public network through the access point of the cloud center area (that is, the "public cloud access point"). The other end of the public network is connected to the "client-side core switch" for communication with the customer's local IDC. The CDC cabinet will be deployed inside the customer's local IDC computer room and connected through the "CDC docking switch". The other end of the "CDC docking switch" is connected to the "VPN Gateway (inside CDC)" for transmitting traffic from or to the "VPN Gateway (on the cloud)". At this point, the interconnection between the cloud center area and the distributed cloud product CDC is realized.

[0099] Among them, by transmitting mirrored traffic between the "VPN Gateway (within CDC)" and the "Traffic Audit Service", the security requirement of local storage of control data can be achieved. Before the mirrored traffic is transmitted, the header field of the mirrored traffic is compressed to save bandwidth and optimize transmission performance.

[0100] The following is a detailed description of the message data processing method of the present application. It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0101] In one embodiment, Figure 1d As shown, a method for processing message data is provided, and the method is applied to Figure 1a The sending device 100 in FIG. 1 is taken as an example to illustrate, and the specific process is as follows:

[0102] 101. Obtain message data to be transmitted and local message header mapping information, where the message data includes message header information and message content, and the message header mapping information includes a mapping relationship between preset message header fields and preset index information.

[0103] Among them, the sending device can obtain the message data, process the message data, and then send it to the receiving device. The message data may include message header information and message content. The message header information contains some control information about the message, such as message type, source address, destination address, protocol version, message length, etc.; the message content is the message body, which contains specific data content, such as text, pictures, audio, video, etc. The message content here can be compressed message content or uncompressed message content, which is not limited in this embodiment.

[0104] Specifically, the message data may include the business data to be sent, which may be the control traffic data or mirror traffic in the above embodiment. The sending device may be the "VPN Gateway (within CDC)" device in the above embodiment; for the control traffic data, the receiving device is the "VPN Gateway (on the cloud)"; for the mirror traffic, the receiving device is the device corresponding to the "traffic audit service". Among them, the control traffic data may include information such as the status of the traffic in the network, the traffic priority setting, and security control; by controlling the traffic data, the performance, stability, and security of the network can be improved, and the normal operation of the business can be guaranteed.

[0105] Among them, the sending device locally stores a copy of the message header mapping information, which is also a dictionary for querying the index information of the message header field. The dictionary can be a static table. Specifically, the message header mapping information includes a mapping relationship between a preset message header field and preset index information. In this embodiment, the message header field can be compressed by the index information to reduce the amount of data transmitted.

[0106] The static table is a predefined set of header fields and their corresponding index values. These header fields can include request headers and response headers, such as: method,: scheme,: authority,: path, content-type, etc.

[0107] Specifically, the local message header mapping information of the sending device may include first message header mapping information and second message header mapping information, the first message header mapping information may be a commonly used static dictionary, and the second message header mapping information may be a less commonly used static dictionary. The commonly used static dictionary may be used to store the most commonly used header names and header key-value pairs, the key in the header key-value pair is the header name, and the value in the header key-value pair is the header value. The less commonly used static dictionary may be used to store an enumeration of header values, such as an enumeration value of a URI header.

[0108] The enumeration value of the URI (Uniform Resource Identifier) ​​header refers to a set of possible values ​​defined in a specific context. These enumeration values ​​are usually used to represent different states or options in the URI header. The specific content of the enumeration value varies depending on the specific URI header and application scenario. For example, in some cases, the enumeration value of the URI header may include:

[0109] a. Success: indicates that an operation or request is completed successfully;

[0110] b. Failure: indicates that an error occurred in the operation or request;

[0111] c. Processing: indicates that a task is currently being processed;

[0112] d. Cancelled: Indicates that the operation or request has been canceled.

[0113] 102. Parse the message header information in the message data to obtain the message header field of the message data.

[0114] Among them, the message header field can be obtained by parsing the message header information. The message header field can include a field name (i.e., header name, Header Name) and a field value (i.e., header value, Header Value). In a specific scenario, information exchanged using the HTTP protocol is called an HTTP message. In the HTTP protocol, the message header field exists in the form of a key-value pair, where the key is the header name and the value is the header value.

[0115] Specifically, some header names are more commonly used, but their values ​​vary depending on the specific request or response. For example, the User-Agent header is used to specify the client's browser type and version, but the value for each client may be different. For another example, for the Content-Type header name, it is used to specify the content type, and its value can be text / html, application / json, etc., where text / html represents an HTML (Hyper Text Markup Language) document and application / json represents JSON data.

[0116] 103. Query the index information of the message header field in the local message header mapping information.

[0117] The message header mapping information may be a relation table between a preset message header field and preset index information. Based on the message header field, the corresponding index information is searched from the relation table.

[0118] Specifically, the local header mapping information of the sending device may include first header mapping information and second header mapping information, the first header mapping information may be a commonly used static dictionary, and the second header mapping information may be a less commonly used static dictionary. The first header mapping information may be a relationship table of commonly used field names, field values, and index information, or a relationship table of commonly used field names and index information; the second header mapping information may be a relationship table of less commonly used field values ​​and index information.

[0119] In a specific scenario, the number of internal services in the traffic audit service is fixed, and HTTP "header information" can be enumerated, so a static dictionary (i.e., the above-mentioned message header mapping information) is used to save HTTP header information data. The static dictionary is mainly divided into a commonly used static dictionary and a second commonly used static dictionary. The commonly used and second commonly used here can be determined based on the frequency of use of the message header field within a preset time period.

[0120] Among them, the commonly used static dictionary is used to store the most commonly used header names and header key-value pairs. Commonly used Header key-value pairs can be directly stored in the commonly used static dictionary and indexed by index. In addition, for commonly used Header names and less commonly used Header values, the names can be stored in the commonly used static dictionary, and the values ​​can be stored in the less commonly used static dictionary. For example, the commonly used header name Host can be stored in the commonly used static dictionary, and the less commonly used value corresponding to Host can be obtained through the less commonly used dictionary index. Among them, Host is used by the client to specify the domain name and port number of the http server to be accessed.

[0121] The commonly used static dictionary structure is shown in Table 1, which mainly includes Index (index information), Header Name (field name), and Header Value (field value):

[0122] Table 1

[0123] Index Header Name Header Value 120 method GET

[0124] Among them, method is the method header, and GET means requesting data from the specified resource.

[0125] The less commonly used static dictionary can be used to store enumerations of header values, such as the enumeration values ​​of URI headers. The structure of the less commonly used static dictionary is shown in Table 2, which contains index (index information) and Header Value (field value):

[0126] Table 2

[0127] Index Header Value 1 / sa_report.cgi

[0128] When the common static dictionary stores only the common header name Header Name, the corresponding second common header value Header Value can be stored in the second common static dictionary. When the common static dictionary stores common key-value pairs, the key is the common header name Header Name, and the value is the corresponding common header value Header Value.

[0129] The overall relationship between the commonly used static dictionary and the less commonly used static dictionary is as follows Figure 1e shown. Figure 1eThe left side is the commonly used static dictionary, and the right side is the less commonly used static dictionary. The capacity of both is 7 bits. Among them, the commonly used static dictionary can include 40 commonly used headers (header names), 80 commonly used key-value pairs (key-value pairs formed by header names and header values) and corresponding index information. The 40 commonly used headers can include method, uri, accept-encoding (acceptable encoding methods), accept-language (acceptable languages), accept-charset (acceptable character sets), host, etc. The 80 commonly used key-value pairs can include method and GET, method and POST, uri and / ma_report.cgi, etc. The less commonly used static dictionary can be commonly used. A set of mapping relationships between the less commonly used header values ​​and index information corresponding to the commonly used headers in the static dictionary. For example, the less commonly used static dictionary may include the less commonly used header values ​​corresponding to the commonly used header name uri (uniform resource identifier) ​​such as / sa_report.cgi, / dpdk_pvgw.cgi and / cvcle_pull, etc. For another example, the less commonly used static dictionary may also include the less commonly used header values ​​corresponding to the commonly used header name host such as yhcq.cbs.apiserver.xyun.com, sh.receiver.barad.xyun.com and receiver.barad.xyun.com, etc.

[0130] 104. When the index information of the message header field is not found in the local message header mapping information, remote message header mapping information is pulled from the server to update the local message header mapping information based on the remote message header mapping information.

[0131] Specifically, the remote header mapping information can be pulled from the remote configuration center module in the server, and the remote header mapping information includes the mapping relationship between the preset header field and the preset index information. In a specific scenario, the remote configuration center module can store the latest header mapping information (dictionary), and it can actively push the updated dictionary configuration to the sending device, and the sending device can also passively pull the latest header mapping information from the remote configuration center module.

[0132] Specifically, the remote message header mapping information may include first remote mapping information and second remote mapping information, the first remote mapping information may be a commonly used static dictionary in the remote configuration center, and the second remote mapping information may be a less commonly used static dictionary in the remote configuration center. After the remote message header mapping information is pulled, the first message header mapping information of the local sending device may be updated based on the first remote mapping information, and the second message header mapping information of the local sending device may be updated based on the second remote mapping information, so that the local dictionary configuration of the sending device is synchronized with the dictionary configuration of the remote configuration center.

[0133] In other embodiments, if index information is found in the local message header mapping information, the target index information corresponding to the found message header field can directly replace the original message header field, and the target index information and message content can be packaged and transmitted.

[0134] 105. Re-query the index information of the message header field in the updated local message header mapping information.

[0135] Specifically, it is possible to query in the updated local message header mapping information whether the field name or field value corresponding to the message header field exists, and if so, query the corresponding index information.

[0136] 106. When the target index information corresponding to the message header field is found, the target index information and the message content are packaged to obtain compressed message data, and the compressed message data is transmitted.

[0137] Among them, the target index information can be used to replace the message header field, so that only the index information needs to be sent instead of the entire field, and the amount of transmitted data can be saved by transmitting the index instead of the full byte data.

[0138] The message content may be compressed content or uncompressed content. If the message content is uncompressed, the message content may be compressed first, and then the compressed message content and the target index information are packaged and transmitted.

[0139] Optionally, in this embodiment, the method for processing the message data may further include:

[0140] When the target index information corresponding to the message header field cannot be found in the updated local message header mapping information, the message header field and the message content are packaged to obtain packaged message data;

[0141] The packaged message data is transmitted.

[0142] Among them, if the target index information corresponding to the message header field cannot be queried, the message header field can be transparently transmitted, and the message header field can be directly packaged and transmitted with the compressed message content, so that the receiving device receives the packaged message data, and then parses the message header field from the packaged message data for storage. Finally, the remote configuration center performs statistical analysis on the message header fields stored in the receiving device, updates the remote message header mapping information according to the statistical analysis results, and then actively pushes the updated remote message header mapping information to the sending device and the receiving device to update the local message header mapping information of the sending device and the receiving device.

[0143] As can be seen from the above, this embodiment can obtain message data to be transmitted and local message header mapping information, the message data including message header information and message content, the message header mapping information including a mapping relationship between a preset message header field and a preset index information; parse the message header information in the message data to obtain the message header field of the message data; query the index information of the message header field in the local message header mapping information; when the index information of the message header field is not found in the local message header mapping information, pull the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information; re-query the index information of the message header field in the updated local message header mapping information; when the target index information corresponding to the message header field is queried, package the target index information and the message content to obtain compressed message data, and transmit the compressed message data.

[0144] The present application can not only use the corresponding index information to compress the message header field through the message header mapping information to reduce the amount of transmitted data; but also the sender has its own message header mapping information locally, and in the compression process, the index information is first searched locally to improve the data compression efficiency. When the corresponding index information cannot be found locally on the sender, the remote message header mapping information saved on the server can be pulled to update the local message header mapping information of the sending device, so as to query the corresponding index information again to achieve compression of the message header field; among them, this method of deploying the configuration update center of the message header mapping information on the server is conducive to better maintaining the sender's message header mapping information.

[0145] In this embodiment, Figure 2a As shown, a method for processing message data is provided, and the method is applied to Figure 1a The receiving device 120 in the example is used as an example to illustrate, and the specific process is as follows:

[0146] 201. Receive compressed message data, where the compressed message data includes message header information and message content.

[0147] Among them, the receiving device receives the compressed message data, and the message header information of the compressed message data can be compressed or uncompressed; and the message content in the compressed message data is generally compressed. The message header information contains some control information about the message, such as message type, source address, destination address, protocol version, message length, etc.; the message content is the message body, which contains specific data content, such as text, pictures, audio, video, etc.

[0148] Specifically, if the sending device does not query the corresponding index information of the message header field, it can directly transparently transmit the message header field to the receiving device, and the message header information is not compressed at this time. If the sending device queries the corresponding index information of the message header field, it can replace the message header field with the index information and send it to the receiving device. At this time, the message header information received by the receiving device is the index information, that is, the compressed message header information.

[0149] 202. Extract the message header information from the compressed message data, and detect the message header information.

[0150] Among them, the message header information can be extracted from the compressed message data according to the protocol format, and then the message header information can be detected whether it has been compressed. If the message header information is detected to be index information, it means that the message header information has been compressed. If the message header information is detected to be the full message header field, it means that the message header information has not been compressed.

[0151] 203. When it is detected that the message header information is target index information after message header compression, query the message header field corresponding to the target index information in local message header mapping information, wherein the message header mapping information includes a mapping relationship between preset message header fields and preset index information.

[0152] Among them, the local message header mapping information of the receiving device may include a mapping relationship between a preset message header field and preset index information, and the preset message header field may include a field name (ie, header name, Header Name) and a field value (ie, header value, Header Value).

[0153] Specifically, the local header mapping information of the receiving device may include third header mapping information and fourth header mapping information. The third header mapping information may be a commonly used static dictionary, and the fourth header mapping information may be a less commonly used static dictionary.

[0154] Among them, the commonly used static dictionary can be used to save the most commonly used header names and header key-value pairs. The key in the header key-value pair is the header name, and the value in the header key-value pair is the header value. Commonly used Header key-value pairs can be directly stored in the common static dictionary and indexed by index. In addition, for commonly used Header names and less commonly used Header values, the names can be stored in the common static dictionary, and the values ​​can be stored in the less commonly used static dictionary. For example, the commonly used header name Host can be stored in the common static dictionary, and the less commonly used value corresponding to Host can be obtained through the less commonly used dictionary index.

[0155] The less commonly used static dictionary can be used to store an enumeration of header values, such as the enumeration of URI header values.

[0156] For example, the third message header mapping information can be a relationship table of commonly used field names, field values ​​and index information, or a relationship table of commonly used field names and index information; the fourth message header mapping information can be a relationship table of less commonly used field values ​​and index information.

[0157] 204. When the message header field corresponding to the target index information is not found in the local message header mapping information, remote message header mapping information is pulled from the server to update the local message header mapping information based on the remote message header mapping information.

[0158] Specifically, the remote header mapping information can be pulled from the remote configuration center module in the server, and the remote header mapping information includes the mapping relationship between the preset header field and the preset index information. In a specific scenario, the remote configuration center module can store the latest header mapping information (dictionary), and it can actively push the updated dictionary configuration to the receiving device, and the receiving device can also passively pull the latest header mapping information from the remote configuration center module.

[0159] Specifically, the remote header mapping information may include first remote mapping information and second remote mapping information, the first remote mapping information may be a commonly used static dictionary in the remote configuration center, and the second remote mapping information may be a less commonly used static dictionary in the remote configuration center. After the remote header mapping information is pulled, the third header mapping information of the local receiving device may be updated based on the first remote mapping information, and the fourth header mapping information of the local receiving device may be updated based on the second remote mapping information, so that the local dictionary configuration of the receiving device is synchronized with the dictionary configuration of the remote configuration center.

[0160] In other embodiments, if the message header field corresponding to the target index information is found in the local message header mapping information, the found message header field can be directly merged with other message contents in the compressed message data for storage.

[0161] 205. Re-query the message header field corresponding to the target index information in the updated local message header mapping information.

[0162] The message header field here may include a field name Header Name and / or a field value HeaderValue.

[0163] 206. When the target message header field corresponding to the target index information is found, the target message header field and the message content in the compressed message data are stored.

[0164] The message content in the compressed message data may be decompressed, and then the decompressed message content and the target message header field are stored.

[0165] Optionally, in this embodiment, the method for processing the message data may further include:

[0166] When it is detected that the message header information is an uncompressed message header field, the uncompressed message header field and the message content in the compressed message data are stored.

[0167] In a specific scenario, Figure 2b As shown in FIG. 1 , it is a flowchart of using a remote configuration center to update the dictionaries of the sender and the receiver, which is described in detail as follows:

[0168] Commonly used static dictionaries are divided into local static dictionaries and remote static dictionaries. Similarly, less commonly used static dictionaries are divided into local less commonly used static dictionaries and remote less commonly used static dictionaries. The local (sender and receiver) dictionary content should be consistent with the remote dictionary content.

[0169] The local commonly used static dictionary is used to index the dictionary locally. For example, the sender can find the index based on the (HeaderName, Header Value) tuple, and the receiver can find the (Header Name, HeaderValue) tuple based on the index. The local less commonly used static dictionary is similar.

[0170] The remote commonly used static dictionary is stored in the remote configuration center and is used to actively push it to the sender and receiver, or wait for the sender and receiver to passively pull the dictionary data. When the sender or receiver cannot find the index locally, it can pull the remote static dictionary to check whether the remote commonly used static dictionary exists. If it exists, it will be updated locally. The remote less commonly used static dictionary is similar.

[0171] In addition, there is a scheduled task dedicated to maintaining and updating the dictionary information of the remote configuration center. When the dictionary information changes, the remote configuration center actively pushes the information and triggers the update of the dictionary information of the sender and the receiver.

[0172] Specifically, reference Figure 2b ,The specific steps of data processing on the sender are as follows:

[0173] 1) First, the sender receives the mirrored traffic;

[0174] 2) Parse the HTTP header information of the mirrored traffic to obtain the Header Name and Header Value;

[0175] 3) According to Header Name / Value (Header Name / Value is key, index is value), search for index in the sender's local common / secondary common static dictionary to determine whether there is a corresponding index in the local common / secondary common static dictionary;

[0176] 4) If it does not exist, pull the remote common / less common static dictionary from the remote configuration center;

[0177] 5) Based on the pulled remote common / secondary common static dictionary, update the local remote common / secondary common static dictionary, and again determine whether the Header Name / Value exists in the updated local common / secondary common static dictionary;

[0178] 6) If it exists, use the corresponding index for compression; if it does not exist, transparently transmit the Header Name / Value;

[0179] 7) After the other data in the packet is compressed, start sending the mirrored traffic.

[0180] Similarly, reference Figure 2b , the specific steps of data processing at the receiving end are as follows:

[0181] 1) First, the receiver receives the compressed mirrored traffic;

[0182] 2) Extract HTTP header information according to the protocol format;

[0183] 3) Pull the commonly used / less commonly used static dictionary from the remote configuration center as needed and update it locally;

[0184] 4) Get a single header information in HTTP;

[0185] 5) Determine whether there is compression based on the header information;

[0186] 6) If the header information is compressed, decompress it according to the index in the common / less common static dictionary to get the HeaderName (header name) / Value (header value);

[0187] 7) If the header information is not compressed, directly store the currently parsed Header Name / Value;

[0188] 8) Other information in the merge package is formatted and stored.

[0189] In addition to the sender and receiver, there are remote configuration centers and background scheduled tasks. The background scheduled tasks can also filter the formatted storage data of the receiving device, perform statistical analysis on the Header Name / Value, and filter out the commonly used / less commonly used dictionary items that can be added / deleted, so as to cope with real-time changes in traffic and achieve a balance between storage space and compression efficiency.

[0190] The present application can save HTTP header compression configuration data (i.e., the above-mentioned dictionary data) by using a remote configuration center, and data consistency is fully guaranteed. The remote configuration center is the only data source, and both the sender and the receiver pull data from the same source, so the data must be consistent. Specifically, the present application uses a remote configuration center to uniformly manage the dictionary data in the message header, centrally configure, and has strong maintainability. In a distributed cloud scenario, the present invention application receives the local customer room mirrored layer 3 traffic and uses HTTP header compression to save bandwidth traffic. During the compression process, a unified remote configuration center is used to save the HTTP header compression configuration data, and the sender and the receiver only need to obtain the corresponding configuration data from the remote configuration center, which is beneficial to maintaining the data of the sender and the receiver. If there is no unified configuration maintenance center, data inconsistency is prone to occur, and data maintainability is very poor. For example, if the configuration data is scattered between the sender and the receiver, if the sender updates its local dictionary data by itself (such as adding or deleting entries in the dictionary), and the receiver does not make corresponding updates to the receiver's local dictionary data, the data between the sender and the receiver will be inconsistent, which will cause the receiver to be unable to decode according to the index, and when there is a problem of being unable to compress / decompress, the problem troubleshooting link is long, the troubleshooting process is complicated, and the system maintenance is difficult. The unified remote configuration center provided by this application is clear in implementation, easy to maintain data, has better maintainability, and can more effectively ensure the consistency of configuration data.

[0191] As can be seen from the above, this embodiment can receive compressed message data, wherein the compressed message data includes message header information and message content; extract the message header information from the compressed message data, and detect the message header information; when it is detected that the message header information is the target index information after the message header is compressed, query the message header field corresponding to the target index information in the local message header mapping information, and the message header mapping information includes the mapping relationship between the preset message header field and the preset index information; when the message header field corresponding to the target index information is not queried in the local message header mapping information, pull the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information; re-query the message header field corresponding to the target index information in the updated local message header mapping information; when the target message header field corresponding to the target index information is queried, store the target message header field and the message content in the compressed message data.

[0192] The present application can compress the message header information through indexing to reduce the amount of transmitted data, and in the process of decompressing the message data, it can first be decompressed through the local message header mapping information of the receiver, which can improve the efficiency of decompression; when the corresponding message header field cannot be queried locally at the receiver, the remote message header mapping information saved on the server can be pulled to update the local message header mapping information of the receiving device, so that the corresponding message header field can be queried again to achieve decompression of the message header information; among them, this method of deploying the configuration update center of the message header mapping information on the server is conducive to better maintaining the message header mapping information of the receiver.

[0193] In this embodiment, Figure 3a As shown, a method for processing message data is provided, and the method is applied to Figure 1a The server 110 in the example is used for explanation, and the specific process is as follows:

[0194] 301. Obtain target message header data stored in a receiving device, where the target message header data includes multiple message header fields.

[0195] After receiving the message data each time, the receiving device may parse the message header information in the message data to obtain the message header fields therein, and then store the message header fields. The message header fields may include a field name and a field value.

[0196] The target message header data of this embodiment may be a message header field stored by the receiving device within a preset time period, and the preset time period may be set according to actual conditions, such as one day. Specifically, the server may obtain the target message header data in the receiving device at every preset time period to perform statistical analysis on the target message header data, thereby updating the remote message header mapping information according to the statistical analysis results, and actively push the updated remote message header mapping information to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information based on the updated remote message header mapping information.

[0197] 302. Determine a newly added message header field and a non-newly added message header field from the target message header data based on remote message header mapping information, wherein the remote message header mapping information includes a mapping relationship between preset message header fields and preset index information.

[0198] Among them, the remote message header mapping information can be stored in the remote configuration center, and the remote message header mapping information includes the mapping relationship between the preset message header field and the preset index information. In a specific scenario, the remote configuration center module can store the latest message header mapping information (dictionary), which can actively push the updated dictionary configuration to the sending device and the receiving device, and the sending device and the receiving device can also passively pull the latest message header mapping information from the remote configuration center module.

[0199] Specifically, the remote message header mapping information may include first remote mapping information and second remote mapping information, the first remote mapping information may be a commonly used static dictionary in the remote configuration center, and the second remote mapping information may be a second commonly used static dictionary in the remote configuration center. When the commonly used static dictionary in the remote configuration center is updated, it may trigger the synchronization update of the updated commonly used static dictionary content in the remote configuration center to the commonly used static dictionary local to the sending device and the receiving device; similarly, when the second commonly used static dictionary in the remote configuration center is updated, it may trigger the synchronization update of the updated second commonly used static dictionary content in the remote configuration center to the second commonly used static dictionary local to the sending device and the receiving device; the synchronization update here includes content addition and content deletion.

[0200] Optionally, in this embodiment, the step of “determining the newly added message header fields and the non-newly added message header fields from the target message header data based on the remote message header mapping information” may include:

[0201] For each message header field in the target message header data, detecting whether the message header field exists in the remote message header mapping information;

[0202] If it is detected that the message header field exists in the remote message header mapping information, determining that the message header field is a non-newly added message header field;

[0203] If it is detected that the message header field does not exist in the remote message header mapping information, it is determined that the message header field is a newly added message header field.

[0204] 303. Count the frequency information of the corresponding occurrence of each non-newly added message header field in the target message header data.

[0205] 304. Update remote message header mapping information according to the frequency information and the newly added message header field.

[0206] Optionally, in this embodiment, the message header field includes a field name and a field value; the step of “determining the newly added message header field and the non-newly added message header field from the target message header data based on the remote message header mapping information” may include:

[0207] Based on the field names in the current remote message header mapping information, filter out the newly added field names and non-newly added field names from the target message header data;

[0208] The step of “counting the frequency information of each non-newly added message header field in the target message header data” may include:

[0209] In the target message header data, statistics are collected on each field value corresponding to the non-newly added field name to obtain frequency information of occurrence of each field value corresponding to the non-newly added field name;

[0210] The step of “updating the remote message header mapping information according to the frequency information and the newly added message header field” may include:

[0211] Adding the newly added field name and its corresponding index information to the remote message header mapping information;

[0212] The added remote message header mapping information is updated according to the frequency information of each field value corresponding to the non-newly added field name.

[0213] The message header field may include a field name (ie, header name, Header Name) and a field value (ie, header value, Header Value).

[0214] In this embodiment, the newly added message header field may specifically refer to the newly added field name, and the non-newly added message header field may refer to the non-newly added field name. If the field name in the target message header data does not appear in the current remote message header mapping information, then the field name is the newly added field name; if the field name in the target message header data appears in the current remote message header mapping information, then the field name is the non-newly added field name.

[0215] For a newly added field name, a mapping relationship between the newly added field name and index information may be established, and the mapping relationship may be added to the remote message header mapping information.

[0216] For non-newly added field names, statistics are collected on the field values ​​corresponding to the non-newly added field names in the target message header data, and the frequency of occurrence of each corresponding field value in the target message header data is counted. Based on the frequency of occurrence, the added remote message header mapping information is updated, and the update method can be: adding and deleting field values ​​of the added remote message header mapping information.

[0217] Optionally, in this embodiment, the step of “updating the added remote message header mapping information according to the frequency information of each field value corresponding to the non-newly added field name” may include:

[0218] When the frequency information of the field value corresponding to the non-newly added field name is greater than the first preset frequency information, updating the field value corresponding to the non-newly added field name to the added remote message header mapping information;

[0219] When the frequency information of the field value corresponding to the non-newly added field name is not greater than the first preset frequency information, and when it is detected that the field value exists in the added remote message header mapping information, the field value is deleted from the added remote message header mapping information.

[0220] Among them, the first preset frequency information can be set according to actual conditions, such as being set to 2. When the first preset frequency information is set to 2, if the number of times a certain field value of a non-newly added field name appears in the target message header data is greater than a threshold value of 2, and the field value does not exist in the remote message header mapping information, the field value can be added to the remote message header mapping information; if the number of times a certain field value of a non-newly added field name appears in the target message header data is not greater than 2, it means that the field value is an uncommon field value, and if it is detected that the field value exists in the remote message header mapping information, the entry corresponding to the field value is deleted from the remote message header mapping information, so that the remote configuration center has enough space to accommodate the new dictionary entry.

[0221] Optionally, in this embodiment, the remote message header mapping information includes first remote mapping information and second remote mapping information; the step of “adding the newly added field name and its corresponding index information to the remote message header mapping information” may include:

[0222] Add the newly added field name and its corresponding index information to the first remote mapping information;

[0223] The step of “updating the field value corresponding to the non-newly added field name to the added remote message header mapping information when the frequency information of the field value corresponding to the non-newly added field name is greater than the first preset frequency information” may include:

[0224] When the frequency information of the field value corresponding to the non-newly added field name is greater than the second preset frequency information, the non-newly added field name, the field value of the non-newly added field name and the corresponding index information are updated to the first remote mapping information after the addition, and the first preset frequency information is less than the second preset frequency information;

[0225] When the frequency information of the field value corresponding to the non-newly added field name is not greater than the second preset frequency information but greater than the first preset frequency information, the field value corresponding to the non-newly added field name and its corresponding index information are added to the second remote mapping information.

[0226] Specifically, the first remote mapping information may be a common static dictionary in the remote configuration center, and the second remote mapping information may be a less common static dictionary in the remote configuration center. In this embodiment, the newly added field name and its corresponding index information may be added to the common static dictionary of the remote configuration center.

[0227] In a specific embodiment, the first preset frequency information can be set to 2, and the second preset frequency information can be set to 5. Specifically, if the number of times a field value of a non-newly added field name appears in the target message header data is greater than a threshold value of 5, it means that the field value is a commonly used header field value, and it is detected that there is no corresponding entry for the field value in the commonly used static dictionary of the remote configuration center, then a mapping relationship between the non-newly added field name, the field value, and the index information can be established (the non-newly added field name is the key in the key-value pair, and the field value is the value in the key-value pair), and the mapping relationship is added to the commonly used static dictionary. If the number of times a field value of a non-newly added field name appears in the target message header data is greater than a threshold value of 2, but not greater than a threshold value of 5, it means that the field value is a less commonly used header field value, then a mapping relationship between the field value and the index information is established, and the mapping relationship is added to the less commonly used static dictionary.

[0228] 305. Trigger sending the updated remote message header mapping information to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information.

[0229] Specifically, if only the first remote mapping information is updated and the second remote mapping information is not updated, only the updated first remote mapping information can be sent to the sending device and the receiving device to trigger the sending device and the receiving device to update their local commonly used static dictionaries. Similarly, if only the second remote mapping information is updated and the first remote mapping information is not updated, only the updated second remote mapping information can be sent to the sending device and the receiving device to trigger the sending device and the receiving device to update their local second commonly used static dictionaries. If both the first remote mapping information and the second remote mapping information in the remote configuration center are updated, the first remote mapping information and the second remote mapping information are synchronously updated to the sending device and the receiving device, so as to update the local commonly used static dictionaries of the sending device and the receiving device based on the first remote mapping information, and update the local second commonly used static dictionaries of the sending device and the receiving device based on the second remote mapping information.

[0230] Optionally, in this embodiment, the method for processing the message data may further include:

[0231] Receiving pull request information sent by a target device, wherein the pull request information indicates to pull remote message header mapping information, and the target device is a sending device or a receiving device;

[0232] Based on the pull request information, the current remote message header mapping information is sent to the target device, so that the target device updates the local message header mapping information of the target device based on the current remote message header mapping information.

[0233] In a specific scenario, the common / less common static dictionary of the remote configuration center will have an initial value when the service is first deployed. The initial value is a dictionary data configured according to the specific business; and the data in the dictionary is continuously updated as the traffic audit service runs. The components used to update the remote configuration center are Figure 3b The scheduled task center in .

[0234] This scheduled task is mainly used to dynamically update the dictionary data of the remote configuration center in real time. The background scheduled task obtains the HTTP header statistics within a certain time range by performing statistical analysis on the logs (specifically the target message header data) stored in the traffic audit service (specifically the above-mentioned receiving device). For these HTTP header statistics, the Header Name is first analyzed. When the Header Name is a newly added field name, it will be directly saved in the commonly used static dictionary. Secondly, the threshold analysis is performed based on the Header Value, and different processing measures are taken according to the threshold, so as to achieve the purpose of dynamically updating the dictionary data.

[0235] Specifically, refer to Figure 3b The process of dynamic maintenance and update of the remote configuration center dictionary by the background scheduled task center is as follows:

[0236] A. First, the scheduled task will filter the formatted traffic data saved by the receiver;

[0237] B. Then, the HTTP header information in the traffic data will be statistically analyzed;

[0238] C. Determine whether the Header Name is a newly added field name. If so, it will be directly updated to the common static dictionary of the remote configuration center. Because the Header Name in the traffic audit service is limited, the common static dictionary is sufficient to save all Header Name header information. At this time, the remote configuration center will be triggered to actively push the updated common static dictionary data to the sender and receiver;

[0239] D. If the Header Name is not a newly added field name, count the field value Value corresponding to the Header Name;

[0240] E. Determine the frequency of the current Value. If it is greater than the threshold of 5, it means that it is a frequently appearing HTTP header information, so the Value is merged with its Header Name information and updated together to the common static dictionary. At this time, the remote configuration center will be triggered to actively push the updated common static dictionary data to the sender and receiver (note that the threshold of 5 is configurable);

[0241] F. If it is greater than threshold 2, and the Value does not exist in the second most frequently used static dictionary, then the Value is added to the second most frequently used static dictionary. This will trigger the remote configuration center to actively push the updated second most frequently used static dictionary data to the sender and receiver (note that threshold 2 is configurable);

[0242] G. If it is less than or equal to the threshold 2, and the Value exists in the second most frequently used static dictionary, then the Value is removed from the second most frequently used static dictionary. This will trigger the remote configuration center to actively push the updated second most frequently used static dictionary data to the sender and receiver.

[0243] It should be noted that each time a scheduled task updates the common / secondary common static dictionary, it will trigger the remote configuration center to actively push the dictionary data to the sender and receiver. This centralized remote configuration center ensures the unique reliability of the data source, and both the sender and receiver pull from it, with a very low probability of data inconsistency.

[0244] In some embodiments, the sender can also update the local common / less common static dictionary based on the parsed HTTP header information, and then push it to the remote configuration center. This may cause a concurrency conflict with the update of the scheduled task, and a concurrent lock is needed to control it. Figure 3c As shown in the figure, when the sender cannot find data in the local common / secondary common static dictionary, it will first pull it from the remote configuration center. When the data is still not found after pulling, it will determine whether to update the common / secondary common static dictionary according to the local rules. If it needs to be updated, an index will be generated according to a specific algorithm, and the data will be pushed to the remote configuration center while being compressed. Therefore, the sender will not only pull the data from the remote configuration center, but also update the data from the remote configuration center, which will cause a concurrency conflict with the update of the remote configuration center data in the scheduled task. At this time, concurrency control can be performed by adding a mutex concurrency lock. However, concurrency control may cause problems with the sender's packet processing speed. At present, the qps (Queries Per Second) of the traffic audit service is up to 4w / s, but the computing resources are 4 cores and 8G. In the face of extremely demanding computing resources, if concurrency locks are used for concurrency control, it will affect the concurrent processing in the service, and may even be unable to cope with peak traffic.

[0245] The message data processing method provided in this application can also control the update uniformly in the scheduled task, and the scheduled task updates the dictionary data of the remote configuration center according to the statistical data. The previous operation of the sender to update the dictionary data is abandoned, which can reduce the performance problems caused by concurrent lock control and improve the packet processing speed of the traffic audit service. The overall schematic diagram is as follows Figure 3d As shown. Figure 3d It can be seen that the update operation (addition / deletion) of the remote configuration center is uniformly performed by a scheduled task. The scheduled task updates the static common / less common dictionary according to a specific strategy and pushes it to the remote configuration center.

[0246] The sender and receiver will obtain the dictionary configuration from the remote configuration center respectively. There are two ways to obtain the dictionary configuration: one is passive pull, that is, when the sender / receiver needs it, the latest dictionary configuration information is pulled from the remote configuration center on demand; the other is active push, that is, when the scheduled task updates the remote configuration center, a callback will be triggered for the remote configuration center to actively update the dictionary configuration to the sender and receiver.

[0247] At this time, there may be problems with concurrent data processing. That is, when the scheduled task pushes the latest dictionary configuration to the remote configuration center, the sender pulls the latest remote configuration center data. At this time, the data updated by the scheduled task cannot be pulled, and data inconsistency will occur. However, this situation can be avoided by pulling multiple times, and no mutex control is required, so the concurrency performance is better, and the concurrency of the traffic audit service is not affected, achieving better service performance.

[0248] This application can use a unified remote configuration center to save HTTP header compression configuration data in a distributed cloud scenario to improve data maintainability. Specifically, in a distributed cloud scenario, this application uses HTTP header compression to save bandwidth traffic by receiving the local customer room mirrored layer 3 traffic. During the compression process, a remote configuration center is used to save HTTP header compression configuration data, which is beneficial for maintaining the data of the sender and receiver. The sender and receiver only need to obtain the corresponding configuration data from the remote configuration center without the need for the sender to trigger the receiver to add new entries. This processing method is clear and easy to maintain data.

[0249] As can be seen from the above, the present embodiment can obtain the target message header data stored in the receiving device, wherein the target message header data includes multiple message header fields; based on the remote message header mapping information, the newly added message header fields and the non-newly added message header fields are determined from the target message header data, wherein the remote message header mapping information includes a mapping relationship between preset message header fields and preset index information; the frequency information of the corresponding occurrence of each non-newly added message header field in the target message header data is counted; according to the frequency information and the newly added message header field, the remote message header mapping information is updated; and the updated remote message header mapping information is triggered to be sent to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information.

[0250] The present application can obtain the target message header data in the receiving device, and then perform statistical analysis on the target message header data to update the remote message header mapping information according to the statistical analysis results, thereby improving the integrity of the compression configuration data of the message header field stored in the remote message header mapping information; moreover, after the remote message header mapping information is updated, the updated data is actively pushed to the sender and the receiver to trigger the local data update of the sender and the receiver, so that through a unified data configuration center, it is beneficial to maintain the data of the sender and the receiver, and the data consistency of the sender and the receiver can be more effectively guaranteed.

[0251] In order to better implement the above method, the embodiment of the present application also provides a message data processing system. The message data processing system may include a first message data processing device 41, a second message data processing device 42, and a third message data processing device 43. Figure 4a As shown, the first message data processing device 41 may include an acquisition unit 4101, a parsing unit 4102, a first query unit 4103, a first pull unit 4104, a second query unit 4105 and a transmission unit 4106; Figure 4b As shown, the second message data processing device 42 may include a receiving unit 4201, an extracting unit 4202, a third query unit 4203, a second pulling unit 4204, a fourth query unit 4205, and a storage unit 4206; Figure 4c As shown, the third message data processing device 43 may include a data acquisition unit 4301, a determination unit 4302, a statistics unit 4303, a first updating unit 4304, and a second updating unit 4305, as follows:

[0252] A. First message data processing device 41

[0253] (1) Acquisition unit 4101;

[0254] The acquisition unit is used to acquire the message data to be transmitted and the local message header mapping information, wherein the message data includes the message header information and the message content, and the message header mapping information includes the mapping relationship between the preset message header field and the preset index information.

[0255] (2) parsing unit 4102;

[0256] The parsing unit is used to parse the message header information in the message data to obtain the message header field of the message data.

[0257] (3) a first query unit 4103;

[0258] The first query unit is used to query the index information of the message header field in the local message header mapping information.

[0259] (4) a first pulling unit 4104;

[0260] The first pulling unit is used to pull the remote header mapping information from the server when the index information of the header field is not found in the local header mapping information, so as to update the local header mapping information based on the remote header mapping information.

[0261] (5) second query unit 4105;

[0262] The second query unit is used to re-query the index information of the message header field in the updated local message header mapping information.

[0263] (6) Transmission unit 4106;

[0264] The transmission unit is used to package the target index information and the message content to obtain compressed message data and transmit the compressed message data when the target index information corresponding to the message header field is queried.

[0265] Optionally, in some embodiments of the present application, the message data processing device further includes a data transmission unit, as follows:

[0266] The data transmission unit is used to package the message header field and the message content to obtain packaged message data when the target index information corresponding to the message header field cannot be found in the updated local message header mapping information; and transmit the packaged message data.

[0267] B. Second message data processing device 42

[0268] (7) receiving unit 4201;

[0269] The receiving unit is used to receive compressed message data, wherein the compressed message data includes message header information and message content.

[0270] (8) Extraction unit 4202;

[0271] An extraction unit is used to extract the message header information from the compressed message data and detect the message header information.

[0272] (9) third query unit 4203;

[0273] The third query unit is used to query the message header field corresponding to the target index information in the local message header mapping information when it is detected that the message header information is the target index information after message header compression. The message header mapping information includes a mapping relationship between a preset message header field and a preset index information.

[0274] (10) a second pulling unit 4204;

[0275] The second pulling unit is used to pull remote header mapping information from the server when the header field corresponding to the target index information is not found in the local header mapping information, so as to update the local header mapping information based on the remote header mapping information.

[0276] (11) fourth query unit 4205;

[0277] The fourth query unit is used to re-query the message header field corresponding to the target index information in the updated local message header mapping information.

[0278] (12) storage unit 4206;

[0279] A storage unit is used to store the target message header field and the message content in the compressed message data when the target message header field corresponding to the target index information is queried.

[0280] Optionally, in some embodiments of the present application, the message data processing device further includes a data storage unit, as follows:

[0281] The data storage unit is used to store the uncompressed message header field and the message content in the compressed message data when it is detected that the message header information is an uncompressed message header field.

[0282] C. Third message data processing device 43

[0283] (13) data acquisition unit 4301;

[0284] The data acquisition unit is used to acquire target message header data stored in the receiving device, wherein the target message header data includes a plurality of message header fields.

[0285] (14) determining unit 4302;

[0286] A determination unit is used to determine the newly added message header fields and non-newly added message header fields from the target message header data based on the remote message header mapping information, wherein the remote message header mapping information includes a mapping relationship between preset message header fields and preset index information.

[0287] (15) Statistics unit 4303;

[0288] The statistical unit is used to count the frequency information of the corresponding occurrence of each non-newly added message header field in the target message header data.

[0289] (16) first update unit 4304;

[0290] The first updating unit is used to update the remote message header mapping information according to the frequency information and the newly added message header field.

[0291] (17) second updating unit 4305;

[0292] The second updating unit is used to trigger the sending of updated remote message header mapping information to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information.

[0293] Optionally, in some embodiments of the present application, the message header field includes a field name and a field value; the determination unit may be specifically configured to filter out the newly added field name and the non-newly added field name from the target message header data based on the field name in the current remote message header mapping information;

[0294] The statistical unit may be specifically configured to count each field value corresponding to the non-newly added field name in the target message header data to obtain the frequency information of occurrence of each field value corresponding to the non-newly added field name;

[0295] The first updating unit may include an adding subunit and an updating subunit, as follows:

[0296] The adding subunit is used to add the newly added field name and its corresponding index information to the remote message header mapping information;

[0297] The updating subunit is used to update the added remote message header mapping information according to the frequency information of each field value corresponding to the non-newly added field name.

[0298] Optionally, in some embodiments of the present application, the update sub-unit can be specifically used to update the field value corresponding to the non-newly added field name to the remote message header mapping information after being added when the frequency information of the field value corresponding to the non-newly added field name is greater than the first preset frequency information; when the frequency information of the field value corresponding to the non-newly added field name is not greater than the first preset frequency information, and when it is detected that the field value exists in the remote message header mapping information after being added, delete the field value from the remote message header mapping information after being added.

[0299] Optionally, in some embodiments of the present application, the remote message header mapping information includes first remote mapping information and second remote mapping information;

[0300] The adding subunit may be specifically used to add the newly added field name and its corresponding index information to the first remote mapping information;

[0301] The step of “updating the field value corresponding to the non-newly added field name to the added remote message header mapping information when the frequency information of the field value corresponding to the non-newly added field name is greater than the first preset frequency information” may include:

[0302] When the frequency information of the field value corresponding to the non-newly added field name is greater than the second preset frequency information, the non-newly added field name, the field value of the non-newly added field name and the corresponding index information are updated to the first remote mapping information after the addition, and the first preset frequency information is less than the second preset frequency information;

[0303] When the frequency information of the field value corresponding to the non-newly added field name is not greater than the second preset frequency information but greater than the first preset frequency information, the field value corresponding to the non-newly added field name and its corresponding index information are added to the second remote mapping information.

[0304] Optionally, in some embodiments of the present application, the message data processing method device may include a request update unit as follows:

[0305] The request update unit is used to receive pull request information sent by a target device, where the pull request information indicates pulling remote message header mapping information, and the target device is a sending device or a receiving device; based on the pull request information, the current remote message header mapping information is sent to the target device, so that the target device updates the local message header mapping information of the target device based on the current remote message header mapping information.

[0306] As can be seen from the above, in this embodiment, the acquisition unit 4101 can acquire the message data to be transmitted and the local message header mapping information, wherein the message data includes message header information and message content, and the message header mapping information includes a mapping relationship between a preset message header field and a preset index information; the parsing unit 4102 parses the message header information in the message data to obtain the message header field of the message data; the first query unit 4103 queries the index information of the message header field in the local message header mapping information; when the local message header mapping information is not found, the local message header mapping information is searched; When the index information of the message header field is queried, the first pulling unit 4104 pulls the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information; the second query unit 4105 re-queries the index information of the message header field in the updated local message header mapping information; when the target index information corresponding to the message header field is queried, the transmission unit 4106 packages the target index information and the message content to obtain compressed message data, and transmits the compressed message data. The present application can not only use the corresponding index information to compress the message header field through the message header mapping information to reduce the amount of transmitted data; but also the sender has its own message header mapping information locally, and in the compression process, the index information is first searched locally to improve the data compression efficiency. When the corresponding index information cannot be found locally on the sender, the remote message header mapping information saved on the server can be pulled to update the local message header mapping information of the sending device, so as to query the corresponding index information again to achieve compression of the message header field; wherein, this method of deploying the configuration update center of the message header mapping information on the server is conducive to better maintenance of the sender's message header mapping information;

[0307] Alternatively, the receiving unit 4201 may receive compressed message data, wherein the compressed message data includes message header information and message content; the extraction unit 4202 may extract the message header information from the compressed message data, and detect the message header information; when it is detected that the message header information is target index information after message header compression, the third query unit 4203 may query the message header field corresponding to the target index information in the local message header mapping information, wherein the message header mapping information includes a mapping relationship between a preset message header field and a preset index information; when the message header information is not found in the local When the message header field corresponding to the target index information is queried in the local message header mapping information, the remote message header mapping information is pulled from the server through the second pulling unit 4204 to update the local message header mapping information based on the remote message header mapping information; the message header field corresponding to the target index information is re-queried in the updated local message header mapping information through the fourth query unit 4205; when the target message header field corresponding to the target index information is queried, the storage unit 4206 stores the target message header field and the message content in the compressed message data. The present application can compress the message header information through indexing to reduce the amount of transmitted data, and in the process of decompressing the message data, it can first be decompressed through the local message header mapping information of the receiver, which can improve the efficiency of decompression; when the corresponding message header field cannot be queried locally at the receiver, the remote message header mapping information saved in the server can be pulled to update the local message header mapping information of the receiving device, so as to query the corresponding message header field again to achieve decompression of the message header information; wherein, this method of deploying the configuration update center of the message header mapping information on the server is conducive to better maintenance of the message header mapping information of the receiver;

[0308] Alternatively, the data acquisition unit 4301 can acquire the target message header data stored in the receiving device, and the target message header data includes multiple message header fields; the determination unit 4302 determines the newly added message header fields and non-new message header fields from the target message header data based on the remote message header mapping information, and the remote message header mapping information includes a mapping relationship between preset message header fields and preset index information; the statistical unit 4303 counts the frequency information of the corresponding occurrence of each non-new message header field in the target message header data; the first update unit 4304 updates the remote message header mapping information according to the frequency information and the newly added message header fields; and the second update unit 4305 triggers the sending of the updated remote message header mapping information to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information. The present application can obtain the target message header data in the receiving device, and then perform statistical analysis on the target message header data to update the remote message header mapping information according to the statistical analysis results, thereby improving the integrity of the compression configuration data of the message header field stored in the remote message header mapping information; moreover, after the remote message header mapping information is updated, the updated data is actively pushed to the sender and the receiver to trigger the local data update of the sender and the receiver, so that through a unified data configuration center, it is beneficial to maintain the data of the sender and the receiver, and the data consistency of the sender and the receiver can be more effectively guaranteed.

[0309] The present application also provides an electronic device, such as Figure 5 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present application, and the electronic device may be a terminal or a server, etc. Specifically:

[0310] The electronic device may include components such as a processor 501 with one or more processing cores, a memory 502 with one or more computer-readable storage media, a power supply 503, and an input unit 504. Those skilled in the art will appreciate that Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0311] The processor 501 is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 502, and calling data stored in the memory 502, the processor 501 performs various functions of the electronic device and processes data. Optionally, the processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 501.

[0312] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502. The memory 502 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0313] The electronic device also includes a power supply 503 for supplying power to each component. Preferably, the power supply 503 can be logically connected to the processor 501 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 503 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0314] The electronic device may further include an input unit 504, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0315] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 501 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 502 according to the following instructions, and the processor 501 will run the application programs stored in the memory 502, thereby realizing various functions, as follows:

[0316] Acquire message data to be transmitted and local message header mapping information, wherein the message data includes message header information and message content, and the message header mapping information includes a mapping relationship between a preset message header field and a preset index information; parse the message header information in the message data to obtain the message header field of the message data; query the index information of the message header field in the local message header mapping information; when the index information of the message header field is not found in the local message header mapping information, pull the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information; re-query the index information of the message header field in the updated local message header mapping information; when the target index information corresponding to the message header field is found, package the target index information and the message content to obtain compressed message data, and transmit the compressed message data;

[0317] Alternatively, receiving compressed message data, the compressed message data including message header information and message content; extracting the message header information from the compressed message data, and detecting the message header information; when it is detected that the message header information is the target index information after message header compression, querying the message header field corresponding to the target index information in the local message header mapping information, the message header mapping information including the mapping relationship between the preset message header field and the preset index information; when the message header field corresponding to the target index information is not found in the local message header mapping information, pulling the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information; re-querying the message header field corresponding to the target index information in the updated local message header mapping information; when the target message header field corresponding to the target index information is found, storing the target message header field and the message content in the compressed message data;

[0318] Alternatively, target message header data stored in a receiving device is obtained, the target message header data including a plurality of message header fields; based on remote message header mapping information, newly added message header fields and non-newly added message header fields are determined from the target message header data, the remote message header mapping information including a mapping relationship between preset message header fields and preset index information; frequency information of corresponding occurrences of each non-newly added message header field in the target message header data is counted; based on the frequency information and the newly added message header field, the remote message header mapping information is updated; and the updated remote message header mapping information is triggered to be sent to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information.

[0319] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0320] As can be seen from the above, this embodiment can obtain message data to be transmitted and local message header mapping information, the message data including message header information and message content, the message header mapping information including a mapping relationship between a preset message header field and a preset index information; parse the message header information in the message data to obtain the message header field of the message data; query the index information of the message header field in the local message header mapping information; when the index information of the message header field is not found in the local message header mapping information, pull the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information; re-query the index information of the message header field in the updated local message header mapping information; when the target index information corresponding to the message header field is queried, package the target index information and the message content to obtain compressed message data, and transmit the compressed message data. The present application can not only use the corresponding index information to compress the message header field through the message header mapping information to reduce the amount of transmitted data; but also the sender has its own message header mapping information locally, and in the compression process, the index information is first searched locally to improve the data compression efficiency. When the corresponding index information cannot be found locally on the sender, the remote message header mapping information saved on the server can be pulled to update the local message header mapping information of the sending device, so as to query the corresponding index information again to achieve compression of the message header field; wherein, this method of deploying the configuration update center of the message header mapping information on the server is conducive to better maintenance of the sender's message header mapping information;

[0321] Alternatively, compressed message data can be received, wherein the compressed message data includes message header information and message content; the message header information is extracted from the compressed message data, and the message header information is detected; when it is detected that the message header information is the target index information after the message header is compressed, the message header field corresponding to the target index information is queried in the local message header mapping information, and the message header mapping information includes a mapping relationship between a preset message header field and a preset index information; when the message header field corresponding to the target index information is not queried in the local message header mapping information, the remote message header mapping information is pulled from the server to update the local message header mapping information based on the remote message header mapping information; the message header field corresponding to the target index information is re-queried in the updated local message header mapping information; when the target message header field corresponding to the target index information is queried, the target message header field and the message content in the compressed message data are stored. The present application can compress the message header information through indexing to reduce the amount of transmitted data, and in the process of decompressing the message data, it can first be decompressed through the local message header mapping information of the receiver, which can improve the efficiency of decompression; when the corresponding message header field cannot be queried locally at the receiver, the remote message header mapping information saved in the server can be pulled to update the local message header mapping information of the receiving device, so as to query the corresponding message header field again to achieve decompression of the message header information; wherein, this method of deploying the configuration update center of the message header mapping information on the server is conducive to better maintenance of the message header mapping information of the receiver;

[0322] Alternatively, the target message header data stored in the receiving device can be obtained, the target message header data including multiple message header fields; based on the remote message header mapping information, the newly added message header fields and the non-newly added message header fields are determined from the target message header data, the remote message header mapping information including the mapping relationship between the preset message header fields and the preset index information; the frequency information of the corresponding occurrence of each non-newly added message header field in the target message header data is counted; according to the frequency information and the newly added message header field, the remote message header mapping information is updated; and the updated remote message header mapping information is triggered to be sent to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information. The present application can obtain the target message header data in the receiving device, and then perform statistical analysis on the target message header data to update the remote message header mapping information according to the statistical analysis results, thereby improving the integrity of the compression configuration data of the message header field stored in the remote message header mapping information; moreover, after the remote message header mapping information is updated, the updated data is actively pushed to the sender and the receiver to trigger the local data update of the sender and the receiver, so that through a unified data configuration center, it is beneficial to maintain the data of the sender and the receiver, and the data consistency of the sender and the receiver can be more effectively guaranteed.

[0323] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0324] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any one of the message data processing methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0325] Acquire message data to be transmitted and local message header mapping information, wherein the message data includes message header information and message content, and the message header mapping information includes a mapping relationship between a preset message header field and a preset index information; parse the message header information in the message data to obtain the message header field of the message data; query the index information of the message header field in the local message header mapping information; when the index information of the message header field is not found in the local message header mapping information, pull the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information; re-query the index information of the message header field in the updated local message header mapping information; when the target index information corresponding to the message header field is found, package the target index information and the message content to obtain compressed message data, and transmit the compressed message data;

[0326] Alternatively, receiving compressed message data, the compressed message data including message header information and message content; extracting the message header information from the compressed message data, and detecting the message header information; when it is detected that the message header information is the target index information after message header compression, querying the message header field corresponding to the target index information in the local message header mapping information, the message header mapping information including the mapping relationship between the preset message header field and the preset index information; when the message header field corresponding to the target index information is not found in the local message header mapping information, pulling the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information; re-querying the message header field corresponding to the target index information in the updated local message header mapping information; when the target message header field corresponding to the target index information is found, storing the target message header field and the message content in the compressed message data;

[0327] Alternatively, target message header data stored in a receiving device is obtained, the target message header data including a plurality of message header fields; based on remote message header mapping information, newly added message header fields and non-newly added message header fields are determined from the target message header data, the remote message header mapping information including a mapping relationship between preset message header fields and preset index information; frequency information of corresponding occurrences of each non-newly added message header field in the target message header data is counted; based on the frequency information and the newly added message header field, the remote message header mapping information is updated; and the updated remote message header mapping information is triggered to be sent to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information.

[0328] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0329] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0330] Since the instructions stored in the computer-readable storage medium can execute the steps in any message data processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any message data processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0331] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementations of the above-mentioned message data processing.

[0332] The above is a detailed introduction to a method for processing message data and related equipment provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for processing message data, characterized in that: Applicable to servers, including: Acquire target message header data stored in a receiving device, wherein the target message header data includes a plurality of message header fields, and the message header fields include a field name and a field value; Based on the field names in the current remote message header mapping information, newly added field names and non-newly added field names are screened out from the target message header data, wherein the remote message header mapping information includes a mapping relationship between preset message header fields and preset index information; In the target message header data, statistics are collected on each field value corresponding to the non-newly added field name to obtain frequency information of occurrence of each field value corresponding to the non-newly added field name; Adding the newly added field name and its corresponding index information to the remote message header mapping information; According to the frequency information of each field value corresponding to the non-newly added field name, the added remote message header mapping information is updated; The updated remote message header mapping information is sent to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information.

2. The method according to claim 1, characterized in that The updating of the added remote message header mapping information according to the frequency information of each field value corresponding to the non-newly added field name includes: When the frequency information of the field value corresponding to the non-newly added field name is greater than the first preset frequency information, updating the field value corresponding to the non-newly added field name to the added remote message header mapping information; When the frequency information of the field value corresponding to the non-newly added field name is not greater than the first preset frequency information, and when it is detected that the field value exists in the added remote message header mapping information, the field value is deleted from the added remote message header mapping information.

3. The method according to claim 2, characterized in that The remote message header mapping information includes first remote mapping information and second remote mapping information; and adding the newly added field name and its corresponding index information to the remote message header mapping information includes: Add the newly added field name and its corresponding index information to the first remote mapping information; When the frequency information of the field value corresponding to the non-newly added field name is greater than the first preset frequency information, updating the field value corresponding to the non-newly added field name to the added remote message header mapping information includes: When the frequency information of the field value corresponding to the non-newly added field name is greater than the second preset frequency information, the non-newly added field name, the field value of the non-newly added field name and the corresponding index information are updated to the first remote mapping information after the addition, and the first preset frequency information is less than the second preset frequency information; When the frequency information of the field value corresponding to the non-newly added field name is not greater than the second preset frequency information but greater than the first preset frequency information, the field value corresponding to the non-newly added field name and its corresponding index information are added to the second remote mapping information.

4. The method according to claim 1, characterized in that: The method further comprises: Receiving pull request information sent by a target device, wherein the pull request information indicates to pull remote message header mapping information, and the target device is a sending device or a receiving device; Based on the pull request information, the current remote message header mapping information is sent to the target device, so that the target device updates the local message header mapping information of the target device based on the current remote message header mapping information.

5. A method for processing message data, characterized in that: Applicable to sending equipment, including: Acquire message data to be transmitted and local message header mapping information, wherein the message data includes message header information and message content, and the message header mapping information includes a mapping relationship between preset message header fields and preset index information; Parsing the message header information in the message data to obtain the message header field of the message data; Querying the index information of the message header field in the local message header mapping information; When the index information of the message header field is not found in the local message header mapping information, pull the remote message header mapping information from the server to update the local message header mapping information based on the remote message header mapping information, wherein the remote message header mapping information is updated by the method according to any one of claims 1 to 4; Re-querying the index information of the message header field in the updated local message header mapping information; When the target index information corresponding to the message header field is found, the target index information and the message content are packaged to obtain compressed message data, and the compressed message data is transmitted.

6. The method according to claim 5, characterized in that The method further comprises: When the target index information corresponding to the message header field cannot be found in the updated local message header mapping information, the message header field and the message content are packaged to obtain packaged message data; The packaged message data is transmitted.

7. A method for processing message data, characterized in that: Applicable to receiving equipment, including: Receiving compressed message data, wherein the compressed message data includes message header information and message content; Extracting the message header information from the compressed message data, and detecting the message header information; When it is detected that the message header information is the target index information after message header compression, query the message header field corresponding to the target index information in the local message header mapping information, wherein the message header mapping information includes a mapping relationship between a preset message header field and preset index information; When the message header field corresponding to the target index information is not found in the local message header mapping information, remote message header mapping information is pulled from the server to update the local message header mapping information based on the remote message header mapping information, wherein the remote message header mapping information is updated by the method described in any one of claims 1 to 4; Re-querying the message header field corresponding to the target index information in the updated local message header mapping information; When the target message header field corresponding to the target index information is found, the target message header field and the message content in the compressed message data are stored.

8. The method according to claim 7, characterized in that The method further comprises: When it is detected that the message header information is an uncompressed message header field, the uncompressed message header field and the message content in the compressed message data are stored.

9. A method and device for processing message data, characterized in that: Applicable to sending equipment, including: An acquisition unit, used to acquire message data to be transmitted and local message header mapping information, wherein the message data includes message header information and message content, and the message header mapping information includes a mapping relationship between a preset message header field and preset index information; A parsing unit, used to parse the message header information in the message data to obtain the message header field of the message data; A first query unit, used to query the index information of the message header field in the local message header mapping information; A first pulling unit, configured to pull remote header mapping information from a server when index information of the header field is not found in local header mapping information, so as to update local header mapping information based on the remote header mapping information, wherein the remote header mapping information is updated by the method according to any one of claims 1 to 4; A second query unit, used to re-query the index information of the message header field in the updated local message header mapping information; The transmission unit is used to package the target index information and the message content to obtain compressed message data and transmit the compressed message data when the target index information corresponding to the message header field is queried.

10. A message data processing device, characterized in that: Applicable to receiving equipment, including: A receiving unit, configured to receive compressed message data, wherein the compressed message data includes message header information and message content; An extraction unit, used to extract the message header information from the compressed message data and detect the message header information; A third query unit is used to query the message header field corresponding to the target index information in the local message header mapping information when it is detected that the message header information is the target index information after message header compression, wherein the message header mapping information includes a mapping relationship between a preset message header field and a preset index information; A second pulling unit, configured to pull remote header mapping information from a server when the header field corresponding to the target index information is not found in the local header mapping information, so as to update the local header mapping information based on the remote header mapping information, wherein the remote header mapping information is updated by the method described in any one of claims 1 to 4; A fourth query unit, configured to re-query the message header field corresponding to the target index information in the updated local message header mapping information; A storage unit is used to store the target message header field and the message content in the compressed message data when the target message header field corresponding to the target index information is queried.

11. A message data processing device, characterized in that: Applicable to servers, including: A data acquisition unit, used to acquire target message header data stored in a receiving device, wherein the target message header data includes a plurality of message header fields, and the message header fields include a field name and a field value; A determination unit, configured to filter out newly added field names and non-newly added field names from the target message header data based on field names in current remote message header mapping information, wherein the remote message header mapping information includes a mapping relationship between preset message header fields and preset index information; A statistical unit, configured to count the field values ​​corresponding to the non-newly added field names in the target message header data, and obtain the frequency information of the occurrence of the field values ​​corresponding to the non-newly added field names; A first updating unit, configured to add the newly added field name and its corresponding index information to the remote message header mapping information; and update the added remote message header mapping information according to the frequency information of each field value corresponding to the non-newly added field name; The second updating unit is used to trigger the sending of updated remote message header mapping information to the sending device and the receiving device, so that the sending device and the receiving device update their local message header mapping information.

12. An electronic device, characterized in that: It comprises a memory and a processor; the memory stores an application program, and the processor is used to run the application program in the memory to execute the operations in the message data processing method according to any one of claims 1 to 8.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the method for processing message data according to any one of claims 1 to 8.

14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps in the method for processing message data described in any one of claims 1 to 8 are implemented.

Images