Data transmission method and device, storage medium and electronic device

Abstract

The application discloses a data transmission method and device, a storage medium and an electronic device. It relates to the field of financial technology. The method comprises the following steps: acquiring a first compression dictionary and a starting use time of the first compression dictionary, wherein the first compression dictionary is used for compressing and decompressing data; when the starting use time arrives, using the first compression dictionary and a first identifier of the first compression dictionary to compress first to-be-compressed data to obtain first compressed data, wherein the first identifier is carried in the first compressed data; and sending the first compressed data to a decompression device corresponding to the to-be-compressed data, wherein the first compression dictionary is stored on the decompression device, and the first identifier is used to instruct the decompression device to use the first compression dictionary to decompress the first compressed data and compressed data received after the first compressed data. Through the application, the problem of low efficiency of dictionary updating in the related art is solved.

Description

Technical Field

[0001] This application relates to the field of financial technology, and more specifically, to a data transmission method, apparatus, storage medium, and electronic device. Background Technology

[0002] With the rapid development of the Internet, the application and scale of the Internet, as well as network traffic, are also showing a rapid growth trend. Due to the limited and expensive network resources, the rapidly growing Internet traffic has brought tremendous pressure to data center network operations. Therefore, in response to the ever-increasing demand for business data interaction between data centers, research on wide area network compression technology has gradually become a hot topic in the industry. In existing technologies, correlations are generally captured by learning from a portion of sample data. Data with strong correlations is made into a dictionary, and short identifiers are used to replace frequently occurring data blocks. A general pattern dictionary is built using compression and decompression devices. The compression and decompression devices are controlled to simultaneously refer to this dictionary to compress and decompress network traffic packet by packet. A timed mechanism is used to implement the routing protocol to distribute the updates of the dictionary for compression and decompression devices. However, the static dictionary has a single pattern and cannot select a suitable training dictionary for different types of business, which reduces the compression effect. Moreover, dictionary training depends on more storage space and fast traversal algorithms. This approach usually requires compression and decompression devices to negotiate information such as the dictionary, compression protocol, and session ID (Identity Document), which places high demands on hardware resource configuration and consumption.

[0003] There is currently no effective solution to the problem of low efficiency in dictionary updates in related technologies. Summary of the Invention

[0004] The main objective of this application is to provide a data transmission method, apparatus, storage medium, and electronic device to solve the problem of low efficiency in dictionary updates in related technologies.

[0005] To achieve the above objectives, according to one aspect of this application, a data transmission method is provided. The method includes:

[0006] Obtain the first compression dictionary and the start time of the first compression dictionary, wherein the first compression dictionary is used to compress and decompress data;

[0007] When the start time arrives, the first data to be compressed is compressed using the first compression dictionary and the first identifier of the first compression dictionary to obtain the first compressed data, wherein the first compressed data carries the first identifier;

[0008] The first compressed data is sent to the decompression device corresponding to the data to be compressed, wherein the decompression device stores the first compression dictionary, and the first identifier is used to instruct the decompression device to use the first compression dictionary to decompress the first compressed data and the compressed data received after the first compressed data.

[0009] Optionally, when the start time of use arrives, the first compression dictionary and the first identifier of the first compression dictionary are used to compress the first data to be compressed to obtain the first compressed data, including:

[0010] When the start time arrives, the first compression dictionary is used to compress the first data to be compressed to obtain reference compressed data;

[0011] The first identifier is added to the reference compressed data to obtain the first compressed data.

[0012] Optionally, when the start time of use arrives, the first compression dictionary is used to compress the first data to be compressed to obtain reference compressed data, including:

[0013] When the start time for use arrives, obtain the service type corresponding to the first compressed dictionary;

[0014] Extract the first piece of data from the received data belonging to the aforementioned service type and use it as the first piece of data to be compressed;

[0015] The first compression dictionary is used to compress the first data to be compressed to obtain the reference compressed data.

[0016] Optionally, obtaining the first compressed dictionary and its start time of use includes:

[0017] The first compressed dictionary is obtained by training on the second compressed dictionary that has already been used;

[0018] Determine the start time of use of the first compressed dictionary;

[0019] The first compression dictionary is sent to the decompression device, wherein the first compression dictionary is used to instruct the decompression device to store the first compression dictionary and to start using the first compression dictionary to decompress the data when it receives data carrying the first identifier.

[0020] Optionally, the method further includes:

[0021] Receive the second compressed data;

[0022] Detect whether the second compressed data carries a dictionary identifier;

[0023] If the second compressed data is detected to carry a second identifier, the second compressed dictionary corresponding to the second identifier is extracted from the dictionary database, wherein the dictionary identifier includes the second identifier;

[0024] Use the second compression dictionary to decompress the second compressed data.

[0025] Optionally, after detecting whether the second compressed data carries a dictionary identifier, the method further includes:

[0026] If the second compressed data does not carry a dictionary identifier, obtain the third compressed dictionary used to decompress the previous compressed data;

[0027] The second compressed data is decompressed using the third compression dictionary.

[0028] Optionally, after decompressing the second compressed data using the second compression dictionary, the method further includes:

[0029] The compressed data received after the second compressed data is decompressed using the second compression dictionary until compressed data carrying the third identifier is received.

[0030] To achieve the above objectives, according to another aspect of this application, a data transmission apparatus is provided. The apparatus includes:

[0031] The first acquisition module is used to acquire the first compressed dictionary and the start time of the first compressed dictionary, wherein the first compressed dictionary is used to compress and decompress data;

[0032] The compression module is used to compress the first data to be compressed using the first compression dictionary and the first identifier of the first compression dictionary when the start time is reached, to obtain the first compressed data, wherein the first compressed data carries the first identifier;

[0033] The sending module is used to send the first compressed data to the decompression device corresponding to the data to be compressed, wherein the decompression device stores the first compression dictionary, and the first identifier is used to instruct the decompression device to use the first compression dictionary to decompress the first compressed data and the compressed data received after the first compressed data.

[0034] This application employs the following steps: obtaining a first compression dictionary for compressing and decompressing data, and the start time of the first compression dictionary; upon the start time, compressing the first data to be compressed using the first compression dictionary and its first identifier, obtaining first compressed data carrying the first identifier; and sending the first compressed data to the decompression device that stores the first compression dictionary corresponding to the data to be compressed. By sending the new dictionary to both the compression and decompression devices, and using the first identifier to enable the decompression device to decompress the first compressed data and any subsequent compressed data using the first compression dictionary upon receiving the first compressed data carrying the first identifier, this approach not only updates the dictionaries used by the compression and decompression devices in real time but also maintains synchronization between them. Therefore, it solves the problem of low dictionary update efficiency in related technologies, thereby improving the efficiency of dictionary updates. Attached Figure Description

[0035] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0036] Figure 1 This is a flowchart of a data transmission method provided according to an embodiment of this application;

[0037] Figure 2 This is a flowchart illustrating the acquisition of the first compressed dictionary according to an embodiment of this application;

[0038] Figure 3 This is a schematic diagram of the first data to be compressed according to an embodiment of this application;

[0039] Figure 4 This is a schematic diagram illustrating the usage method of the first compressed dictionary provided in the embodiments of this application;

[0040] Figure 5 This is a flowchart illustrating the usage process of the first compressed dictionary provided in the embodiments of this application;

[0041] Figure 6 This is a flowchart illustrating the use of a compression dictionary by a decompression device according to an embodiment of this application;

[0042] Figure 7 This is a schematic diagram of a data transmission apparatus according to an embodiment of this application;

[0043] Figure 8 A schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0044] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0045] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0046] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0047] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent information from the aforementioned user or organization.

[0048] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of a data transmission method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0049] Step S101: Obtain the first compression dictionary and the start time of the first compression dictionary, wherein the first compression dictionary is used to compress and decompress data;

[0050] Step S102: When the start time arrives, the first compression dictionary and the first identifier of the first compression dictionary are used to compress the first data to be compressed to obtain the first compressed data, wherein the first compressed data carries the first identifier.

[0051] Step S103: Send the first compressed data to the decompression device corresponding to the data to be compressed, wherein the decompression device stores the first compression dictionary, and the first identifier is used to instruct the decompression device to use the first compression dictionary to decompress the first compressed data and the compressed data received after the first compressed data.

[0052] In the technical solution provided in step S101 above, the first compression dictionary can be used, but is not limited to, for compression and decompression devices. The compression device can compress data using the compression dictionary, and the decompression device can decompress data using the compression dictionary. When the compression dictionary used by the compression device and the decompression device is the same dictionary, it can be ensured that the decompression device has a complete decompression function for the compressed data transmitted by the compression device.

[0053] Optionally, in this embodiment, the first compression dictionary can be, but is not limited to, obtained through training. For example, the compression device has its initial compression dictionary, and training can be performed based on the initial compression dictionary according to various business scenarios to obtain a first compression dictionary corresponding to various business scenarios. Alternatively, the content of the business scenario can be trained according to different business scenarios to obtain a first compression dictionary corresponding to various business scenarios.

[0054] Optionally, in this embodiment, a dictionary training device may be used to train the dictionary to obtain the first compressed dictionary. The dictionary training device may be, but is not limited to, a compression device with dictionary training function, a decompression device with dictionary training function, or an external device focused on training the dictionary.

[0055] Optionally, in this embodiment, the start time for using the first compressed dictionary can be determined based on changes in the dictionary's usage scenario, business operations, etc. For example, the start time for using the first compressed dictionary can be, but is not limited to, when the dictionary's usage scenario changes, to enable the use of the first compressed dictionary. Alternatively, the start time for using the first compressed dictionary can be, but is not limited to, when the compression device and decompression device receive and store the first compressed dictionary, etc.

[0056] Optionally, in this embodiment, it may include, but is not limited to, multiple compression devices and decompression devices, and may send the first compressed dictionary to each compression device and decompression device, and may, but is not limited to, connect the dictionary training device used to train the first compressed dictionary to each compression device and each decompression device through a management network.

[0057] Optionally, in this embodiment, the compressed dictionary can be trained using a dictionary training device that has deployed a dictionary compression algorithm, without increasing the overhead of maintaining the dictionary and timers on the compression device.

[0058] Optionally, in this embodiment, port mirroring can be performed on the input and output traffic of the compression device to synchronously transmit the traffic of the compression device to the dictionary training device used for dictionary training. For example, port mirroring can be performed on the input and output traffic of the compression device according to a time period or time point to transmit it to the dictionary training device. This can be achieved by controlling the duration or time point of the time period of the port mirrored message data or by using a timing mechanism to realize the automatic dictionary training process under certain conditions.

[0059] Optionally, in this embodiment, the compression function of the compression device and the decompression function of the decompression device may be deployed on the same device, that is, a device may have both the compression function of the compression device and the decompression function of the decompression device.

[0060] Optionally, in this embodiment, the compression device and the decompression device may, but are not limited to, be able to transmit data. For example, when a compression device receives a first compression dictionary, the compression device that receives the first compression dictionary may, but is not limited to, be able to synchronously transmit the compressed first compression dictionary to all compression devices and decompression devices. Alternatively, when a decompression device receives a first compression dictionary, the decompression device that receives the first compression dictionary may, but is not limited to, be able to synchronously transmit the compressed first compression dictionary to all compression devices and decompression devices.

[0061] In one exemplary embodiment, the first compressed dictionary and its start time of use may be obtained, but are not limited to, by: training the first compressed dictionary with a second compressed dictionary that has been used to obtain the first compressed dictionary; determining the start time of use of the first compressed dictionary; and sending the first compressed dictionary to the decompression device, wherein the first compressed dictionary is used to instruct the decompression device to store the first compressed dictionary and to start using the first compressed dictionary to decompress data when receiving data carrying the first identifier.

[0062] Optionally, in this embodiment, the second compressed dictionary may be, but is not limited to, the currently used compressed dictionary, or may be, but is not limited to, a previously used compressed dictionary.

[0063] Optionally, in this embodiment, the first compressed dictionary can be obtained by training the second compressed dictionary using a dictionary training device that has deployed a dictionary compression algorithm, without increasing the overhead of maintaining the dictionary and timers on the compression device. The dictionary training device can be, but is not limited to, a compression device with dictionary training function, a decompression device with dictionary training function, or an external device focused on training the dictionary.

[0064] Optionally, in this embodiment, the start time of use of the first compressed dictionary can be determined according to the usage scenario of the first compressed dictionary. For example, when the service is about to enter the usage scenario of the first compressed dictionary, the start time of use of the first compressed dictionary can be determined as the time when the service is switched to the usage scenario of the first compressed dictionary.

[0065] Optionally, in this embodiment, the trained first compressed dictionary may be sent to a decompression device, and the compression device may be used for decompression using the first compressed dictionary, but is not limited to that used for decompression.

[0066] Optionally, in this embodiment, when the decompression device receives the first compression dictionary, it may, but is not limited to, instruct the decompression device to start using the first compression dictionary to decompress the data by means of the first identifier, or it may, but is not limited to, carry the first identifier on the data compressed by the compression device, and send the data carrying the first identifier to the decompression device, and the decompression device starts using the first compression dictionary to decompress the data according to the first identifier.

[0067] Figure 2 This is a flowchart illustrating the acquisition of the first compressed dictionary according to an embodiment of this application, such as... Figure 2 As shown, the second compressed dictionary is a previously used dictionary. The first compressed dictionary is obtained by training the second compressed dictionary that has been used. The start time of the first compressed dictionary is then determined. The first compressed dictionary is sent to the decompression device, and the first compressed dictionary and its start time are sent to the compression device.

[0068] In the technical solution provided in step S102 above, the first compression dictionary can be used to compress the first data to be compressed, and then the first identifier of the first compression dictionary can be attached to the data packet after the first data to be compressed.

[0069] Optionally, in this embodiment, the first identifier of the first compression dictionary may be used, but is not limited to, to instruct the decompression device that receives the first compressed data to decompress it using the first compression dictionary.

[0070] Optionally, in this embodiment, the decompression device may have, but is not limited to, multiple data to be compressed, and the first data to be compressed may be, but is not limited to, the data with the highest priority among the data to be compressed by the decompression device.

[0071] In one exemplary embodiment, when the start usage time arrives, the first data to be compressed may be compressed using the first compression dictionary and the first identifier of the first compression dictionary in the following manner, to obtain first compressed data: when the start usage time arrives, the first data to be compressed is compressed using the first compression dictionary to obtain reference compressed data; the first identifier is added to the reference compressed data to obtain the first compressed data.

[0072] Optionally, in this embodiment, the reference compressed data may be, but is not limited to, obtained by compressing the first data to be compressed using the first compression dictionary when the start time of the first compression dictionary is reached.

[0073] Optionally, in this embodiment, the first data to be compressed may be, but is not limited to, the highest priority data in the data to be sent in the first compression dictionary. Alternatively, the first data to be compressed may be the data in the data to be sent in the first compression dictionary that needs to be sent in the next moment.

[0074] Optionally, in this embodiment, the first identifier may be marked on the reference compressed data, but is not limited to, so that when the decompression device receives the first compressed data, it can first identify the first identifier.

[0075] In an exemplary embodiment, when the start usage time arrives, the first data to be compressed may be compressed using the first compression dictionary to obtain reference compressed data in the following manner, including: when the start usage time arrives, obtaining the service type corresponding to the first compression dictionary; extracting the first data from the received data to be compressed belonging to the service type as the first data to be compressed; and compressing the first data to be compressed using the first compression dictionary to obtain the reference compressed data.

[0076] Optionally, in this embodiment, the service type corresponding to the first compression dictionary may be, but is not limited to, determined by the data processed by the compression device. For example, the compression device processes different data types at different stages. Or, the compression device processes different data types in different scenarios.

[0077] Optionally, in this embodiment, the service type corresponding to the first compression dictionary may be determined during the training phase of the first compression dictionary, for example: training the corresponding compression dictionary according to the service type (the service type may include, but is not limited to, time period, time point, IP information of the packet data, port number, etc.) to obtain the trained compression dictionary.

[0078] Optionally, in this embodiment, the data to be compressed, which belongs to the service type, may be, but is not limited to, data that needs to be compressed by the first compression dictionary. It may include, but is not limited to, data received before receiving the first compression dictionary, and data received during the use of the first compression dictionary.

[0079] Optionally, in this embodiment, the data to be compressed belonging to the service type may include, but is not limited to, multiple data to be compressed, and the priority of each data may be determined as the first data to be compressed. For example, the data with the highest priority is determined as the first data to be compressed based on the compression priority of each data. Alternatively, the data with the highest priority is determined as the first data to be compressed based on the transmission priority of each data.

[0080] Figure 3 This is a schematic diagram of the first data to be compressed according to an embodiment of this application, such as... Figure 3 As shown, the transmission queue includes: data1, data2, data3... datan, a total of n data to be transmitted. Among them, data1 is the first data to be transmitted in the transmission queue. Data1 can be, but is not limited to, being the first data to be compressed.

[0081] If data 2 has the highest transmission priority in the transmission queue, it is possible, but not limited to, to determine data 2, which has the highest transmission priority in the transmission queue, as the first data to be compressed and to be transmitted first.

[0082] In the technical solution provided in step S103 above, there may be, but is not limited to, multiple decompression devices, wherein the decompression device that receives the first compressed data may be, but is not limited to, the device corresponding to the data to be compressed.

[0083] Optionally, in this embodiment, the decompression device stores a first compressed dictionary, and can send the first compressed dictionary to both the compression device and the decompression device simultaneously, but not limited to this. The compression device and the decompression device can, but not limited to this, record the first compressed dictionary and the historical dictionary used before the first compressed dictionary through caching.

[0084] Optionally, in this embodiment, when receiving first compressed data carrying a first identifier, the decompression device may, but is not limited to, receive an instruction to decompress using a first compression dictionary by recognizing the first identifier on the first compressed data, and then use the first compression dictionary to decompress the received first compressed data and each subsequently received compressed data.

[0085] Figure 4 This is a schematic diagram of the method of using the first compressed dictionary according to the embodiments of this application. As shown in Figure 4, taking the use of a dictionary training device to obtain the first compressed dictionary through training and sending it to the compression device and decompression device as an example, the first compressed dictionary can be used for data processing in the following ways, but not limited to:

[0086] The dictionary training device deploys a dictionary compression algorithm without increasing the overhead of dictionary maintenance and timer maintenance on the compression device. If the compression device and decompression device do not receive a new first compressed dictionary, the compression device and decompression device can, but are not limited to, use a general static dictionary for data transmission.

[0087] The dictionary training device connects with compression devices (which may include, but are not limited to, multiple compression devices) and decompression devices (which may include, but are not limited to, multiple decompression devices) through a management network. The input and output traffic of the compression devices are transmitted to the dictionary training device through port mirroring. The dictionary training device may, but is not limited to, obtain data from the compression devices through mirroring and train a new first compressed dictionary in real time.

[0088] The dictionary training device obtains the first compressed dictionary through training and determines time T as the start time for using the first compressed dictionary. The first compressed dictionary is sent to all compression and decompression devices in advance. The compression and decompression devices may, but are not limited to, store the first compressed dictionary in a cache.

[0089] When time T arrives, the compression device receives the first data to be compressed, an. After compressing the first data to be compressed, an and marking it with the first identifier bn to obtain the first compressed data, the compression device sends the first compressed data to the corresponding decompression device.

[0090] For compressed data prior to the first identifier bn, the decompression device still uses a static dictionary for decompression.

[0091] Upon receiving and recognizing the first identifier bn and the first compressed data an, the decompression device initiates the first compression dictionary to decompress the first compressed data an and removes the first identifier bn from the first compressed data an, then forwards it to the receiving end.

[0092] Figure 5 This is a flowchart illustrating the usage process of the first compressed dictionary provided in the embodiments of this application, such as... Figure 5As shown, the compression and decompression devices can process the first compressed data and the compressed data received after the first compressed data using a first compression dictionary through, but not limited to, the following process:

[0093] The compression and decompression devices initially use the default dictionary M;

[0094] Detecting the input and output flow of the compression equipment determines whether to train a new dictionary relevant to the business.

[0095] Without needing to train a new dictionary relevant to the business, a new dictionary M+1 (the first compressed dictionary) is trained in real time based on the input and output traffic of the compression device, and dictionary M+1 is sent to the compression and decompression devices.

[0096] When the decompression device receives a message an (first compressed data) carrying a tag bn (first identifier) ​​sent by the compression device, the decompression device uses dictionary M+1 to decompress the message an.

[0097] If the decompression device does not receive a message an (first compressed data) carrying tag bn (first identifier) ​​sent by the compression device, the decompression device uses dictionary M to decompress.

[0098] When it is necessary to train a new dictionary related to business, a new dictionary Mn (the first compressed dictionary) is trained based on parameters such as the business duration and time point of the compression device, and the dictionary Mn is sent to the compression device and the decompression device.

[0099] When it is necessary to use the dictionary Mn corresponding to the business, the compression device sends a message an (first compressed data) carrying the tag bn (first identifier) ​​to the decompression device;

[0100] When the decompression device receives a message an (first compressed data) carrying a tag bn (first identifier) ​​sent by the compression device, the decompression device uses the dictionary Mn to decompress the message an.

[0101] If the decompression device does not receive a message an (first compressed data) carrying tag bn (first identifier) ​​sent by the compression device, the decompression device uses dictionary M to decompress.

[0102] In one exemplary embodiment, the method may include, but is not limited to, the following: receiving second compressed data; detecting whether the second compressed data carries a dictionary identifier; if the second compressed data is detected to carry a second identifier, extracting a second compressed dictionary corresponding to the second identifier from a dictionary database, wherein the dictionary identifier includes the second identifier; and decompressing the second compressed data using the second compressed dictionary.

[0103] Optionally, in this embodiment, the second compressed data may be, but is not limited to, compressed data sent by a compression device, and the decompression device may be, but is not limited to, used to decompress the second compressed data. The second compressed data may, but is not limited to, carry an identifier for a dictionary used to indicate the decompression device.

[0104] Optionally, in this embodiment, the dictionary identifier can be, but is not limited to, used to indicate the dictionary used by the decompression device. The second identifier can be, but is not limited to, used to indicate that the decompression device uses the corresponding dictionary to decompress the second compressed data. The dictionary identifier can be, but is not limited to, an identifier marked on the second compression device after the compression device compresses the second compressed data.

[0105] Optionally, in this embodiment, the decompression device may, but is not limited to, store multiple compression dictionaries, and may, but is not limited to, determine the compression dictionary used to decompress the second compressed data based on the second identifier.

[0106] In one exemplary embodiment, after detecting whether the second compressed data carries a dictionary identifier, the method may include, but is not limited to, the following: if the second compressed data does not carry a dictionary identifier, obtaining a third compression dictionary used to decompress the previous compressed data; and using the third compression dictionary to decompress the second compressed data.

[0107] Optionally, in this embodiment, the third compression dictionary is the compression dictionary used by the decompression device before receiving the second compressed data. If the dictionary identifier is not identified, the decompression device may, but is not limited to, continue to use the compression dictionary used before receiving the second compressed data for decompression.

[0108] In one exemplary embodiment, after decompressing the second compressed data using the second compression dictionary, the method may include, but is not limited to, decompressing compressed data received after the second compressed data using the second compression dictionary until compressed data carrying a third identifier is received.

[0109] Optionally, in this embodiment, when it is detected that the second compressed data carries a dictionary identifier, the second compressed dictionary corresponding to the second identifier can be determined according to the second identifier carried by the second compressed data, and the second compressed data can be decompressed using the second compressed dictionary and each compressed data received after the second compressed data can be decompressed using the second compressed dictionary until the next dictionary identifier is detected.

[0110] Figure 6 This is a flowchart illustrating the use of a compression dictionary by a decompression device according to an embodiment of this application, such as... Figure 6 As shown, the compression dictionary used by the decompression device can be determined through, but is not limited to, the following steps:

[0111] Step S602: The decompression device receives the second compressed data;

[0112] Step S604: The decompression device detects whether the second compressed data carries a dictionary identifier;

[0113] Step S606: If the second compressed data is detected to carry a second identifier, extract the second compressed dictionary corresponding to the second identifier from the dictionary library, and use the second compressed dictionary to decompress the second compressed data and the compressed data received after the second compressed data until the compressed data carrying the third identifier is received.

[0114] Step S608: If no dictionary identifier is detected in the second compressed data, obtain the third compression dictionary used to decompress the previous compressed data, and use the third compression dictionary to decompress the second compressed data and the compressed data received after the second compressed data until compressed data carrying the third identifier is received.

[0115] The data transmission method provided in this application embodiment obtains a first compression dictionary for compressing and decompressing data, as well as the start time of the first compression dictionary. When the start time arrives, it uses the first compression dictionary and a first identifier of the first compression dictionary to compress the first data to be compressed, obtaining first compressed data carrying the first identifier. The first compressed data is then sent to the decompression device corresponding to the data to be compressed, which stores the first compression dictionary. By sending the new dictionary to both the compression and decompression devices, and using the first identifier to enable the decompression device to decompress the first compressed data and any compressed data received after the first compressed data upon receiving the first compressed data carrying the first identifier, the method not only updates the dictionaries used by the compression and decompression devices in real time but also maintains synchronization between the dictionaries used by the compression and decompression devices. Therefore, it solves the problem of low dictionary update efficiency in related technologies, thereby improving the efficiency of dictionary updates.

[0116] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0117] This application also provides a data transmission apparatus. It should be noted that the data transmission apparatus of this application can be used to execute the data transmission method provided in this application. The data transmission apparatus provided in this application will be described below.

[0118] Figure 7 This is a schematic diagram of a data transmission apparatus according to an embodiment of this application. Figure 7 As shown, the device includes:

[0119] The first acquisition module 702 is used to acquire the first compressed dictionary and the start time of the first compressed dictionary, wherein the first compressed dictionary is used to compress and decompress data;

[0120] Compression module 704 is used to compress the first data to be compressed using the first compression dictionary and the first identifier of the first compression dictionary when the start time is reached, to obtain first compressed data, wherein the first compressed data carries the first identifier;

[0121] The sending module 706 is used to send the first compressed data to the decompression device corresponding to the data to be compressed, wherein the decompression device stores the first compression dictionary, and the first identifier is used to instruct the decompression device to use the first compression dictionary to decompress the first compressed data and the compressed data received after the first compressed data.

[0122] Optionally, in the data transmission apparatus provided in the embodiments of this application, the compression module includes:

[0123] A compression unit is used to compress the first data to be compressed using the first compression dictionary when the start time of use arrives, so as to obtain reference compressed data;

[0124] An adding unit is used to add the first identifier to the reference compressed data to obtain the first compressed data.

[0125] Optionally, in the data transmission apparatus provided in the embodiments of this application, the compression unit is further configured to:

[0126] When the start time for use arrives, obtain the service type corresponding to the first compressed dictionary;

[0127] Extract the first piece of data from the received data belonging to the aforementioned service type and use it as the first piece of data to be compressed;

[0128] The first compression dictionary is used to compress the first data to be compressed to obtain the reference compressed data.

[0129] Optionally, in the data transmission apparatus provided in the embodiments of this application, the first acquisition module includes:

[0130] The training unit is used to train the first compressed dictionary from the second compressed dictionary already in use;

[0131] A determining unit is configured to determine the start time of use of the first compressed dictionary;

[0132] A sending unit is configured to send the first compression dictionary to the decompression device, wherein the first compression dictionary is used to instruct the decompression device to store the first compression dictionary and to start using the first compression dictionary to decompress the data when it receives data carrying the first identifier.

[0133] Optionally, in the data transmission apparatus provided in the embodiments of this application, the apparatus further includes:

[0134] The receiving module is used to receive the second compressed data;

[0135] The detection module is used to detect whether the second compressed data carries a dictionary identifier;

[0136] An extraction module is configured to extract a second compressed dictionary corresponding to the second identifier from a dictionary database when the second compressed data is detected to carry a second identifier, wherein the dictionary identifier includes the second identifier;

[0137] The first decompression module is used to decompress the second compressed data using the second compression dictionary.

[0138] Optionally, in the data transmission apparatus provided in the embodiments of this application, the apparatus further includes:

[0139] The second acquisition module is used to acquire the third compression dictionary used to decompress the previous compressed data if the second compressed data does not carry a dictionary identifier.

[0140] The second decompression module is used to decompress the second compressed data using the third compression dictionary.

[0141] Optionally, in the data transmission apparatus provided in the embodiments of this application, the apparatus further includes:

[0142] The third decompression module is used to decompress the compressed data received after the second compressed data using the second compression dictionary until compressed data carrying the third identifier is received.

[0143] The data transmission apparatus provided in this application embodiment acquires a first compression dictionary for compressing and decompressing data, as well as the start time of the first compression dictionary. When the start time arrives, it uses the first compression dictionary and a first identifier of the first compression dictionary to compress first data to be compressed, obtaining first compressed data carrying the first identifier. The first compressed data is then sent to a decompression device that stores the first compression dictionary corresponding to the data to be compressed. By sending a new dictionary to both the compression and decompression devices, and using the first identifier to enable the decompression device to decompress the first compressed data and any compressed data received after the first compressed data upon receiving the first compressed data carrying the first identifier, the apparatus not only updates the dictionaries used by the compression and decompression devices in real time but also maintains synchronization between the dictionaries used by the compression and decompression devices. Therefore, it solves the problem of low dictionary update efficiency in related technologies, thereby improving the efficiency of dictionary updates.

[0144] The data transmission device includes a processor and a memory. The aforementioned modules and units are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0145] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and the efficiency of dictionary updates can be improved by adjusting kernel parameters.

[0146] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0147] This invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a method for transmitting the data.

[0148] This invention provides a processor for running a program, wherein the program executes a data transmission method during runtime.

[0149] like Figure 8 As shown, an embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps:

[0150] Obtain the first compression dictionary and the start time of the first compression dictionary, wherein the first compression dictionary is used to compress and decompress data;

[0151] When the start time arrives, the first data to be compressed is compressed using the first compression dictionary and the first identifier of the first compression dictionary to obtain the first compressed data, wherein the first compressed data carries the first identifier;

[0152] The first compressed data is sent to the decompression device corresponding to the data to be compressed, wherein the decompression device stores the first compression dictionary, and the first identifier is used to instruct the decompression device to use the first compression dictionary to decompress the first compressed data and the compressed data received after the first compressed data.

[0153] Optionally, when the start time of use arrives, the first compression dictionary and the first identifier of the first compression dictionary are used to compress the first data to be compressed to obtain the first compressed data, including:

[0154] When the start time arrives, the first compression dictionary is used to compress the first data to be compressed to obtain reference compressed data;

[0155] The first identifier is added to the reference compressed data to obtain the first compressed data.

[0156] Optionally, when the start time of use arrives, the first compression dictionary is used to compress the first data to be compressed to obtain reference compressed data, including:

[0157] When the start time for use arrives, obtain the service type corresponding to the first compressed dictionary;

[0158] Extract the first piece of data from the received data belonging to the aforementioned service type and use it as the first piece of data to be compressed;

[0159] The first compression dictionary is used to compress the first data to be compressed to obtain the reference compressed data.

[0160] Optionally, obtaining the first compressed dictionary and its start time of use includes:

[0161] The first compressed dictionary is obtained by training on the second compressed dictionary that has already been used;

[0162] Determine the start time of use of the first compressed dictionary;

[0163] The first compression dictionary is sent to the decompression device, wherein the first compression dictionary is used to instruct the decompression device to store the first compression dictionary and to start using the first compression dictionary to decompress the data when it receives data carrying the first identifier.

[0164] Optionally, the method further includes:

[0165] Receive the second compressed data;

[0166] Detect whether the second compressed data carries a dictionary identifier;

[0167] If the second compressed data is detected to carry a second identifier, the second compressed dictionary corresponding to the second identifier is extracted from the dictionary database, wherein the dictionary identifier includes the second identifier;

[0168] Use the second compression dictionary to decompress the second compressed data.

[0169] Optionally, after detecting whether the second compressed data carries a dictionary identifier, the method further includes:

[0170] If the second compressed data does not carry a dictionary identifier, obtain the third compressed dictionary used to decompress the previous compressed data;

[0171] The second compressed data is decompressed using the third compression dictionary.

[0172] Optionally, after decompressing the second compressed data using the second compression dictionary, the method further includes:

[0173] The compressed data received after the second compressed data is decompressed using the second compression dictionary until compressed data carrying the third identifier is received.

[0174] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0175] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes the following method steps:

[0176] Obtain the first compression dictionary and the start time of the first compression dictionary, wherein the first compression dictionary is used to compress and decompress data;

[0177] When the start time arrives, the first data to be compressed is compressed using the first compression dictionary and the first identifier of the first compression dictionary to obtain the first compressed data, wherein the first compressed data carries the first identifier;

[0178] The first compressed data is sent to the decompression device corresponding to the data to be compressed, wherein the decompression device stores the first compression dictionary, and the first identifier is used to instruct the decompression device to use the first compression dictionary to decompress the first compressed data and the compressed data received after the first compressed data.

[0179] Optionally, when the start time of use arrives, the first compression dictionary and the first identifier of the first compression dictionary are used to compress the first data to be compressed to obtain the first compressed data, including:

[0180] When the start time arrives, the first compression dictionary is used to compress the first data to be compressed to obtain reference compressed data;

[0181] The first identifier is added to the reference compressed data to obtain the first compressed data.

[0182] Optionally, when the start time of use arrives, the first compression dictionary is used to compress the first data to be compressed to obtain reference compressed data, including:

[0183] When the start time for use arrives, obtain the service type corresponding to the first compressed dictionary;

[0184] Extract the first piece of data from the received data belonging to the aforementioned service type and use it as the first piece of data to be compressed;

[0185] The first compression dictionary is used to compress the first data to be compressed to obtain the reference compressed data.

[0186] Optionally, obtaining the first compressed dictionary and its start time of use includes:

[0187] The first compressed dictionary is obtained by training on the second compressed dictionary that has already been used;

[0188] Determine the start time of use of the first compressed dictionary;

[0189] The first compression dictionary is sent to the decompression device, wherein the first compression dictionary is used to instruct the decompression device to store the first compression dictionary and to start using the first compression dictionary to decompress the data when it receives data carrying the first identifier.

[0190] Optionally, the method further includes:

[0191] Receive the second compressed data;

[0192] Detect whether the second compressed data carries a dictionary identifier;

[0193] If the second compressed data is detected to carry a second identifier, the second compressed dictionary corresponding to the second identifier is extracted from the dictionary database, wherein the dictionary identifier includes the second identifier;

[0194] Use the second compression dictionary to decompress the second compressed data.

[0195] Optionally, after detecting whether the second compressed data carries a dictionary identifier, the method further includes:

[0196] If the second compressed data does not carry a dictionary identifier, obtain the third compressed dictionary used to decompress the previous compressed data;

[0197] The second compressed data is decompressed using the third compression dictionary.

[0198] Optionally, after decompressing the second compressed data using the second compression dictionary, the method further includes:

[0199] The compressed data received after the second compressed data is decompressed using the second compression dictionary until compressed data carrying the third identifier is received.

[0200] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0201] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0202] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0203] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0204] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0205] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0206] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0207] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0208] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0209] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data transmission method, characterized in that, The method is applied to a compression device, including: Obtain the first compression dictionary and the start time of the first compression dictionary, wherein the first compression dictionary is used to compress and decompress data; When the start time arrives, the first data to be compressed is compressed using the first compression dictionary and the first identifier of the first compression dictionary to obtain the first compressed data, wherein the first compressed data carries the first identifier; The first compressed data is sent to the decompression device corresponding to the data to be compressed, wherein the decompression device stores the first compression dictionary, and the first identifier is used to instruct the decompression device to use the first compression dictionary to decompress the first compressed data and the compressed data received after the first compressed data; The step of obtaining the first compressed dictionary and its start time of use includes: training the second compressed dictionary that has been used to obtain the first compressed dictionary; determining the start time of use of the first compressed dictionary; and sending the first compressed dictionary to the decompression device, wherein the first compressed dictionary is used to instruct the decompression device to store the first compressed dictionary and to start using the first compressed dictionary to decompress data when it receives data carrying the first identifier; The determination of the start time of the first compressed dictionary includes: determining the time when the service is switched to the usage scenario of the first compressed dictionary as the start time; or, determining the time when the decompression device receives and stores the first compressed dictionary as the start time. The method further includes, prior to obtaining the first compressed dictionary and its initial usage time, the following steps: 1) transmitting the input and output traffic of the compression device to the dictionary training device via port mirroring; 2) obtaining data from the compression device using the mirrored input and output traffic; 3) training the first compressed dictionary in real-time using the data, and sending the first compressed dictionary to the connected compression and decompression devices; 4) storing the received first compressed dictionary in a cache. The dictionary training device is equipped with a dictionary compression algorithm and connected to the compression and decompression devices. The step of using the dictionary training device to train the first compressed dictionary in real time using the data, and then sending the first compressed dictionary to the connected compression and decompression devices, includes: detecting the input traffic and the output traffic through the dictionary training device to determine whether to train a new dictionary related to the service; if training the new dictionary is not required, training the first compressed dictionary in real time based on the input traffic and the output traffic, and sending the first compressed dictionary to the compression and decompression devices; if training the new dictionary is required, training the first compressed dictionary corresponding to the service based on the service duration and / or time point of the compression device, and sending the first compressed dictionary to the compression and decompression devices.

2. The method according to claim 1, characterized in that, When the initial usage time arrives, the first compression dictionary and the first identifier of the first compression dictionary are used to compress the first data to be compressed to obtain the first compressed data, including: When the start time arrives, the first compression dictionary is used to compress the first data to be compressed to obtain reference compressed data; The first identifier is added to the reference compressed data to obtain the first compressed data.

3. The method according to claim 2, characterized in that, When the initial usage time arrives, the first compression dictionary is used to compress the first data to be compressed to obtain reference compressed data, including: When the start time for use arrives, obtain the service type corresponding to the first compressed dictionary; Extract the first piece of data from the received data belonging to the aforementioned service type and use it as the first piece of data to be compressed; The first data to be compressed is compressed using the first compression dictionary to obtain the reference compressed data.

4. The method according to claim 1, characterized in that, The method further includes: Receive the second compressed data; Detect whether the second compressed data carries a dictionary identifier; If the second compressed data is detected to carry a second identifier, the second compressed dictionary corresponding to the second identifier is extracted from the dictionary database, wherein the dictionary identifier includes the second identifier; Use the second compression dictionary to decompress the second compressed data.

5. The method according to claim 4, characterized in that, After detecting whether the second compressed data carries a dictionary identifier, the method further includes: If the second compressed data does not carry a dictionary identifier, obtain the third compressed dictionary used to decompress the previous compressed data; The second compressed data is decompressed using the third compression dictionary.

6. The method according to claim 4, characterized in that, After decompressing the second compressed data using the second compression dictionary, the method further includes: The compressed data received after the second compressed data is decompressed using the second compression dictionary until compressed data carrying the third identifier is received.

7. A data transmission device, characterized in that, include: The first acquisition module is used to acquire the first compressed dictionary and the start time of the first compressed dictionary, wherein the first compressed dictionary is used to compress and decompress data; The compression module is used to compress the first data to be compressed using the first compression dictionary and the first identifier of the first compression dictionary when the start time is reached, to obtain the first compressed data, wherein the first compressed data carries the first identifier; The sending module is used to send the first compressed data to the decompression device corresponding to the data to be compressed, wherein the decompression device stores the first compression dictionary, and the first identifier is used to instruct the decompression device to use the first compression dictionary to decompress the first compressed data and the compressed data received after the first compressed data; The first acquisition module includes: a training unit for training the second compressed dictionary already in use to obtain the first compressed dictionary; a determination unit for determining the start time of the first compressed dictionary; and a sending unit for sending the first compressed dictionary to the decompression device, wherein the first compressed dictionary is used to instruct the decompression device to store the first compressed dictionary and start using the first compressed dictionary to decompress data when it receives data carrying the first identifier. The determining unit is further configured to: determine the time when the service is switched to the usage scenario of the first compressed dictionary as the start usage time; or, determine the time when the decompression device receives and stores the first compressed dictionary as the start usage time; The dictionary training device, which deploys a dictionary compression algorithm, is connected to the compression device and the decompression device via a management network. The device is further configured to: before acquiring the first compressed dictionary and its initial usage time, transmit the input and output traffic of the compression device to the dictionary training device via port mirroring; acquire data from the compression device based on the input and output traffic mirroring; train the first compressed dictionary in real-time using the data; and send the first compressed dictionary to the connected compression device and decompression device, wherein the compression device and the decompression device store the received first compressed dictionary in a cache. The device is further configured to: detect the input traffic and the output traffic through the dictionary training device to determine whether to train a new dictionary related to the service; if the new dictionary does not need to be trained, train the first compressed dictionary in real time based on the input traffic and the output traffic, and send the first compressed dictionary to the compression device and the decompression device; if the new dictionary needs to be trained, train the first compressed dictionary corresponding to the service based on the service duration and / or time point of the compression device, and send the first compressed dictionary to the compression device and the decompression device.

8. A processor, characterized in that, The processor is used to run a program, wherein the program executes the data transmission method according to any one of claims 1 to 6 when it runs.

9. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of the data transmission methods of claims 1 to 6.

Images