Method, device, equipment, medium and product for processing transmitted data
By converting string object and replacing constant data on the original data during data transmission, the problem of low data transmission rate is solved and more efficient data transmission is achieved.
Patent Information
- Application Number
- CN202410955055.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-07-17
AI Technical Summary
During data transmission, due to network bandwidth limitations, direct transmission of raw data will lead to the problem of low data transmission rate, especially in the transmission of large amounts of business certificates and financial statement data in the field of industry and finance.
By converting the original data in string object at the data sending end, determining the constant data and replacing it with data identification, generating the replacement transmission data and identification corresponding information, transmitting it to the data receiving end, and recovering the original data based on the identification information.
The amount of data transmission is reduced, the load on data transmission is alleviated, and the rate of data transmission is increased.
Smart Images

Figure CN118509485B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method, device, equipment, medium and product for processing transmitted data. Background Art
[0002] With the rapid development of Internet technology, more and more businesses have data transmission needs, and their data transmission methods have evolved from paper transmission to electronic transmission, which undoubtedly greatly improves the convenience of data transmission.
[0003] However, in some fields, data transmitted via electronic transmission will carry a large amount of electronic data. For example, in the business and financial field, there will be a large amount of business vouchers and financial statement data. If the original data is transmitted directly between the data sender and the data receiver, due to the limitation of network bandwidth, it will undoubtedly increase the load of data transmission and affect the data transmission rate. Summary of the invention
[0004] The present invention provides a method, device, equipment, medium and product for processing transmitted data to solve the problem of low data transmission rate in current data transmission between a data sending end and a data receiving end.
[0005] According to one aspect of the present invention, a method for processing transmission data is provided, which is executed by a data sending end and includes:
[0006] Acquire original transmission data to be transmitted to the data receiving end, and convert the original transmission data into a string object to generate string transmission data;
[0007] Determine at least one type of constant data from the string transmission data according to the data type of the original transmission data, and determine the data identifier corresponding to each of the constant data;
[0008] Replacing each type of the constant data in the character string transmission data with a data identifier corresponding to each type of the constant data to generate replacement transmission data;
[0009] The replacement transmission data and the identification corresponding information are transmitted to the data receiving end, so that the data receiving end can restore the original transmission data based on the replacement transmission data and the identification corresponding information; wherein the identification corresponding information is generated based on each type of constant data and the data identification corresponding to each type of constant data.
[0010] According to another aspect of the present invention, a method for processing transmission data is provided, which is executed by a data receiving end and includes:
[0011] Receiving the replacement transmission data and the identification corresponding information sent by the data sending end; wherein the identification corresponding information is generated according to various types of constant data and the data identifications corresponding to the various types of constant data;
[0012] replacing each of the data identifiers included in the replacement transmission data with the constant data corresponding to each of the data identifiers according to the identifier corresponding information, to generate character string transmission data;
[0013] The character string transmission data is type-converted according to the data type of the original transmission data to obtain the original transmission data.
[0014] According to another aspect of the present invention, there is provided a transmission data processing device, which is configured at a data transmitting end and includes:
[0015] A string transmission data generating module is used to obtain the original transmission data to be transmitted to the data receiving end, and convert the original transmission data into a string object to generate string transmission data;
[0016] A constant data determination module, used to determine at least one type of constant data from the string transmission data according to the data type of the original transmission data, and determine a data identifier corresponding to each of the constant data;
[0017] A replacement transmission data generation module, used for replacing each type of constant data in the string transmission data with a data identifier corresponding to each type of constant data, to generate replacement transmission data;
[0018] A data sending module is used to transmit the replacement transmission data and the identification corresponding information to the data receiving end, so that the data receiving end can restore the original transmission data based on the replacement transmission data and the identification corresponding information; wherein the identification corresponding information is generated based on each type of constant data and the data identification corresponding to each type of constant data.
[0019] According to another aspect of the present invention, there is provided a transmission data processing device, which is configured at a data receiving end and includes:
[0020] A data receiving module, used for receiving the replacement transmission data and the identification corresponding information sent by the data sending end; wherein the identification corresponding information is generated according to various types of constant data and the data identifications corresponding to the various types of constant data;
[0021] A character string transmission data generating module, used for replacing each of the data identifiers included in the replacement transmission data with the constant data corresponding to each of the data identifiers according to the identifier corresponding information, so as to generate character string transmission data;
[0022] The original transmission data acquisition module is used to perform type conversion on the string transmission data according to the data type of the original transmission data to acquire the original transmission data.
[0023] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
[0024] at least one processor; and
[0025] a memory communicatively connected to the at least one processor; wherein,
[0026] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute any one of the transmission data processing methods of the present invention.
[0027] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement any one of the transmission data processing methods of the present invention when executed.
[0028] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the transmission data processing methods of the present invention.
[0029] Since the amount of constant data is usually much larger than the amount of data identifiers, the present invention generates replacement transmission data by replacing various types of constant data with data identifiers corresponding to the various types of constant data in the character string transmission data; the replacement transmission data and the identification corresponding information are transmitted to the data receiving end, so that the data receiving end can restore the original transmission data based on the replacement transmission data and the identification corresponding information; wherein the identification corresponding information is generated based on the various types of constant data and the data identifiers corresponding to the various types of constant data, and under the premise of ensuring that the data receiving end can restore the original transmission data, since the constant data is replaced with the corresponding data identifier for transmission, the data transmission amount is undoubtedly greatly reduced, the load of data transmission between the data sending end and the data receiving end is alleviated, and the data transmission rate is improved.
[0030] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0032] Figure 1 A flowchart of a method for processing transmission data provided in Embodiment 1 of the present invention;
[0033] Figure 2 A flowchart of a method for processing transmission data provided in Embodiment 2 of the present invention;
[0034] Figure 3 A flowchart of a method for processing transmission data provided in Embodiment 3 of the present invention;
[0035] Figure 4 A flowchart of a method for processing transmission data provided by Embodiment 4 of the present invention;
[0036] Figure 5 A schematic diagram of the structure of a data transmission processing device provided in Embodiment 5 of the present invention;
[0037] Figure 6 A schematic diagram of the structure of a data transmission processing device provided in Embodiment 6 of the present invention;
[0038] Figure 7 It is a structural schematic diagram of an electronic device for implementing the data transmission processing method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0039] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0040] It should be noted that the terms "original", "string", "replacement", "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0041] Embodiment 1
[0042] Figure 1 This is a flowchart of a method for processing transmission data provided in the first embodiment of the present invention. This embodiment can be applied to the case where the original transmission data to be sent from the data sending end to the data receiving end is processed to reduce the data transmission volume. The method can be executed by a transmission data processing device, which is configured at the data sending end, such as a server or a client, and can be implemented in the form of hardware and / or software. Figure 1 As shown, the method includes:
[0043] S101, obtaining original transmission data to be transmitted to a data receiving end, and converting the original transmission data into a string object to generate string transmission data.
[0044] Among them, the data sending end transmits data to the data receiving end, the data sending end represents the subject that sends data outward, and the data receiving end represents the subject that receives data. The data sending end can be a server or a client, and the data receiving end can also be a server or a client. It can be understood that the data sending end and the data receiving end can be servers at the same time, that is, data transmission is carried out between two servers; the data sending end and the data receiving end can also be clients at the same time, that is, data transmission is carried out between two clients; the data sending end and the data receiving end can also be one server and one client, that is, the server transmits data to the client, or the client transmits data to the server. This embodiment does not limit the specific types of the data receiving end and the data sending end.
[0045] The original transmission data refers to the initial data to be transmitted from the data sender to the data receiver. It is understandable that since the original transmission data has not undergone any compression processing, in some business scenarios, the original transmission data will have a large amount of data. For example, in business and financial business scenarios, there will be a large amount of business vouchers and financial statement data. The original transmission data will have a data volume of several megabytes to tens of megabytes. If the data sender directly sends the original transmission data to the data receiver, due to the limitation of network bandwidth, it will undoubtedly increase the load of data transmission and affect the data transmission rate.
[0046] In one embodiment, an interceptor is pre-configured in the data sending end to monitor the data sending end. When the data sending end processes all business logics to generate original transmission data, the interceptor intercepts the original transmission data to obtain the original transmission data to be transmitted to the data receiving end. For example, when the data sending end is a server, a servlet interceptor is configured in the server. When the server processes all business logics to generate original transmission data, the servlet interceptor intercepts the original transmission data to obtain the original transmission data to be transmitted to the data receiving end.
[0047] After obtaining the original transmission data, a preset string conversion algorithm is used to convert the original transmission data into a string object to generate string transmission data for subsequent recognition of constant data.
[0048] S102: Determine at least one type of constant data from the character string transmission data according to the data type of the original transmission data, and determine a data identifier corresponding to each constant data.
[0049] Among them, constant data refers to data items whose values cannot be changed during the program running process. For example, the Key value in the Key-Value pair can be a type of constant data. Since constant data usually generates a large amount of data in the original transmission data, and given that constant data is relatively stable
[0050] Characteristics, therefore, this embodiment replaces various types of constant data with data identifiers corresponding to various types of constant data to reduce the amount of original transmission data. Data identifier refers to identification information that can uniquely identify the corresponding constant data, that is, any type of constant data corresponds to a unique data identifier, and the constant data corresponding to the data identifier can be determined based on any data identifier.
[0051] In one embodiment, the data type of the original transmission data is determined, and a target method for determining constant data corresponding to the data type of the original transmission data is selected, and the target method is used to identify constant data of the string transmission data, so as to determine at least one type of constant data from the string transmission data. For example, assuming that the data type of the original transmission data is a JSON type, the method of identifying JSON object keyword data is used to identify constant data of the string transmission data.
[0052] After determining at least one type of constant data from the string transmission data, a preset identifier generation algorithm is used to generate identifiers for each type of constant data, and data identifiers corresponding to each type of constant data are determined based on the identifier generation results. For example, the number of times each type of constant data appears in the string transmission data is counted, and sequence encoding is performed based on the number of times it appears to determine the data identifiers corresponding to each type of constant data.
[0053] S103 . Replacing various types of constant data in the character string transmission data with data identifiers corresponding to the various types of constant data to generate replacement transmission data.
[0054] In one embodiment, according to the data identifiers corresponding to the various types of constant data, character replacement is performed on the various types of constant data, so as to replace the various types of constant data with the data identifiers corresponding to the various types of constant data. Exemplarily, assuming that any type of constant data is "adcd...xyz", and the data identifier corresponding to the type of constant data is "1234", all the constant data "adcd...xyz" in the string transmission data are replaced with "1234", and if there are three constant data "adcd...xyz" in the string transmission data, the three constant data "adcd...xyz" are replaced with "1234", and the replaced string transmission data is used as the replacement transmission data.
[0055] S104: Transmit the replaced transmission data and the identification corresponding information to the data receiving end, so that the data receiving end can restore the original transmission data according to the replaced transmission data and the identification corresponding information.
[0056] The identification corresponding information is generated according to various types of constant data and data identifications corresponding to the various types of constant data.
[0057] In one embodiment, various types of constant data are associated with data identifiers corresponding to the various types of constant data, and association information of the various types of constant data is generated, and the identification corresponding information is generated by combining the association information of the various types of constant data. For example, assuming that the constant data includes constant data A, constant data B, constant data C, and constant data D, and assuming that the corresponding data identifiers are "111", "222", "333", and "444", the association information "constant data A, 111", "constant data B, 222", "constant data C, 333", and "constant data D, 444" are generated respectively, and the identification corresponding information is generated according to the above four types of association information.
[0058] After the identification corresponding information is generated, the replacement transmission data and the identification corresponding information are transmitted to the data receiving end. It can be understood that since the identification corresponding information is generated according to various types of constant data and the data identifications corresponding to the various types of constant data, the data receiving end can determine the association between various types of constant data and the data identifications according to the identification corresponding information, so that the original transmission data can be restored by replacing each data identification back to the corresponding constant data on the basis of the replacement transmission data.
[0059] Since the amount of constant data is usually much larger than the amount of data identifiers, the present invention generates replacement transmission data by replacing various types of constant data with data identifiers corresponding to the various types of constant data in the character string transmission data; the replacement transmission data and the identification corresponding information are transmitted to the data receiving end, so that the data receiving end can restore the original transmission data based on the replacement transmission data and the identification corresponding information; wherein the identification corresponding information is generated based on the various types of constant data and the data identifiers corresponding to the various types of constant data, and under the premise of ensuring that the data receiving end can restore the original transmission data, since the constant data is replaced with the corresponding data identifier for transmission, the data transmission amount is undoubtedly greatly reduced, the load of data transmission between the data sending end and the data receiving end is alleviated, and the data transmission rate is improved.
[0060] Embodiment 2
[0061] Figure 2 This is a flowchart of a method for processing transmission data provided by Embodiment 2 of the present invention. This embodiment further expands and optimizes the above embodiment and can be combined with the above optional implementations. Figure 2 As shown, the method includes:
[0062] S201. Acquire original transmission data to be transmitted to a data receiving end, and convert the original transmission data into a string object to generate string transmission data.
[0063] S202: When the data type is a JSON type, determine at least one JSON object included in the string transmission data, and use keyword data included in each JSON object as constant data.
[0064] The keyword data represents the Key value in the JSON object.
[0065] In one embodiment, the data type of the original transmission data is identified, and when it is determined that the data type of the original transmission data is of JSON type, the string transmission data is converted into a JSONArray, and then at least one JSON object included in the string transmission data is determined by traversing the JSONArray, and then each JSON object is traversed to extract the keyword data of each JSON object as constant data. For example, assuming that any JSON object includes two keyword data, two constant data are extracted and generated.
[0066] Optionally, the determined constant data may be recorded in a constant area in the serialized format data structure.
[0067] S203, determining the number of times each keyword data appears in the string transmission data, taking the number of times each type of constant data appears in the string transmission data, performing sequence encoding according to the number of times, and using the sequence encoding result as the data identifier corresponding to the constant data.
[0068] In one implementation, the number of times each keyword data appears in the string transmission data is counted, and the number of times each keyword data appears in the string transmission data is determined as the number of times each type of constant data appears in the string transmission data. For example, assuming that the number of times keyword data A appears in the string transmission data is 5 times, since keyword data A itself is a type of constant data, the number of times the constant data represented by keyword data A appears in the string transmission data is determined to be 5 times.
[0069] According to the number of times each type of constant data appears in the string transmission data, sequence encoding is performed, and the sequence encoding results calculated are used as the data identifiers corresponding to the corresponding constant data. Among them, since the sequence encoding results are unique, they are used as the data identifiers corresponding to each type of constant data. For example, assuming that the number of times any type of constant data appears in the string transmission data is X1, assuming that the result of sequence encoding of X1 is Y, then Y is used as the data identifier corresponding to this type of constant data.
[0070] When the data type is JSON type, at least one JSON object included in the string transmission data is determined; the keyword data contained in each JSON object is used as constant data; the number of times each keyword data appears in the string transmission data is determined as the number of times each type of constant data appears in the string transmission data; sequence encoding is performed according to the number of times, and the sequence encoding result is used as the data identifier corresponding to the constant data. On the one hand, the effect of identifying the constant data in the original transmission data of the JSON type is achieved; on the other hand, the number of times each constant data appears in the string transmission data is used for sequence encoding, and the data identifier is determined according to the sequence encoding result, thereby providing a specific implementation method for determining the data identifier and ensuring the uniqueness of the data identifier.
[0071] S204. Replacing various types of constant data in the character string transmission data with data identifiers corresponding to the various types of constant data to generate replacement transmission data.
[0072] S205: Transmit the replaced transmission data and the identification corresponding information to the data receiving end, so that the data receiving end can restore the original transmission data according to the replaced transmission data and the identification corresponding information.
[0073] The identification corresponding information is generated according to various types of constant data and data identifications corresponding to the various types of constant data.
[0074] Optionally, each type of constant data in the string transmission data is replaced with a data identifier corresponding to each type of constant data, and before generating the replacement transmission data, the following is further included:
[0075] The constant data that appears in the string transmission data a number of times less than or equal to the number threshold is used as the first constant data to be removed; the first constant data to be removed is removed from various types of constant data, and the first constant data to be replaced is obtained according to the removal result.
[0076] In one embodiment, various types of constant data are screened according to the number of times they appear in the string transmission data, and the constant data whose number of appearances is less than or equal to the number threshold is used as the first constant data to be removed. The number threshold can be set and adjusted according to actual business needs, and the number threshold can be optionally set to "1", that is, the constant data whose number of appearances is less than or equal to 1 is used as the first constant data to be removed.
[0077] Among all the determined constant data, the first constant data to be removed is removed, and the remaining constant data is used as the first constant data to be replaced. That is, the constant data determined as the first constant data to be removed will be transmitted according to the original data style, and the data identifier will no longer be replaced. Correspondingly, the constant data determined as the first constant data to be replaced will not be transmitted according to the original data style, but will be transmitted after the data identifier is replaced.
[0078] Optionally, various types of constant data are replaced with data identifiers corresponding to the various types of constant data in the string transmission data to generate replacement transmission data, including:
[0079] In the character string transmission data, each type of first constant data to be replaced is replaced with a data identifier corresponding to each type of first constant data to be replaced, so as to generate replacement transmission data.
[0080] In one implementation, if any type of constant data is determined to be the first constant data to be replaced, the constant data of this type (the first constant data to be replaced) is replaced with a data identifier corresponding to the constant data of this type to generate replacement transmission data.
[0081] Since the number of times the first constant data to be removed appears in the string transmission data is less than or equal to the number threshold, that is, the frequency of the first constant data to be removed in the string transmission data is very low and will not generate too much data, it does not have the meaning of data identifier replacement. Therefore, by removing the first constant data to be removed from various types of constant data and obtaining the first constant data to be replaced according to the removal result, each type of first constant data to be replaced in the string transmission data is replaced with the data identifier corresponding to each type of first constant data to be replaced, and replacement transmission data is generated, thereby omitting meaningless data identifier replacement, speeding up the generation efficiency of replacement transmission data, and still ensuring that the data volume of the replacement transmission data is still much smaller than the data volume of the original transmission data.
[0082] Optionally, each type of constant data in the string transmission data is replaced with a data identifier corresponding to each type of constant data, and before generating the replacement transmission data, the following is further included:
[0083] The constant data whose character length is less than the length threshold is used as the second constant data to be removed; the second constant data to be removed is removed from various types of constant data, and the second constant data to be replaced is obtained according to the removal result.
[0084] In one implementation, various types of constant data are screened according to their character lengths, and constant data with character lengths less than a length threshold are used as the second constant data to be removed. The length threshold can be set and adjusted according to actual business needs, and the length threshold can be optionally set to "4 bytes", that is, constant data with character lengths less than 4 bytes are used as the second constant data to be removed.
[0085] Among all the determined constant data, the second constant data to be removed is removed, and the remaining constant data is used as the second constant data to be replaced. That is, the constant data determined to be the second constant data to be removed will be transmitted according to the original data style, and the data identifier will no longer be replaced. Correspondingly, the constant data determined to be the second constant data to be replaced will not be transmitted according to the original data style, but will be transmitted after the data identifier is replaced.
[0086] Optionally, various types of constant data are replaced with data identifiers corresponding to the various types of constant data in the string transmission data to generate replacement transmission data, including:
[0087] In the character string transmission data, each type of second constant data to be replaced is replaced with a data identifier corresponding to each type of second constant data to be replaced, so as to generate replacement transmission data.
[0088] In one implementation, if any type of constant data is determined to be the second constant data to be replaced, the constant data of this type (the second constant data to be replaced) is replaced with a data identifier corresponding to the constant data of this type to generate replacement transmission data.
[0089] Since the character length of the second constant data to be removed is less than the length threshold, that is, the character length of the second constant data to be removed is very short and will not generate too much data, it does not make sense to replace the data identifier. Therefore, the second constant data to be removed is removed from various types of constant data, and the second constant data to be replaced is obtained according to the removal result, and each type of second constant data to be replaced is replaced with the data identifier corresponding to each type of second constant data to be replaced in the character string transmission data to generate replacement transmission data, thereby omitting meaningless data identifier replacement, speeding up the generation efficiency of replacement transmission data, and still ensuring that the data volume of the replacement transmission data is still much smaller than the data volume of the original transmission data.
[0090] Embodiment 3
[0091] Figure 3 This is a flowchart of a method for processing transmission data provided by Embodiment 3 of the present invention. This embodiment further expands and optimizes the above embodiment and can be combined with the above optional implementation modes. Figure 3 As shown, the method includes:
[0092] S301, obtaining original transmission data to be transmitted to a data receiving end, and converting the original transmission data into a string object to generate string transmission data.
[0093] S302: When the data type is other types, perform word segmentation processing on the string transmission data to obtain at least one data word segmentation result, and use the data word segmentation result as the constant data.
[0094] Among them, other types are data types except JSON type.
[0095] In one embodiment, the data type of the original transmission data is identified, and when it is determined that the data type of the original transmission data is a data type other than the JSON type, a natural language model is used to perform word segmentation processing on the string transmission data to obtain at least one data segmentation result. Statistics are collected on each data segmentation result, and the same data segmentation results are regarded as a type of constant data.
[0096] S303, determining the number of times each data segmentation result appears in the string transmission data, taking the number of times each type of constant data appears in the string transmission data, performing sequence encoding according to the number of times, and using the sequence encoding result as the data identifier corresponding to the constant data.
[0097] In one implementation, the number of times each data segmentation result appears in the string transmission data is counted, and the number of times each data segmentation result appears in the string transmission data is determined as the number of times each type of constant data appears in the string transmission data. For example, assuming that the number of times data segmentation result A appears in the string transmission data is 5 times, since data segmentation result A itself is a type of constant data, the number of times the constant data represented by data segmentation result A appears in the string transmission data is determined to be 5 times.
[0098] According to the number of times each type of constant data appears in the string transmission data, sequence encoding is performed, and the sequence encoding results calculated are used as the data identifiers corresponding to the corresponding constant data. Among them, since the sequence encoding results are unique, they are used as the data identifiers corresponding to each type of constant data. For example, assuming that the number of times any type of constant data appears in the string transmission data is X1, assuming that the result of sequence encoding of X1 is Y, then Y is used as the data identifier corresponding to this type of constant data.
[0099] By performing word segmentation processing on the string transmission data when the data type is other types, at least one data segmentation result is obtained; wherein the other types are data types other than the JSON type; the data segmentation result is used as constant data; the number of times each data segmentation result appears in the string transmission data is determined as the number of times each type of constant data appears in the string transmission data; sequence encoding is performed according to the number of times, and the sequence encoding result is used as the data identifier corresponding to the constant data. On the one hand, the effect of identifying the constant data in the original transmission data of other data types except the JSON type is achieved; on the other hand, the number of times each constant data appears in the string transmission data is used for sequence encoding, and the data identifier is determined according to the sequence encoding result, thereby providing a specific implementation method for determining the data identifier and ensuring the uniqueness of the data identifier.
[0100] S304. Replacing various types of constant data in the character string transmission data with data identifiers corresponding to the various types of constant data to generate replacement transmission data.
[0101] S305: Transmit the replaced transmission data and the identification corresponding information to the data receiving end, so that the data receiving end can restore the original transmission data according to the replaced transmission data and the identification corresponding information.
[0102] The identification corresponding information is generated according to various types of constant data and data identifications corresponding to the various types of constant data.
[0103] Optionally, before transmitting the replacement transmission data and the identification corresponding information to the data receiving end, the method further includes:
[0104] 1) When the data receiving end is a Java application, character information of target digits in each type of constant data is extracted starting from the first character of each type of constant data, and used as compressed data corresponding to each type of constant data.
[0105] The target number of bits is the shortest number of bits to ensure that there is no duplication between various types of compressed data.
[0106] Exemplarily, the first characters of each type of constant data are uniformly extracted, and it is determined whether the first characters are repeated. If not, "one digit" is used as the target number of digits, and the first character of each type of constant data is used as the compressed data corresponding to the constant data; if so, the first two characters are extracted, and it is determined again whether the first two characters are repeated. If not, "two digits" is used as the target number of digits, and the first two characters of each type of constant data are used as the compressed data corresponding to the constant data; if so, the first three characters are extracted... and so on, until the character information of the target number of digits is extracted as the compressed data corresponding to the constant data.
[0107] 2) The compressed data corresponding to each type of constant data and the data identifiers corresponding to each type of constant data are respectively associated to generate first associated information corresponding to each type of constant data.
[0108] Exemplarily, assuming that any constant data is "abcdefg...", the compressed data corresponding to the constant data is "ab", and the data identifier corresponding to the constant data is "123", then "ab" and "123" are associated to generate the first association information corresponding to the constant data.
[0109] 3) Each piece of first associated information is concatenated to generate identification corresponding information.
[0110] In one implementation, the first associated information is concatenated by a concatenation symbol, such as a concatenation symbol &, to generate identification corresponding information. The concatenation symbol is used to enable the data receiving end to distinguish different first associated information.
[0111] Exemplarily, assuming that the first associated information includes "ab+123", "cd+456" and "ef+789", the concatenation symbol & is used to concatenate them to obtain ab+123&cd+456&ef+789 as the identification corresponding information.
[0112] Since the identification corresponding information will also be transmitted to the data receiving end as transmission data, the identification corresponding information will also have more or less data volume. Therefore, when the data receiving end is a Java application, the character information of the target number of bits in each type of constant data is extracted starting from the first character of each type of constant data, and used as the compressed data corresponding to each type of constant data; wherein the target number of bits is the shortest number of bits to ensure that there is no duplication between the various types of compressed data; the compressed data corresponding to each type of constant data and the data identification corresponding to each type of constant data are respectively associated to generate the first associated information corresponding to each type of constant data; each first associated information is spliced to generate the identification corresponding information, thereby replacing the various types of constant data in the identification corresponding information with their respective corresponding compressed data, further reducing the data volume of the identification corresponding information, and achieving the effect of further alleviating the load of data transmission between the data sending end and the data receiving end on the premise of ensuring that the data receiving end can restore the original transmission data.
[0113] Optionally, before transmitting the replacement transmission data and the identification corresponding information to the data receiving end, the method further includes:
[0114] When the data receiving end is a non-Java application, various constant data and data identifiers corresponding to the various constant data are associated respectively to generate second associated information corresponding to the various constant data; and the second associated information are concatenated to generate identifier corresponding information.
[0115] In one implementation, if it is determined that the data receiving end is a non-Java application, each type of constant data and the data identifier corresponding to each type of constant data are associated respectively to generate second association information corresponding to each type of constant data. Exemplarily, assuming that any constant data is "abcdefg" and the data identifier corresponding to the constant data is "1", "abcdefg" is associated with "1" to generate the second association information corresponding to the constant data.
[0116] Each second association information is spliced by a splicing symbol, such as a splicing symbol &, to generate identification corresponding information. The splicing symbol is used to enable the data receiving end to distinguish different second association information.
[0117] Exemplarily, assuming that the second association information includes "abcd+1", "efgh+2" and "ijkl+3", the concatenation symbol & is used for concatenation to obtain abcd+1&efgh+2&ijkl+3 as the identification corresponding information.
[0118] When the data receiving end is a non-Java application, various types of constant data and data identifiers corresponding to the various types of constant data are respectively associated to generate second association information corresponding to the various types of constant data; each second association information is spliced to generate identification corresponding information. Since the identification corresponding information completely records the second association information of various types of constant data and the data identifiers corresponding to the various types of constant data, the recoverability of the original transmission data is guaranteed, so that the non-Java application can directly, correctly and accurately recover the original transmission data.
[0119] Embodiment 4
[0120] Figure 4 This is a flowchart of a method for processing transmission data provided by the fourth embodiment of the present invention. This embodiment is applicable to the case where a data receiving end receives replacement transmission data and identification corresponding information sent by a data sending end, and restores the original transmission data accordingly. The method can be executed by a transmission data processing device, which is configured at the data receiving end, such as a server or a client, and can be implemented in the form of hardware and / or software. Figure 4 As shown, the method includes:
[0121] S401: Receive replacement transmission data and identification corresponding information sent by a data sending end.
[0122] The identification corresponding information is generated according to various types of constant data and data identifications corresponding to the various types of constant data.
[0123] S402: Replace each data identifier included in the replacement transmission data with constant data corresponding to each data identifier according to the identifier corresponding information, and generate character string transmission data.
[0124] In one embodiment, the data receiving end parses the identifier corresponding information to determine various types of constant data and data identifiers corresponding to various types of constant data. For the replacement transmission data, each data identifier is replaced according to the data identifier corresponding to each type of constant data, so as to replace each data identifier included in the replacement transmission data with the constant data corresponding to each data identifier. Exemplarily, assuming that according to the analysis result of the identifier corresponding information, it is determined that any type of constant data is "adcd...xyz", and the data identifier corresponding to this type of constant data is "1234", then all data identifiers "1234" in the replacement transmission data are replaced with "adcd...xyz", and if there are three data identifiers "1234" in the string transmission data, then these three data identifiers "1234" are replaced with "adcd...xyz", and the replaced replacement transmission data is used as the string transmission data.
[0125] S403: Perform type conversion on the character string transmission data according to the data type of the original transmission data to obtain the original transmission data.
[0126] In one implementation, the data type of the original transmission data is determined, and the character string transmission data is type-converted according to the data type of the original transmission data, so that the converted data conforms to the data type of the original transmission data, thereby obtaining the original transmission data.
[0127] Since the amount of constant data is usually much larger than the amount of data identifiers, the present invention receives replacement transmission data and identifier corresponding information sent by a data sending end; wherein the identifier corresponding information is generated according to various types of constant data and data identifiers corresponding to various types of constant data; according to the identifier corresponding information, each data identifier included in the replacement transmission data is replaced with constant data corresponding to each data identifier to generate string transmission data; according to the data type of the original transmission data, the string transmission data is type-converted to obtain the original transmission data. Under the premise of ensuring that the data receiving end can restore the original transmission data, since the constant data is replaced with the corresponding data identifier for transmission, the data transmission amount is undoubtedly greatly reduced, the load of data transmission between the data sending end and the data receiving end is alleviated, and the data transmission rate is improved.
[0128] Optionally, when the data receiving end is a Java application, each data identifier included in the replacement transmission data is replaced with constant data corresponding to each data identifier according to the identifier corresponding information to generate string transmission data, including:
[0129] 1) Parse the information corresponding to the identifier to obtain at least one first associated information.
[0130] Each first association information is generated by associating the compressed data corresponding to each type of constant data and the data identifier corresponding to each type of constant data.
[0131] 2) According to each first association information, each type of compressed data is input into a constant data prediction model to determine the constant data corresponding to each type of compressed data.
[0132] Among them, the constant data prediction model is trained based on sample compression data and sample constant data.
[0133] In one embodiment, the compressed data carried in each first associated information obtained by analysis is obtained, and each compressed data is input into a constant data prediction model, the constant data corresponding to the compressed data is predicted using the constant data prediction model, and the constant data corresponding to each type of compressed data is output.
[0134] Optionally, the constant data prediction model can be trained as follows:
[0135] After the Java program is started, all entity classes marked with preset identifiers are automatically scanned, and the attribute information of all entity classes is extracted through the reflection extraction method;
[0136] Extracting target digits of character information from each of the attribute information starting from the first character of each of the attribute information, and using them as compressed information corresponding to each of the attribute information; wherein the target digits are the shortest digits to ensure that there is no duplication between the compressed information;
[0137] Determine the entity class name corresponding to each of the entity classes, use each of the compressed information as sample compressed data, and use each of the entity class names and each of the attribute information as sample constant data to perform machine learning model training to generate a constant data prediction model.
[0138] Optionally, various types of compressed data are input into a constant data prediction model to determine constant data corresponding to various types of compressed data, including:
[0139] Input any compressed data into the constant data prediction model, and use the k-nearest neighbor algorithm to identify the corresponding entity class name and attribute information in the constant data prediction model, and use the identified entity class name and attribute information together as the constant data corresponding to the compressed data.
[0140] 3) Each data identifier included in the replacement transmission data is replaced with constant data corresponding to each data identifier to generate character string transmission data.
[0141] By parsing the information corresponding to the identifier when the data receiving end is a Java application, at least one first associated information is obtained; wherein, each first associated information is generated by associating the compressed data corresponding to each constant data and the data identifiers corresponding to each constant data; according to each first associated information, each compressed data is input into a constant data prediction model to determine the constant data corresponding to each compressed data; wherein, the constant data prediction model is trained based on sample compressed data and sample constant data; each data identifier included in the replacement transmission data is replaced with the constant data corresponding to each data identifier to generate string transmission data, and since the constant data corresponding to the compressed data is predicted by the constant data prediction model, it is ensured that the constant data can be accurately and completely obtained, and the accuracy and completeness of the generation of the string transmission data are further ensured.
[0142] Optionally, when the data receiving end is a non-Java application, each data identifier included in the replacement transmission data is replaced with constant data corresponding to each data identifier according to the identifier corresponding information to generate string transmission data, including:
[0143] Parse the information corresponding to the identifier to obtain at least one second associated information; wherein each second associated information is generated by associating various types of constant data and the data identifiers corresponding to the various types of constant data; according to each second associated information, replace each data identifier included in the replacement transmission data with the constant data corresponding to each data identifier to generate string transmission data.
[0144] For example, assuming that the corresponding information of the identifier is "abcdefg+1&hijklmn+2&opqrstu+3", and the second associated information obtained by parsing includes "abcdefg+1", "hijklmn+2" and "opqrstu+3", then all data identifiers "1" in the replacement transmission data are replaced with "abcdefg", all data identifiers "2" are replaced with "hijklmn", and all data identifiers "3" are replaced with "opqrstu".
[0145] By parsing the information corresponding to the identifier when the data receiving end is a non-Java application, at least one second associated information is obtained; wherein each second associated information is associated and generated according to each type of constant data and the data identifiers corresponding to each type of constant data; according to each second associated information, each data identifier included in the replacement transmission data is replaced with the constant data corresponding to each data identifier to generate string transmission data, thereby ensuring the recoverability of the string transmission data, so that the non-Java application can directly, correctly and accurately recover the string transmission data.
[0146] Embodiment 5
[0147] Figure 5 This is a schematic diagram of the structure of a data transmission processing device provided in Embodiment 5 of the present invention, wherein the data transmission processing device is configured at the data transmitting end. Figure 5 As shown, the device comprises:
[0148] The string transmission data generating module 51 is used to obtain the original transmission data to be transmitted to the data receiving end, and convert the original transmission data into a string object to generate the string transmission data;
[0149] The constant data determination module 52 is used to determine at least one type of constant data from the character string transmission data according to the data type of the original transmission data, and determine the data identifier corresponding to each constant data;
[0150] The replacement transmission data generating module 53 is used to replace various types of constant data in the character string transmission data with data identifiers corresponding to the various types of constant data, so as to generate replacement transmission data;
[0151] The data sending module 54 is used to transmit the replacement transmission data and the identification corresponding information to the data receiving end, so that the data receiving end can restore the original transmission data based on the replacement transmission data and the identification corresponding information; wherein the identification corresponding information is generated based on various types of constant data and the data identifications corresponding to the various types of constant data.
[0152] Optionally, the constant data determination module 52 is specifically configured to:
[0153] In the case where the data type is a JSON type, determining at least one JSON object included in the string transmission data;
[0154] The keyword data contained in each JSON object is used as constant data;
[0155] The constant data determination module 52 is further specifically used for:
[0156] Determine the number of times each keyword data appears in the string transmission data, as the number of times each type of constant data appears in the string transmission data;
[0157] The sequence encoding is performed according to the number of times, and the sequence encoding result is used as the data identifier corresponding to the constant data.
[0158] Optionally, the constant data determination module 52 is further configured to:
[0159] When the data type is other types, perform word segmentation processing on the string transmission data to obtain at least one data word segmentation result; wherein the other types are data types other than the JSON type;
[0160] Use the data segmentation results as constant data;
[0161] The constant data determination module 52 is further specifically used for:
[0162] Determine the number of times each data segmentation result appears in the string transmission data, as the number of times each type of constant data appears in the string transmission data;
[0163] The sequence encoding is performed according to the number of times, and the sequence encoding result is used as the data identifier corresponding to the constant data.
[0164] Optionally, the device further includes a first association information generating module, which is specifically configured to:
[0165] When the data receiving end is a Java application, character information of target digits in each type of constant data is extracted from the first character of each type of constant data, and the characters are used as compressed data corresponding to each type of constant data; wherein the target digits are the shortest digits to ensure that there is no duplication between the compressed data of each type;
[0166] Respectively associating the compressed data corresponding to each type of constant data and the data identifiers corresponding to each type of constant data to generate first association information corresponding to each type of constant data;
[0167] Each piece of first association information is concatenated to generate identification corresponding information.
[0168] Optionally, the device further includes a second association information generating module, which is specifically configured to:
[0169] When the data receiving end is a non-Java application, each type of constant data and the data identifier corresponding to each type of constant data are associated to generate second association information corresponding to each type of constant data;
[0170] Each piece of second association information is concatenated to generate identification corresponding information.
[0171] Optionally, the device further includes a first constant data removal module, specifically configured to:
[0172] The constant data that appears in the string transmission data a number of times less than or equal to the number threshold is used as the first constant data to be removed;
[0173] Removing first constant data to be removed from various types of constant data, and obtaining first constant data to be replaced according to the removal result;
[0174] Replace the transmission data generating module 53, specifically used for:
[0175] In the character string transmission data, each type of first constant data to be replaced is replaced with a data identifier corresponding to each type of first constant data to be replaced, so as to generate replacement transmission data.
[0176] Optionally, the device further includes a second constant data removal module, specifically configured to:
[0177] The constant data whose character length is less than the length threshold is used as the second constant data to be removed;
[0178] Removing the second constant data to be removed from various types of constant data, and obtaining the second constant data to be replaced according to the removal result;
[0179] The replacement transmission data generating module 53 is further used for:
[0180] In the character string transmission data, each type of second constant data to be replaced is replaced with a data identifier corresponding to each type of second constant data to be replaced, so as to generate replacement transmission data.
[0181] The transmission data processing device provided in the embodiment of the present invention can execute the transmission data processing method provided in embodiments 1 to 3 of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0182] Embodiment 6
[0183] Figure 6 This is a schematic diagram of the structure of a data transmission processing device provided by Embodiment 6 of the present invention, wherein the data transmission processing device is configured at a data receiving end. Figure 6 As shown, the device comprises:
[0184] The data receiving module 61 is used to receive the replacement transmission data and the identification corresponding information sent by the data sending end; wherein the identification corresponding information is generated according to various types of constant data and the data identifications corresponding to the various types of constant data;
[0185] The string transmission data generating module 62 is used to replace each data identifier included in the replacement transmission data with constant data corresponding to each data identifier according to the identifier corresponding information, so as to generate the string transmission data;
[0186] The original transmission data acquisition module 63 is used to perform type conversion on the character string transmission data according to the data type of the original transmission data to acquire the original transmission data.
[0187] Optionally, when the data receiving end is a Java application, the string transmission data generating module 62 is specifically used to:
[0188] Parsing the information corresponding to the identifier to obtain at least one first associated information; wherein each first associated information is generated by associating the compressed data corresponding to each type of constant data and the data identifier corresponding to each type of constant data;
[0189] According to each first association information, each type of compressed data is input into a constant data prediction model to determine constant data corresponding to each type of compressed data; wherein the constant data prediction model is trained based on sample compressed data and sample constant data;
[0190] Each data identifier included in the replacement transmission data is replaced with constant data corresponding to each data identifier to generate character string transmission data.
[0191] Optionally, when the data receiving end is a non-Java application, the string transmission data generating module 62 is specifically used to:
[0192] Parsing the information corresponding to the identifier to obtain at least one second associated information; wherein each second associated information is generated by associating various constant data and data identifiers corresponding to various constant data;
[0193] According to each second association information, each data identifier included in the replacement transmission data is replaced with constant data corresponding to each data identifier to generate character string transmission data.
[0194] The transmission data processing device provided in the embodiment of the present invention can execute the transmission data processing method provided in the fourth embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0195] Embodiment 7
[0196] Figure 7 A schematic diagram of an electronic device 70 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0197] like Figure 7As shown, the electronic device 70 includes at least one processor 71, and a memory connected to the at least one processor 71 in communication, such as a read-only memory (ROM) 72, a random access memory (RAM) 73, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 71 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 72 or the computer program loaded from the storage unit 78 to the random access memory (RAM) 73. In the RAM 73, various programs and data required for the operation of the electronic device 70 can also be stored. The processor 71, the ROM 72, and the RAM 73 are connected to each other via a bus 74. An input / output (I / O) interface 75 is also connected to the bus 74.
[0198] A number of components in the electronic device 70 are connected to the I / O interface 75, including: an input unit 76, such as a keyboard, a mouse, etc.; an output unit 77, such as various types of displays, speakers, etc.; a storage unit 78, such as a disk, an optical disk, etc.; and a communication unit 79, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 79 allows the electronic device 70 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0199] The processor 71 may be a variety of general and / or dedicated processing components with processing and computing capabilities. Some examples of the processor 71 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 71 performs the various methods and processes described above, such as a processing method for transmitting data.
[0200] In some embodiments, the method for processing transmission data may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 78. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 70 via the ROM 72 and / or the communication unit 79. When the computer program is loaded into the RAM 73 and executed by the processor 71, one or more steps of the method for processing transmission data described above may be performed. Alternatively, in other embodiments, the processor 71 may be configured to execute the method for processing transmission data in any other appropriate manner (e.g., by means of firmware).
[0201] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0202] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0203] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0204] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0205] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0206] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0207] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0208] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for processing transmission data, characterized in that: Executed by the data sender, including: Acquire original transmission data to be transmitted to a data receiving end, and convert the original transmission data into a string object to generate string transmission data; wherein the original transmission data is business voucher data or financial statement data; Determine at least one type of constant data from the string transmission data according to the data type of the original transmission data, and determine the data identifier corresponding to each of the constant data; Replacing each type of the constant data in the character string transmission data with a data identifier corresponding to each type of the constant data to generate replacement transmission data; Transmitting the replacement transmission data and the identification corresponding information to the data receiving end, so that the data receiving end can restore the original transmission data according to the replacement transmission data and the identification corresponding information; wherein the identification corresponding information is generated according to each type of the constant data and the data identification corresponding to each type of the constant data; Wherein, an interceptor is pre-configured in the data sending end, and the obtaining of the original transmission data to be transmitted to the data receiving end includes: The data sending end is monitored by the interceptor, and when the data sending end processes all business logics to generate the original transmission data, the original transmission data is intercepted by the interceptor to obtain the original transmission data to be transmitted to the data receiving end; The determining at least one type of constant data from the string transmission data according to the data type of the original transmission data comprises: In the case where the data type is a JSON type, converting the string transmission data into a JSONArray, and determining at least one JSON object included in the string transmission data by traversing the JSONArray; By traversing each JSON object, keyword data of each JSON object is extracted as the constant data; The determining of the data identifier corresponding to each of the constant data includes: Determine the number of times each keyword data appears in the string transmission data, as the number of times each type of constant data appears in the string transmission data; Perform sequence encoding respectively according to the number of times, and use the sequence encoding result as the data identifier corresponding to the constant data; Before transmitting the replacement transmission data and the identification corresponding information to the data receiving end, the method further includes: When the data receiving end is a Java application, character information of target digits in each type of the constant data is extracted from the first character of each type of the constant data, and the characters are used as compressed data corresponding to each type of the constant data; wherein the target digits are the shortest digits to ensure that there is no duplication between the compressed data of each type; Respectively associating the compressed data corresponding to each type of the constant data and the data identifiers corresponding to each type of the constant data to generate first association information corresponding to each type of the constant data; The first association information is concatenated to generate the identification corresponding information.
2. The method according to claim 1, characterized in that The determining at least one type of constant data from the string transmission data according to the data type of the original transmission data comprises: When the data type is other types, the string transmission data is segmented to obtain at least one data segmentation result; wherein the other types are data types other than the JSON type; Using the data segmentation result as the constant data; The determining of the data identifier corresponding to each of the constant data includes: Determine the number of times each of the data segmentation results appears in the string transmission data, as the number of times each of the constant data appears in the string transmission data; Sequence encoding is performed respectively according to the number of times, and the sequence encoding result is used as the data identifier corresponding to the constant data.
3. The method according to claim 1, before transmitting the replacement transmission data and the identification corresponding information to the data receiving end, further comprising: In the case where the data receiving end is a non-Java application, each type of the constant data and the data identifier corresponding to each type of the constant data are respectively associated to generate second association information corresponding to each type of the constant data; The second association information is concatenated to generate the identification corresponding information.
4. The method according to claim 1 or 2, wherein each type of constant data in the string transmission data is replaced with a data identifier corresponding to each type of constant data, and before generating the replacement transmission data, the method further comprises: The constant data that appears in the string transmission data a number of times that is less than or equal to a number threshold is used as the first constant data to be removed; Removing the first constant data to be removed from the various types of constant data, and obtaining the first constant data to be replaced according to the removal result; The step of replacing each type of constant data in the string transmission data with a data identifier corresponding to each type of constant data to generate replacement transmission data includes: In the character string transmission data, each type of the first constant data to be replaced is replaced with a data identifier corresponding to each type of the first constant data to be replaced, to generate replacement transmission data.
5. The method according to claim 1 or 2, wherein each type of constant data in the string transmission data is replaced with a data identifier corresponding to each type of constant data, and before generating the replacement transmission data, the method further comprises: The constant data whose character length is less than the length threshold is used as the second constant data to be removed; Removing the second constant data to be removed from each type of constant data, and obtaining the second constant data to be replaced according to the removal result; The step of replacing each type of constant data in the string transmission data with a data identifier corresponding to each type of constant data to generate replacement transmission data includes: In the character string transmission data, each type of the second constant data to be replaced is replaced with a data identifier corresponding to each type of the second constant data to be replaced, to generate replacement transmission data.
6. A method for processing transmission data, characterized in that: Executed by the data receiving end, including: Receiving the replacement transmission data and the identification corresponding information sent by the data sending end; wherein the identification corresponding information is generated according to various types of constant data and the data identifications corresponding to the various types of constant data; According to the identifier corresponding information, each of the data identifiers included in the replacement transmission data is replaced with the constant data corresponding to each of the data identifiers to generate character string transmission data; Performing type conversion on the string transmission data according to the data type of the original transmission data to obtain the original transmission data; wherein the original transmission data is business voucher data or financial statement data; The original transmission data is obtained in the following manner: The data sending end monitors the data sending end through an interceptor, and when the data sending end processes all business logics to generate the original transmission data, the data sending end intercepts the original transmission data through the interceptor to obtain the original transmission data to be transmitted to the data receiving end; The constant data is determined as follows: In the case where the data type is a JSON type, the data sending end converts the string transmission data into a JSONArray, and determines at least one JSON object included in the string transmission data by traversing the JSONArray; The data sending end extracts keyword data of each JSON object as the constant data by traversing each JSON object; The data identifier is determined in the following manner: The data transmitting end determines the number of times each keyword data appears in the string transmission data as the number of times each type of constant data appears in the string transmission data; The data transmitting end performs sequence encoding respectively according to the number of times, and uses the sequence encoding result as the data identifier corresponding to the constant data; The identification corresponding information is generated in the following manner: In the case where the data receiving end is a Java application, the data sending end extracts character information of target digits in each type of the constant data starting from the first character of each type of the constant data, and respectively uses the compressed data corresponding to each type of the constant data; wherein the target digits are the shortest digits to ensure that there is no duplication between the compressed data of each type; Respectively associating the compressed data corresponding to each type of the constant data and the data identifiers corresponding to each type of the constant data to generate first association information corresponding to each type of the constant data; The first association information is concatenated to generate the identification corresponding information.
7. The method according to claim 6, characterized in that In the case where the data receiving end is a Java application, the step of replacing each data identifier included in the replacement transmission data with the constant data corresponding to each data identifier according to the identifier corresponding information to generate the string transmission data includes: Parsing the information corresponding to the identifier to obtain at least one first associated information; wherein each first associated information is generated by associating the compressed data corresponding to each type of the constant data and the data identifier corresponding to each type of the constant data; According to each of the first association information, each type of the compressed data is input into a constant data prediction model to determine the constant data corresponding to each type of the compressed data; wherein the constant data prediction model is trained based on the sample compressed data and the sample constant data; Each of the data identifiers included in the replacement transmission data is replaced with the constant data corresponding to each of the data identifiers to generate character string transmission data.
8. The method according to claim 6, characterized in that In the case where the data receiving end is a non-Java application, the step of replacing each data identifier included in the replacement transmission data with the constant data corresponding to each data identifier according to the identifier corresponding information to generate the string transmission data includes: Parsing the information corresponding to the identifier to obtain at least one second associated information; wherein each second associated information is generated by associating each type of the constant data and the data identifier corresponding to each type of the constant data; According to each of the second association information, each of the data identifiers included in the replacement transmission data is replaced with the constant data corresponding to each of the data identifiers to generate character string transmission data.
9. A data transmission processing device, characterized in that: Configured at the data sending end, including: A string transmission data generation module, used for acquiring original transmission data to be transmitted to a data receiving end, and converting the original transmission data into a string object to generate string transmission data; wherein the original transmission data is business voucher data or financial statement data; A constant data determination module, used to determine at least one type of constant data from the string transmission data according to the data type of the original transmission data, and determine a data identifier corresponding to each of the constant data; A replacement transmission data generation module, used for replacing each type of constant data in the string transmission data with a data identifier corresponding to each type of constant data, to generate replacement transmission data; A data sending module, used for transmitting the replacement transmission data and the identification corresponding information to the data receiving end, so that the data receiving end can restore the original transmission data according to the replacement transmission data and the identification corresponding information; wherein the identification corresponding information is generated according to each type of the constant data and the data identification corresponding to each type of the constant data; Wherein, an interceptor is pre-configured in the data sending end, and the string transmission data generating module is specifically used for: The data sending end is monitored by the interceptor, and when the data sending end processes all business logics to generate the original transmission data, the original transmission data is intercepted by the interceptor to obtain the original transmission data to be transmitted to the data receiving end; The constant data determination module is specifically used for: In the case where the data type is a JSON type, converting the string transmission data into a JSONArray, and determining at least one JSON object included in the string transmission data by traversing the JSONArray; By traversing each JSON object, keyword data of each JSON object is extracted as the constant data; The constant data determination module is further specifically used for: Determine the number of times each keyword data appears in the string transmission data, as the number of times each type of constant data appears in the string transmission data; Perform sequence encoding respectively according to the number of times, and use the sequence encoding result as the data identifier corresponding to the constant data; The first association information generating module is specifically used for: When the data receiving end is a Java application, character information of target digits in each type of the constant data is extracted from the first character of each type of the constant data, and the characters are used as compressed data corresponding to each type of the constant data; wherein the target digits are the shortest digits to ensure that there is no duplication between the compressed data of each type; Respectively associating the compressed data corresponding to each type of the constant data and the data identifiers corresponding to each type of the constant data to generate first association information corresponding to each type of the constant data; The first association information is concatenated to generate the identification corresponding information.
10. A data transmission processing device, characterized in that: Configured at the data receiving end, including: A data receiving module, used for receiving the replacement transmission data and the identification corresponding information sent by the data sending end; wherein the identification corresponding information is generated according to various types of constant data and the data identifications corresponding to the various types of constant data; A character string transmission data generating module, used for replacing each of the data identifiers included in the replacement transmission data with the constant data corresponding to each of the data identifiers according to the identifier corresponding information, so as to generate character string transmission data; An original transmission data acquisition module, used for performing type conversion on the string transmission data according to the data type of the original transmission data to acquire the original transmission data; wherein the original transmission data is business voucher data or financial statement data; The original transmission data is obtained in the following manner: The data sending end monitors the data sending end through an interceptor, and when the data sending end processes all business logics to generate the original transmission data, the data sending end intercepts the original transmission data through the interceptor to obtain the original transmission data to be transmitted to the data receiving end; The constant data is determined as follows: In the case where the data type is a JSON type, the data sending end converts the string transmission data into a JSONArray, and determines at least one JSON object included in the string transmission data by traversing the JSONArray; The data sending end extracts keyword data of each JSON object as the constant data by traversing each JSON object; The data identifier is determined in the following manner: The data transmitting end determines the number of times each keyword data appears in the string transmission data as the number of times each type of constant data appears in the string transmission data; The data transmitting end performs sequence encoding respectively according to the number of times, and uses the sequence encoding result as the data identifier corresponding to the constant data; The identification corresponding information is generated in the following manner: In the case where the data receiving end is a Java application, the data sending end extracts character information of target digits in each type of the constant data starting from the first character of each type of the constant data, and respectively uses the compressed data corresponding to each type of the constant data; wherein the target digits are the shortest digits to ensure that there is no duplication between the compressed data of each type; Respectively associating the compressed data corresponding to each type of the constant data and the data identifiers corresponding to each type of the constant data to generate first association information corresponding to each type of the constant data; The first association information is concatenated to generate the identification corresponding information.
11. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for processing transmission data described in any one of claims 1-5 and / or 6-8.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to execute the method for processing transmission data according to any one of claims 1-5 and / or 6-8.
13. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method for processing transmission data according to any one of claims 1-5 and / or 6-8.
