Data processing method and device

Abstract

The invention discloses a data processing method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: in response to a received data serialization request, performing first storage on target data according to a preset storage structure to obtain first general data; performing serialization processing on the first general data to obtain a serialization result of the target data; and in response to a received deserialization request for the serialization result, performing deserialization processing on the serialization result, and performing second storage on the deserialized data according to the storage structure to obtain second general data having the same data composition sequence as the first general data. According to the embodiment, the field sequence of the data is unified on the basis of the preset storage structure, risks existing in an existing serialization mode are avoided, and then the accuracy of data storage and network transmission is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for data processing. Background Art

[0002] Java data objects in memory usually need to be serialized into binary streams before data persistence and network transmission can be performed. Among the existing serialization methods, the Msgpack (MessagePack) serialization method is widely used because it can effectively reduce the amount of binary stream data. A smaller binary stream data volume can effectively save data space, increase the transmission rate, and thus improve the performance of data persistence and network transmission.

[0003] In the process of implementing the present invention, the inventors found that the prior art has the following problems:

[0004] Although Msgpack can effectively reduce the amount of data in the binary stream after serialization, it is necessary to ensure that the order of fields in the Java data object of the deserialization result is completely consistent with the order of fields in the Java data object before serialization. If it is inconsistent, it will cause deserialization exceptions, which will affect the persistence of data and the accuracy of data transmitted over the network. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and device for data processing, which unifies the field order of the constituent fields of the data based on a preset storage structure, avoids the risks existing when using Msgpack for serialization, and thus ensures the accuracy of data storage and network transmission.

[0006] To achieve the above object, according to one aspect of an embodiment of the present invention, a data processing method is provided, comprising:

[0007] In response to receiving the data serialization request, first storing the target data according to a preset storage structure to obtain first general data;

[0008] Performing serialization processing on the first general data to obtain a serialization result of the target data;

[0009] In response to receiving a deserialization request for the serialization result, the serialization result is deserialized, and the deserialized data is second-stored according to the storage structure to obtain second general data having the same data composition order as the first general data.

[0010] Optionally, the target data is of an object type, and the object type includes a field name and a field value; according to a preset storage structure, the target data is first stored, including: obtaining the field name and the corresponding field value of the target data; storing the field name into a preset index storage structure, and storing the field value corresponding to the field name into a storage list.

[0011] Optionally, before storing the field name into a preset index storage structure, the method also includes: assigning a field name identifier to the field name of the target data; storing the field name into a preset index storage structure, and storing the field value corresponding to the field name into a storage list, including: storing the field name into a tree graph structure according to the field name identifier of the field name; converting the tree graph structure to obtain an index storage structure storing the field name; storing the field value corresponding to the field name into the storage list according to the field name identifier of the field name.

[0012] Optionally, the method also includes: in response to receiving a deserialization result query request from a user, obtaining constituent elements of the deserialization result according to the deserialization result query request; according to the constituent elements of the deserialization result, searching for corresponding numerical values ​​from the second general data to obtain a deserialization result corresponding to the deserialization result query request.

[0013] Optionally, when the deserialization result is an object type, the component element is a field name; according to the component element of the deserialization result, searching for a corresponding numerical value from the second general data includes: according to the field name, searching for a corresponding field value from the second general data.

[0014] Optionally, the second general data includes an index storage structure for storing field names and a storage list for storing corresponding field values; according to the field name, the corresponding field value is found from the second general data, including: according to the field name, a matching matching field name is found from the index storage structure, and the matching field name has a field name identifier; according to the field name identifier, the corresponding field value is found from the storage list.

[0015] Optionally, before finding the corresponding field value from the second general data according to the field name, the method also includes: judging and determining that the data type of the field name is a basic data type; when the data type of the field name is a composite data type, the field name exists in the form of a set, and the field name is disassembled according to the basic data type so as to find the corresponding field value according to the disassembled field name.

[0016] According to a second aspect of an embodiment of the present invention, there is provided a data processing device, including:

[0017] A first general data acquisition module, configured to, in response to receiving a data serialization request, first store the target data according to a preset storage structure to obtain first general data;

[0018] A serialization processing module, used for performing serialization processing on the first general data to obtain a serialization result of the target data;

[0019] A second general data acquisition module is used to respond to receiving a deserialization request for the serialization result, deserialize the serialization result, and perform a second storage on the deserialized data according to the storage structure to obtain second general data having the same data composition order as the first general data.

[0020] According to a third aspect of an embodiment of the present invention, there is provided an electronic device for data processing, including:

[0021] one or more processors;

[0022] a storage device for storing one or more programs,

[0023] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.

[0024] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.

[0025] An embodiment of the invention has the following advantages or beneficial effects: in response to receiving a data serialization request, the target data is first stored according to a preset storage structure to obtain first general data; the first general data is serialized to obtain a serialization result of the target data; in response to receiving a deserialization request for the serialization result, the serialization result is deserialized, and the deserialized data is secondly stored according to the storage structure to obtain a technical solution of second general data having the same data composition order as the first general data, which realizes unifying the field order of the constituent fields of the data based on a preset storage structure, avoiding the risks existing when using Msgpack for serialization, and thereby ensuring the accuracy of data storage and network transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0027] Figure 1is a schematic diagram of the main flow of a data processing method according to an embodiment of the present invention;

[0028] Figure 2 It is a complete schematic diagram of the data processing principle of an embodiment of the present invention;

[0029] Figure 3 It is a detailed flowchart of the data processing method of an embodiment of the present invention;

[0030] Figure 4 is a schematic diagram of main modules of a data processing device according to an embodiment of the present invention;

[0031] Figure 5 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0032] Figure 6 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0034] Although the existing Msgpack serialization method can effectively reduce the amount of data in the binary stream after serialization, it needs to ensure that the field order in the Java data object of the deserialization result is completely consistent with the field order in the Java data object before serialization. If it is inconsistent, it will cause deserialization exceptions. In actual applications, the provider of data services will inevitably adjust the field order of Java data objects as needed, so it cannot meet the actual application needs well.

[0035] In order to solve the above problems existing in the prior art, the present invention proposes a data processing method, which converts the data to be serialized into first general data based on a preset storage structure and performs serialization processing; and when receiving a deserialization request, the deserialized data is also converted into second general data according to the storage structure. The field order of the constituent fields of the data is unified based on the preset storage structure, avoiding the risks existing when using Msgpack for serialization, thereby ensuring the accuracy of data storage and network transmission.

[0036] In the introduction to the embodiments of the present invention, the terms and their meanings are as follows:

[0037] Msgpack: is an efficient binary serialization format that allows you to exchange data between multiple languages ​​(like JSON), but it is faster and smaller;

[0038] TreeMap: is a tree graph data structure that stores and organizes objects based on key-value pairs. It provides a quick way to find a specific element and an efficient way to find the key associated with an element.

[0039] FST tree: The structure of the FST tree is constructed from an ordered list of keys, similar to a dictionary tree. Each node represents a key prefix. The FST tree is divided into two parts, the prefix area and the suffix area. The prefix area contains all keys that share the prefix, and the suffix area contains the remaining keys.

[0040] Jar package: Java Archive File, a document format of Java, is a platform-independent file format that can combine multiple files into one file.

[0041] Figure 1 is a schematic diagram of the main flow of the data processing method according to an embodiment of the present invention, such as Figure 1 As shown, the data processing method of the embodiment of the present invention includes the following steps S101 to S103.

[0042] Step S101 : in response to receiving a data serialization request, first storing target data according to a preset storage structure to obtain first general data.

[0043] Specifically, serialization is mainly used for data persistence and network transmission. In order to save the storage space occupied by data persistence and improve network transmission efficiency, the highly efficient Msgpack method is used for serialization processing. For a data set consisting of multiple sub-data, considering that the use of the Msgpack method is based on the premise that the order of each sub-data of the deserialized data should be consistent with the order of each sub-data of the serialized data, and in actual application scenarios, since the user of the data may be the data obtained by calling other services, the service provider and the service caller are not the same subject, there will inevitably be information differences, which will affect the accuracy of the deserialization results and cause deserialization anomalies. The embodiment of the present invention constructs a unified storage structure in advance. When a data serialization request is received, the target data to be serialized is first stored according to the storage structure to obtain the first general data.

[0044] According to one embodiment of the present invention, the target data is of an object type, and the object type includes a field name and a field value; according to a preset storage structure, the target data is first stored, including: obtaining the field name and the corresponding field value of the target data; storing the field name into a preset index storage structure, and storing the field value corresponding to the field name into a storage list.

[0045] Specifically, in a Java environment, the target data usually exists as an object type, and the target data object includes field names and field values. The above Msgpack method requires that the order of each sub-data of the deserialized data be consistent with the order of each sub-data of the serialized data. For the object type, the order of the field names of the deserialized Java data object needs to be consistent with the order of the field names of the serialized Java data object. For example, if an object in the current Java class contains the fields [id, name, age, sex], it needs to be deserialized in the order of [id, name, age, sex]. If the provider of the service where the data is located modifies the field order to [id, age, sex, name], and the service caller still parses according to the original [id, name, age, sex], a deserialization exception will occur, resulting in deserialized data errors.

[0046] Furthermore, the embodiment of the present invention constructs a storage structure for data to be serialized based on an index storage structure FST tree and a list List. Through the reflection mechanism of Java, the Java data object is reflected and parsed, and then each component field in the Java data object is traversed to identify the field name and the corresponding field value, and the field name and the corresponding field value are separated to obtain the field name and the corresponding field value. The field name is stored in a pre-constructed FST tree according to the construction rules of the FST tree, and the field value corresponding to the field name is stored in a pre-constructed storage list List, and a log file is used to record the correspondence between the field name stored in the FST tree and the field value stored in the List to complete the first storage.

[0047] According to another embodiment of the present invention, before storing the field name into a preset index storage structure, the method also includes: assigning a field name identifier to the field name of the target data; storing the field name into a preset index storage structure, and storing the field value corresponding to the field name into a storage list, including: storing the field name into a tree graph structure according to the field name identifier of the field name; converting the tree graph structure to obtain an index storage structure storing the field name; storing the field value corresponding to the field name into the storage list according to the field name identifier of the field name.

[0048] Specifically, in order to improve the storage efficiency of the first storage and save resources, the embodiment of the present invention assigns a field name identifier to each field name in the data object. The field name identifier assignment rule can be defined by initializing an ID number, and each time a field name is stored, the ID number is incremented to define the field name identifier assignment rule. Alternatively, a random identifier generation tool can be used to randomly assign a corresponding field name identifier to each stored field. The embodiment of the present invention does not limit the method for determining the field name identifier. According to the field identifier of each field name, the correspondence between the field name and the field name identifier ID is first stored in the tree map structure TreeMap, and the tree map structure TreeMap storing the field name and the field name identifier ID is optimized and converted to obtain the index storage structure FST tree; at the same time, the field value corresponding to the field name is stored in the storage list List according to the field name identifier ID. Optimally, when the field name is assigned a field name by incrementing the ID number, the corresponding field value can be stored in the storage list in an orderly manner according to the order in which the field name is stored. It is only necessary to mark the starting point of the storage list. In addition, it can be understood that in a Java environment, the FST tree and List storage list can be saved in a common Java object of the service provider for management.

[0049] In essence, the first storage of data in the embodiment of the present invention is to use the field name as the index Key and the field value as the corresponding Value, store the index Key in the FST tree diagram, and store the corresponding Value in the list List, so that no matter how the field order of the Java data object changes, it is converted into a storage structure composed of the FST tree storage structure and the list List, thereby achieving the unification of the field order.

[0050] Step S102: serialize the first general data to obtain a serialization result of the target data.

[0051] Specifically, based on the first common data determined above, the Msgpack serialization method is used to perform serialization processing to obtain a binary stream of the target data, and the binary stream is persisted or transmitted over a network.

[0052] It should be noted that the first common data of the embodiment of the present invention is obtained with the help of the data structure of the FST tree. Based on the characteristic of the FST tree having a shared character code, the memory space can be saved to the greatest extent. Accordingly, compared with the binary stream obtained by Msgpack serialization of the data object that has not undergone the first storage conversion, the binary stream obtained by serializing the first common data using Msgpack is smaller and the network transmission speed is faster.

[0053] Step S103, in response to receiving a deserialization request for the serialization result, deserialize the serialization result, and perform a second storage on the deserialized data according to the storage structure to obtain second general data having the same data composition order as the first general data.

[0054] Specifically, after the service caller receives the binary stream after the serialization of the first general data through the network, it triggers a deserialization request for the serialization result. The service caller deserializes the binary stream of the serialization result through the internal Msgpack deserialization module, or calls an external Msgpack deserialization module, and performs a second storage on the deserialized data according to the storage structure composed of the FST tree storage structure and the list List. The specific storage method is similar to the storage of the above-mentioned first general data. The field name in the deserialized data is stored in the FST tree, and the field value corresponding to the field name is stored in the storage list List, so as to obtain the second general data having the same data composition order as the first general data.

[0055] According to one embodiment of the present invention, the method also includes: in response to receiving a deserialization result query request from a user, obtaining constituent elements of the deserialization result according to the deserialization result query request; according to the constituent elements of the deserialization result, searching for corresponding numerical values ​​from the second general data to obtain a deserialization result corresponding to the deserialization result query request.

[0056] Specifically, before data is persisted or transmitted over the network, the interface information, interface type, data type, and data composition related information involved in the service are usually assembled into an archive file, so that when the service caller user queries the deserialization result, the query request can be used to obtain the data components corresponding to the query request from the archive file, and the components are used as the components of the deserialization result. Although in theory, the archive file can ensure that the caller user can obtain the order of data components consistent with the service provider, in practice, since the service provider and the service caller are usually not in the same physical area, when the service provider modifies the order of data components, or adds or deletes components, the changer may not be updated to the archive file in a timely manner. In addition, the service caller may also obtain from the archive file in the first call, and subsequent calls may habitually default that the components of the acquired data are the components of the data of the current service, instead of querying the archive file every time it is called, resulting in the order of the data components of the deserialization result being inconsistent with the order of the data components before serialization.

[0057] Furthermore, the second general data stores the components of the data and the values ​​corresponding to the components. According to the components of the obtained deserialization result, the corresponding values ​​are searched from the second general data, and the values ​​corresponding to each component are searched from the second general data to obtain the deserialization result corresponding to the serialization result query request.

[0058] According to another embodiment of the present invention, when the deserialization result is an object type, the component element is a field name; according to the component element of the deserialization result, searching for the corresponding numerical value from the second general data includes: according to the field name, searching for the corresponding field value from the second general data.

[0059] Specifically, when the deserialization result is a Java object type, the archive file is usually a Jar package. According to the query request, the relevant information of the corresponding call service is obtained from the Jar package, and the Java data object is parsed using the Java reflection mechanism to obtain the field names included in the Java data object. Then, querying the deserialization result is actually the process of searching for the corresponding field value from the second general data according to the field name. It can be understood that in the Java environment, the FST tree and the List storage list can be saved in a general Java object of the caller for management.

[0060] Figure 2 It is a complete schematic diagram of the data processing principle of an embodiment of the present invention. For business Java data objects that need to be transmitted over the network, the service provider first stores the business Java data objects according to the preset storage structure to obtain a general Java object, that is, the first general data; then uses Msgpack to serialize the general Java object to obtain a binary stream; and transmits the binary over the network. When the service caller receives the binary stream, it performs Msgpack deserialization on the binary stream, and performs a second storage on the deserialized data according to the above storage structure to obtain a general Java object, that is, the second general data; then, according to the components of the deserialization result, the corresponding field value is obtained from the general Java object (the second general data) to obtain the deserialization result (business Java data object) to achieve data processing.

[0061] According to another embodiment of the present invention, the second general data includes an index storage structure for storing field names and a storage list for storing corresponding field values; according to the field name, the corresponding field value is found from the second general data, including: according to the field name, a matching matching field name is found from the index storage structure, and the matching field name has a field name identifier; according to the field name identifier, the corresponding field value is found from the storage list.

[0062] Specifically, the second general data of the embodiment of the present invention includes an FST index storage structure storing field names and a List storage list storing corresponding field values. According to the field name, a matching field name matching the field name is searched from the FST tree of the second general data, and the field name ID of the matching field name is obtained; with the field name ID as the search basis, the field value corresponding to the field name ID is searched from the List, and the field value is the target field value of the search.

[0063] According to another embodiment of the present invention, before finding the corresponding field value from the second general data according to the field name, the method also includes: judging and determining that the data type of the field name is a basic data type; when the data type of the field name is a composite data type, the field name exists in the form of a set, and the field name is disassembled according to the basic data type so as to find the corresponding field value according to the disassembled field name.

[0064] Specifically, usually the data type of the field name of a Java data object is a basic data type such as character type char, integer type int, long integer type long, floating point type float, etc., but sometimes there are composite data types such as mapping map and set set. The second general data is data with basic data types as the granularity, so before finding the corresponding field value from the second general data according to the field name, it is judged and determined that the data type of the field name is a basic data type. For basic data types, the corresponding field value can be directly found from the second general data according to the field name. For composite data types, that is, the field name also includes sub-field names, it is necessary to disassemble the field name that meets the type according to the basic data type. If the disassembled field name is still a composite data type, then recursive disassembly is performed until the data type of the disassembled field name is a basic data type, so that the corresponding field value can be found from the second general data according to the field name of each basic data type.

[0065] Figure 3It is a detailed flow chart of the data processing method of the embodiment of the present invention. The service provider develops a business system including business logic processing and business Java data objects, parses Java data objects through reflection mechanism, separates field names and corresponding field values; uses field name identification ID self-increment method to generate a corresponding field name identification ID for each field name of Java data object; first stores the field name and field name identification ID in the tree map structure TreeMap, then optimizes the TreeMap storing the field name and field name ID into the index storage structure FST, and at the same time stores the field value corresponding to the field name in the storage list List according to the field name identification of the field name; then saves the FST tree and List into a general Java object to obtain the first general data; finally, uses Msgpack to serialize the general Java object, that is, the first general data, to obtain a binary stream to be transmitted on the network.

[0066] Correspondingly, after the service caller deserializes the received binary stream, it performs a second storage on the deserialized data and saves it in the caller's general Java object according to the same storage structure as above to obtain the second general data; based on the service provider's Jar package, the reflection mechanism is used to parse the business Java data object to obtain its components (field names); the field names of the Java data object are traversed to determine whether they are basic data types. If so, the field value corresponding to the field name is searched from the FST tree and List of the second general data, and the field value is assigned to the field name; if it is not a basic data type, the data type of the field name is a composite data type, and the field name needs to be disassembled. Of course, if there is still a composite data type in the disassembled field name, the recursive disassembly continues until the data types of the obtained field names are all basic data types. After assigning the found field value to the current field name, it is necessary to determine whether the traversal of the field name in the business Java data object has been completed. If the traversal has not been completed, continue to search for the corresponding field value from the FST tree and List of the second general data according to the field name in the business Java data object until the traversal is completed, and the deserialization result (business Java data object) required for the call is obtained for use in its own business logic processing.

[0067] Figure 4 Schematic diagram of the main modules of the data processing device according to the embodiment of the present invention. Figure 4 As shown, the data processing device 400 mainly includes a first general data acquisition module 401 , a serialization processing module 402 and a second general data acquisition module 403 .

[0068] A first general data acquisition module 401 is used for, in response to receiving a data serialization request, performing a first storage on the target data according to a preset storage structure to obtain first general data;

[0069] A serialization processing module 402 is used to perform serialization processing on the first general data to obtain a serialization result of the target data;

[0070] The second general data acquisition module 403 is used to respond to receiving a deserialization request for the serialization result, deserialize the serialization result, and perform a second storage on the deserialized data according to the storage structure to obtain second general data having the same data composition order as the first general data.

[0071] According to one embodiment of the present invention, the target data is an object type, and the object type includes a field name and a field value; the first general data acquisition module 401 is also used to: obtain the field name and the corresponding field value of the target data; store the field name into a preset index storage structure, and store the field value corresponding to the field name into a storage list.

[0072] According to another embodiment of the present invention, the data processing device 400 also includes a field name identifier determination module (not shown in the figure), which is used to: assign a field name identifier to the field name of the target data before storing the field name into a preset index storage structure; the first general data acquisition module 401 is also used to: store the field name into a tree graph structure according to the field name identifier of the field name; convert the tree graph structure to obtain an index storage structure storing the field name; store the field value corresponding to the field name in a storage list according to the field name identifier of the field name.

[0073] According to another embodiment of the present invention, the data processing device 400 also includes a deserialization result query module (not shown in the figure), which is used to: in response to receiving a deserialization result query request from a user, obtain the constituent elements of the deserialization result according to the deserialization result query request; according to the constituent elements of the deserialization result, search for the corresponding numerical value from the second general data to obtain the deserialization result corresponding to the deserialization result query request.

[0074] According to another embodiment of the present invention, when the deserialization result is an object type, the component element is a field name; the deserialization result query module (not shown in the figure) is also used to: search for a corresponding field value from the second general data according to the field name.

[0075] According to another embodiment of the present invention, the second general data includes an index storage structure for storing field names and a storage list for storing corresponding field values; the deserialization result query module (not shown in the figure) is also used to: according to the field name, find a matching matching field name from the index storage structure, and the matching field name has a field name identifier; according to the field name identifier, find the corresponding field value from the storage list.

[0076] According to another embodiment of the present invention, the data processing device 400 also includes a basic data type confirmation module (not shown in the figure), which is used to: judge and determine that the data type of the field name is a basic data type before finding the corresponding field value from the second general data based on the field name; in the case that the data type of the field name is a composite data type, the field name exists in the form of a set, and the field name is disassembled according to the basic data type so as to find the corresponding field value based on the disassembled field name.

[0077] Figure 5 is an exemplary system architecture diagram to which embodiments of the present invention can be applied.

[0078] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0079] Users can use terminal devices 501, 502, 503 to interact with server 505 through network 504 to receive or send messages, etc. Various communication client applications, such as data processing applications, etc., can be installed on terminal devices 501, 502, 503 (only as examples).

[0080] The terminal devices 501 , 502 , and 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0081] The server 505 may be a server that provides various services, such as a background management server (only an example) that provides support for data processing performed by users using terminal devices 501, 502, and 503. In response to receiving a data serialization request, the background management server may perform a first storage on the target data according to a preset storage structure to obtain first general data; perform serialization processing on the first general data to obtain a serialization result of the target data; in response to receiving a deserialization request for the serialization result, perform a deserialization processing on the serialization result, and perform a second storage on the deserialized data according to the storage structure to obtain second general data having the same data composition order as the first general data, and the processing, and feedback the processing result (such as the second general data, etc. - only an example) to the terminal device.

[0082] It should be noted that the data processing method provided in the embodiment of the present invention is generally executed by the server 505 , and accordingly, the data processing device is generally set in the server 505 .

[0083] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0084] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 of a terminal device or a server suitable for implementing an embodiment of the present invention. Figure 6 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0085] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0086] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0087] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0088] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination thereof.

[0089] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0090] The units involved in the embodiments of the present invention may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes: a first general data acquisition module, a serialization processing module and a second general data acquisition module.

[0091] The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the serialization processing module can also be described as "a module for performing serialization processing on the first common data to obtain a serialization result of the target data."

[0092] On the other hand, the present invention also provides a computer-readable medium, which may be included in the device described in the embodiment; or it may exist independently without being assembled into the device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by a device, the device includes: in response to receiving a data serialization request, first storing the target data according to a preset storage structure to obtain first general data; serializing the first general data to obtain a serialization result of the target data; in response to receiving a deserialization request for the serialization result, deserializing the serialization result, and second storing the deserialized data according to the storage structure to obtain second general data having the same data composition order as the first general data.

[0093] The technical solution according to the embodiment of the present invention has the following advantages or beneficial effects: in response to receiving a data serialization request, the target data is first stored according to a preset storage structure to obtain first general data; the first general data is serialized to obtain a serialization result of the target data; in response to receiving a deserialization request for the serialization result, the serialization result is deserialized, and the deserialized data is secondly stored according to the storage structure to obtain a technical solution of second general data having the same data composition order as the first general data, which realizes unifying the field order of the constituent fields of the data based on a preset storage structure, avoiding the risks existing when using Msgpack for serialization, and thereby ensuring the accuracy of data storage and network transmission.

[0094] The specific implementation methods described herein do not constitute limitations on the scope of protection of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A method of data processing, It is characterized in that include: In response to receiving the data serialization request, first storing the target data according to a preset storage structure to obtain first general data; Performing serialization processing on the first general data to obtain a serialization result of the target data; In response to receiving a deserialization request for the serialization result, the serialization result is deserialized, and the deserialized data is second-stored according to the storage structure to obtain second general data having the same data composition order as the first general data.

2. The method according to claim 1, It is characterized in that The target data is of object type, and the object type includes a field name and a field value; According to a preset storage structure, the target data is first stored, including: Obtain the field name and corresponding field value of the target data; The field name is stored in a preset index storage structure, and the field value corresponding to the field name is stored in a storage list.

3. The method according to claim 2, It is characterized in that Before storing the field name into a preset index storage structure, the method further includes: assigning a field name identifier to the field name of the target data; Storing the field name in a preset index storage structure and storing the field value corresponding to the field name in a storage list includes: storing the field name in a tree graph structure according to the field name identifier of the field name; Converting the tree graph structure to obtain an index storage structure storing the field names; The field value corresponding to the field name is stored in a storage list according to the field name identifier of the field name.

4. The method according to claim 1, It is characterized in that The method further comprises: In response to receiving a deserialization result query request from a user, obtaining constituent elements of the deserialization result according to the deserialization result query request; According to the constituent elements of the deserialization result, the corresponding value is searched from the second general data to obtain the deserialization result corresponding to the deserialization result query request.

5. The method according to claim 4, It is characterized in that When the deserialization result is of object type, the component element is a field name; According to the components of the deserialization result, searching for a corresponding value from the second general data includes: According to the field name, the corresponding field value is searched from the second general data.

6. The method according to claim 5, It is characterized in that The second general data includes an index storage structure storing field names and a storage list storing corresponding field values; According to the field name, searching for a corresponding field value from the second general data includes: According to the field name, a matching field name is searched from the index storage structure, where the matching field name has a field name identifier; According to the field name identifier, the corresponding field value is searched from the storage list.

7. The method according to claim 5, It is characterized in that Before searching the second general data for a corresponding field value according to the field name, the method further includes: Determine and confirm that the data type of the field name is a basic data type; In the case where the data type of the field name is a composite data type, the field name exists in a set form, and the field name is disassembled according to the basic data type so as to find the corresponding field value according to the disassembled field name.

8. A data processing device, It is characterized in that include: A first general data acquisition module, configured to, in response to receiving a data serialization request, first store the target data according to a preset storage structure to obtain first general data; A serialization processing module, used for performing serialization processing on the first general data to obtain a serialization result of the target data; A second general data acquisition module is used to respond to receiving a deserialization request for the serialization result, deserialize the serialization result, and perform a second storage on the deserialized data according to the storage structure to obtain second general data having the same data composition order as the first general data.

9. A mobile electronic device terminal, It is characterized in that include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images