Data processing method and apparatus

Abstract

The application provides a data processing method and device, wherein the data processing method comprises: obtaining to-be-stored binary data and writing the to-be-stored binary data into a binary file; determining a data length of the to-be-stored binary data and a relative offset corresponding to the to-be-stored binary data; wherein the relative offset is a distance between a target address of the to-be-stored binary data in the binary file and a segment address of a field where the to-be-stored binary data is located; and updating a data structure object corresponding to the binary file based on the data length and the relative offset. Data stored in this form can be directly loaded into memory when called, and target data can be obtained without deserialization operation, so that data loading is more efficient, and in particular in a situation requiring real-time calculation of data, the operation without deserialization greatly increases system running efficiency, thereby solving the problems of slow system running and lag.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a data processing method and a data reading method. The present application also relates to a data processing device, a data reading device, a computing device, and a computer readable storage medium. BACKGROUND

[0002] With the development of Internet technology, more and more data information is applied to computing devices, which makes data processing and reading more difficult. Before the hardware of computing devices undergoes revolutionary iteration, it is increasingly difficult to process an increasing amount of data. In the prior art, when data is stored, a container marker is written based on the container, and then the container size is added, and finally the data is written into the container in sequence. When reading the data content, the data also needs to be read in sequence to ensure that the target data object can be correctly constructed. However, when a large amount of data in the container is processed in real time, the read data needs to be deserialized before it can be recognized and applied by the computing system, which will make the real-time computing process of the system inefficient. Therefore, how to speed up the computing efficiency based on the original hardware of the computing device has become a problem to be solved. SUMMARY

[0003] In view of this, the embodiments of the present application provide a data processing method and a data reading method to solve the technical defects in the prior art. The embodiments of the present application also provide a data processing device, a data reading device, a computing device, and a computer readable storage medium.

[0004] According to a first aspect of the embodiments of the present application, a data processing method is provided, comprising:

[0005] obtaining to-be-stored binary data and writing the to-be-stored binary data into a binary file;

[0006] determining a data length of the to-be-stored binary data and a relative offset corresponding to the to-be-stored binary data; wherein the relative offset is a distance between a target address of the to-be-stored binary data in the binary file and a segment address of a field where the to-be-stored binary data is located;

[0007] updating a data structure object corresponding to the binary file based on the data length and the relative offset.

[0008] Optionally, the obtaining to-be-stored binary data and writing the to-be-stored binary data into a binary file comprises:

[0009] obtaining to-be-stored data and detecting a type of the to-be-stored data;

[0010] In a case where the data to be stored is of a binary data type, the data to be stored is taken as the binary data to be stored, and is written into the binary file;

[0011] In a case where the data to be stored is not of a binary data type, a data conversion method is determined according to a type of the data to be stored and a data structure, the data to be stored is converted into the binary data to be stored based on the data conversion method, and is written into the binary file.

[0012] Optionally, the determining of the data length of the binary data to be stored comprises:

[0013] detecting a number of bytes of the binary data to be stored;

[0014] determining the data length of the binary data to be stored based on the number of bytes.

[0015] Optionally, the determining of the relative offset corresponding to the binary data to be stored comprises:

[0016] determining a position field corresponding to the binary data to be stored in the binary file, taking address information of the position field as a segment address;

[0017] determining a target address of the binary data based on address information of a field in which data content of the binary data to be stored is stored in the binary file;

[0018] calculating a distance between the segment address and the target address, and determining the relative offset according to a result of the calculation.

[0019] Optionally, the data structure object comprises file layout information and field identification.

[0020] The file layout information comprises a data structure of data stored in a field in the binary file.

[0021] The field identification comprises a field name of the field in the binary file, and is used to distinguish between the fields in the binary file.

[0022] According to a second aspect of an embodiment of the present application, a data reading method is provided, comprising:

[0023] obtaining field identification;

[0024] querying a data structure object based on the field identification, and obtaining a data length and a relative offset corresponding to the field identification;

[0025] determining a target address in a binary file based on the relative offset;

[0026] read target binary data according to the data length and the target address in the binary file and load to memory.

[0027] Optionally, further comprising:

[0028] save the target address and the data length of the target binary data;

[0029] In the case of receiving a continuous binary data read request, determine a neighboring target address of neighboring binary data adjacent to the target binary data based on the saved target address and the data length;

[0030] query the data structure object according to the field identifier of the neighboring binary data, and determine a neighboring data length of the neighboring binary data;

[0031] read the neighboring binary data in the binary file based on the neighboring target address and the neighboring data length, and save the target address and the neighboring data length of the neighboring binary data.

[0032] Optionally, the determining a target address in a binary file based on the relative offset comprises:

[0033] determine that the queried field identifier corresponds to a segment address of the binary file;

[0034] add the segment address to the relative offset to obtain the target address of the target binary data.

[0035] Optionally, the reading target binary data according to the data length and the target address in the binary file comprises:

[0036] add the target address and the data length to obtain a termination position of the target binary data;

[0037] determine a byte range of the target binary data in the binary file based on a start position of the target binary data indicated by the target address and in combination with the termination position;

[0038] sequentially read data content in the byte range to obtain the target binary data.

[0039] According to a third aspect of the embodiments of the present application, a data processing apparatus is provided, comprising:

[0040] an obtaining module configured to obtain to-be-stored binary data and write the binary data into a binary file;

[0041] The determining module is configured to determine a data length of the binary data to be stored and a relative offset corresponding to the binary data to be stored; wherein the relative offset is a distance between a target address of the binary data to be stored in a binary file and a segment address of a field where the binary data to be stored is located.

[0042] The updating module is configured to update a data structure object corresponding to the binary file based on the data length and the relative offset.

[0043] According to a fourth aspect of the embodiments of the present application, a data reading apparatus is provided, comprising:

[0044] The identification obtaining module is configured to obtain a field identification.

[0045] The querying module is configured to query a data structure object based on the field identification, and obtain a data length and a relative offset corresponding to the field identification.

[0046] The address confirming module is configured to determine a target address in a binary file based on the relative offset.

[0047] The reading module is configured to read target binary data from the binary file according to the data length and the target address and load the target binary data to a memory.

[0048] According to a fifth aspect of the embodiments of the present application, a computing device is provided, comprising:

[0049] a memory and a processor;

[0050] The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to implement steps of the data processing method or the data reading method.

[0051] According to a sixth aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores computer executable instructions, and the instructions are executed by a processor to implement steps of the data processing method or the data reading method.

[0052] According to a seventh aspect of the embodiments of the present application, a chip is provided, which stores a computer program, and the computer program is executed by the chip to implement steps of the data processing method or the data reading method.

[0053] The data processing method provided in this application obtains binary data to be stored and writes it into a binary file. Then, it determines the data length based on the number of bytes occupied by the binary data and determines the relative offset based on the distance between the target address of the data to be stored in the binary file and the segment address of its current segment. Finally, it updates the data structure object based on the data length and relative offset of the binary data to be stored. Data stored in this form can be directly loaded into memory when called, eliminating the need for deserialization. This makes data loading more efficient, especially in scenarios requiring real-time data computation. The elimination of deserialization significantly increases system efficiency and solves the problems of slow and laggy system operation. Attached Figure Description

[0054] Figure 1 This is a flowchart of a data processing method provided in an embodiment of this application;

[0055] Figure 2 This is a schematic diagram illustrating the correspondence between binary files and data structure objects in a data processing method provided in an embodiment of this application;

[0056] Figure 3 This is a schematic diagram of data structure transformation in a data processing method provided in an embodiment of this application;

[0057] Figure 4 This is a schematic diagram of the data structure in a data processing method provided in an embodiment of this application;

[0058] Figure 5 This is a flowchart of a data reading method provided in an embodiment of this application;

[0059] Figure 6 This is a flowchart illustrating a data processing method and a data reading method applied to a data reading and storage scenario, provided by an embodiment of this application.

[0060] Figure 7 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0061] Figure 8 This is a schematic diagram of the structure of a data reading device provided in one embodiment of this application;

[0062] Figure 9 This is a structural block diagram of a computing device provided in one embodiment of this application. Detailed Implementation

[0063] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced without the specific details. In other instances, well-known methods have not been described in detail in order not to unnecessarily obscure aspects of the present application.

[0064] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present application. As used in this disclosure and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0065] It will be understood that, although the terms first, second, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used solely to distinguish one from another only. For example, a first item could be termed a second item, and, similarly, a second item could be termed a first item without departing from the scope of one or more embodiments of the present application.

[0066] First, the noun terms related to one or more embodiments of the present application are explained.

[0067] Configuration file: In the field of computer science, a configuration file is a computer file that can configure parameters and initial settings for some computer programs.

[0068] Conversion code: Code used to handle encoding conversion tasks, which can convert files into formats required by end users. The conversion code in this application is to convert files into binary format, specifically to read data from original files (*.txt, *.ini, *.json, etc.), construct memory objects according to the basic container data structure, and then serialize them into binary files.

[0069] Variable-length list: A variable-length list refers to a container type that can store elements of any length. Note that the variable-length list cannot dynamically update the container size in the file or memory. If you need to update it, you need to rebuild the complete data and serialize it to the file.

[0070] In this application, a data processing method and a data reading method are provided. This application also relates to a data processing device, a data reading device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0071] With the development of the Internet, each data processing will involve more and more data, and the data length is also increasing. In the process of reading data, due to the length of the data, the step of deserialization also becomes more time-consuming. Therefore, a data processing method and a data reading method are introduced, so that deserialization is no longer performed in the data reading process, thereby accelerating the data reading speed. Especially in the context of real-time data calculation, without the operation of deserialization, the system running efficiency is greatly increased, and the problems of slow system running and lag are solved.

[0072] Figure 1 A flowchart of a data processing method according to an embodiment of the present application is shown, which specifically includes the following steps:

[0073] Step S102: Obtain the binary data to be stored and write into the binary file.

[0074] The binary data writing and reading are performed by the file list program in the storage file.

[0075] The internal storage content of the binary file is binary data. Before storing the binary file, the data needs to be serialized so that the data to be stored conforms to the storage rules of the binary file.

[0076] Specifically, after the file list program obtains the data to be stored, it is found that the data format of the data to be stored is binary, which conforms to the storage rules of the binary file. At this time, the file list program stores the binary data to be stored in the specified position of the binary file. Conversely, if it is found that the data format of the data to be stored is not binary, that is, the data to be stored does not conform to the storage rules of the binary file, the file list program will not store the data to be stored.

[0077] Based on this, step S102 makes the data stored in the file binary data, and the data read out during data reading is also binary data, which can be directly recognized by the system.

[0078] Further, in the process of obtaining binary data and storing, the file list program can directly store the binary data, but for non-binary data, the file list program cannot store it in the binary file with different data types. The diversity of data types leads to the fact that the data to be stored is not binary, which is very common. In this case, the storage conditions are harsh, and the application scope of the overall scheme is too narrow. Therefore, it is particularly important to increase the application scope of the scheme so that non-binary data can also be stored by the scheme. In the present embodiment, the specific implementation is as follows:

[0079] Obtaining the to-be-stored data, and detecting the type of the to-be-stored data; in the case that the to-be-stored data is of a binary data type, storing the to-be-stored data as the to-be-stored binary data and writing the to-be-stored binary data into the binary file; in the case that the to-be-stored data is not of the binary data type, determining a data conversion method according to the type of the to-be-stored data and a data structure, converting the to-be-stored data into the to-be-stored binary data based on the data conversion method, and writing the to-be-stored binary data into the binary file.

[0080] Specifically, the data conversion method is to convert data that is not originally binary into binary data through encoding using a conversion code.

[0081] Based on this, after receiving the to-be-stored data, the file list program judges the to-be-stored data, and if the type of the to-be-stored data is binary data, it is stored in the binary file; if the type of the to-be-stored data is not binary data, the file list program detects the specific type of the to-be-stored data, converts the to-be-stored data through a conversion code corresponding to the type of the to-be-stored data according to the type of the to-be-stored data, generates corresponding binary data, and then stores the converted binary data into the binary file.

[0082] For example, during game running, the player is playing the first level, the player adjusts the resolution of the game to the highest to improve the game performance, at this time the terminal needs to store the resolution data set by the player, after receiving the resolution data set by the player, the file list program judges that the resolution data is in the form of 16 hexadecimal VARCHAR, at this time a conversion code corresponding to 16 hexadecimal VARCHAR is selected to convert it into binary data code. Then the converted resolution data is stored in the binary file.

[0083] In summary, the type conversion of the to-be-stored data further increases the selected range of the to-be-stored data, and non-binary data can also be converted into binary data that can be stored in this scheme through encoding, which greatly increases the scope of application of this scheme.

[0084] Step S104: determining the data length of the to-be-stored binary data and the relative offset corresponding to the to-be-stored binary data; wherein the relative offset is the distance between the target address of the to-be-stored binary data in the binary file and the segment address of the field where the to-be-stored binary data is located.

[0085] Specifically, after the above-mentioned to-be-stored binary data is stored into the binary file, the position information thereof also needs to be marked, so that the stored binary data can be conveniently selected by the file list program in the subsequent real-time reading process. The relative offset corresponding to the to-be-stored binary data is used to indicate the distance relationship between the storage position of the attribute information and the position information of the binary data and the storage position of the content of the binary data.

[0086] Specifically, the file list program first confirms the data length of the stored binary data and the relative offset thereof, and stores the data length and the relative offset into the binary file. The position information of the two has been confirmed before the storage of the binary file is completed. It needs to be noted that the position selection rule is diverse, and the embodiment is not limited. Preferably, the selection rule of the embodiment is the nearest unoccupied rated length field at the front position in the binary file.

[0087] Based on this, the file list program stores the data length and the relative offset of the stored binary data. When reading the stored binary data, the stored binary data can be directly indexed and read according to the information of the data length and the relative offset.

[0088] Further, for the to-be-stored binary data, the data length thereof needs to be first confirmed, so that the length of the binary data can be clearly known in the reading process, and the reading range can be selected. In the embodiment, the specific implementation manner is as follows:

[0089] Detecting the byte quantity of the to-be-stored binary data; determining the data length of the to-be-stored binary data based on the byte quantity.

[0090] Specifically, the file list program detects the byte quantity of the to-be-stored binary data.

[0091] Based on this, the file list program counts the stored bytes when storing the obtained binary data into the binary list, obtains the byte quantity of the binary data, and determines the data length of the stored binary data according to the obtained byte quantity.

[0092] Further, for the stored binary data, the storage position thereof also needs to be known, so that the reading pointer of the file list program can be directed to the head position of the stored binary data in the process of reading the binary data, and the binary data can be completely read. In the embodiment, the specific implementation manner is as follows:

[0093] determining a corresponding position field of the binary data to be stored in the binary file, taking the address information of the position field as a segment address; determining a target address of the binary data based on the address information of the field in which the data content of the binary data to be stored is stored in the binary file; calculating the distance between the segment address and the target address, and determining the relative offset according to the calculation result.

[0094] Specifically, the offset refers to the position distance between the data stored in the file, and the relative offset in the embodiment indicates the relationship between the address at which the binary data stores its data length and relative offset and the actual address at which the binary data content is stored; the position field is a storage space pre-established for storing the data length information and relative offset information of the binary data.

[0095] Based on this, the file list program stores the binary data content in a variable-length container, wherein the variable-length container is a container that can store elements of any length, and the variable-length list is a kind of variable-length container. Then the variable-length container is stored in the binary file, the address information of the binary file in which the binary data content is stored is determined and stored, and then the file list program detects the idle storage space in the binary file from the beginning. In the nearest one of the nominal length fields not occupied in the front position in the binary file, the data length and relative offset of the binary file will be stored in the nominal length field, and the nominal length is set as the byte length occupied by the data length and relative offset. After determining the storage position of the data length and relative offset of the binary data, the address information of the field position is saved.

[0096] Then, the file list program calculates the distance between the above saved two address information, and this distance is the relative offset of the stored binary data.

[0097] The determination process of the data length and the determination process of the relative offset are described as follows. During the game running, the player is playing the first level, the player sets the resolution of the game to the highest to improve the game performance, and the terminal needs to store the resolution data set by the player. After the file list program receives the binary resolution data set by the player, the binary resolution data is calculated and stored in a variable length container Data.c. During the storage, the file list program counts the number of bytes of the binary resolution data to obtain the data length information count, then stores the variable length container Data.c in which the binary resolution data is located in the binary file, and records the address information of the binary file. Then the file list program queries the binary file, and finds the nearest one of the rated length fields which is not occupied in the front position of the binary file, and records the address information of the field. The relative offset offset is obtained by combining the address information of the variable length container Data.c. Finally, the data length count and the relative offset offset are stored in the queried rated length field.

[0098] In summary, by recording the data length and the relative offset of the stored binary data, the content of the binary data can be stored in a variable length container, and accurate indexing can be achieved.

[0099] Step S106: updating the data structure object corresponding to the binary file based on the data length and the relative offset.

[0100] Specifically, after the data length and the relative offset of the stored binary data are determined, the two are stored in the binary file. However, if it is needed to read a certain independent binary data in the binary file, in the conventional case, the binary file needs to be sequentially read, and the storage position thereof cannot be directly and specifically queried. Therefore, at this time, the binary data to be queried can be selected to be queried at the code level, and the storage position of the data to be queried in the binary file can be directly located by the data structure object at the code level, so that the sequential query mode which consumes more resources is avoided.

[0101] The data structure object is part of the code at the code level, and records the structure information of the binary data stored in the binary file and the field name thereof. The binary data in the binary file can be found by querying the part of the code.

[0102] Specifically, after the file list program stores the binary data into the binary file, the data structure object in the code layer is updated based on the data length and the relative offset of the binary data, so that the data structure object can save the structural relationship of the binary data in the binary file, facilitating the indexing during the reading process of the binary data in the binary file.

[0103] Therefore, when the file list program reads the binary data in the binary file, the structural relationship and the positional relationship of all the binary data in the binary file can be obtained by comparing the data structure object, so that the reading process becomes efficient.

[0104] Further, the data structure object includes file layout information and field identifiers; the file layout information includes the data structure of the data stored in the fields in the binary file; and the field identifiers include the field names of the fields in the binary file, and are used to distinguish the fields in the binary file.

[0105] In combination with Figure 2 The binary file and the data structure object corresponding relationship in the data processing method shown in the diagram, Figure 2 In the code part described in the right side block, the schematic form of the data structure object is in the dashed box, and it can be seen that value1, value2, list1 and list2 in the data structure object represent field identifiers, which can identify the binary data in the binary file, and the data structure of the binary data can be obtained according to the field identifiers. Figure 2 As shown in the diagram, the field identifiers are the "names" of the fields, and the naming method is not limited in this embodiment, but preferably, the field identifiers can indicate the data structure of the corresponding binary data, such as value representing integer data and list representing list data. The file layout information is also included in the code layer data corresponding to the data structure object, and the file layout information is Figure 2 As shown in the diagram, {count=3, offset=16} and {count=4, offset=20} are recorded after list1 and list2, and in them, the count data records the number of elements in the list, in which list1 contains three elements m1, m2 and m3, and list2 contains four elements n1, n2, n3 and n4. To some extent, since the binary file carries data, the number of bytes corresponding to each data element is fixed, such as Figure 2 As shown in the diagram, each data element in this embodiment corresponds to 4 bytes. As can be seen, the number of elements represented by count also represents the data length to some extent, because the data length = the number of elements * the length of a single element in bytes, and offset is the offset.

[0106] For example, during game operation, when a player is playing the first level, the player sets the game resolution to the highest level to improve game performance. At this time, the terminal needs to store the resolution data set by the player. After the file list program has stored the binary resolution data set by the player, the file list program updates the data structure object based on the data length count of the stored binary resolution and the relative offset offset.

[0107] In summary, the data structure object includes file layout information and field identifiers, which makes it simple and fast for file listing programs to read binary data from binary files.

[0108] It should be noted that the process of locating binary data in a binary file based on data structure objects is an interaction between the data layer and the code layer. Figure 2 This was demonstrated, such as Figure 2 As shown, in the data structure object in the code layer, value1 and value2 represent integer data, such as... Figure 2 The values ​​shown correspond to the positions of the 1st and 2nd elements in the binary file of the data layer. `list1` and `list2` represent list data, where the length and offset of `list1` are stored at the 3rd and 4th elements in the binary file of the data layer, respectively, and the length and offset of `list2` are stored at the 5th and 6th elements in the binary file of the data layer, respectively.

[0109] Further steps can be taken through Figure 2 We can see that the third element of the binary file stores the data length information of list1, count = 3; the fourth element stores the offset information of list1, offset = 16; the fifth element stores the data length information of list2, count = 4; and the sixth element stores the offset information of list2, offset = 20. Figure 2 In the example, since the offset of list1 is 16 bytes, counting 16 bytes from the position in the binary file where the data length information count=3 of list1 is stored, we find the starting positions for storing the data elements m1, m2, and m3 in list1. Since count=3 for list1, the three binary file elements following the starting positions of the data elements m1, m2, and m3 in list1 respectively store the three elements m1, m2, and m3 of list1. Furthermore, because... Figure 2In this embodiment, each element in the binary file consists of 4 bytes. Therefore, counting 16 bytes forward from the position where the data length information `count=3` of `list1` is stored, which is equivalent to counting 4 binary file elements (the pointer relationship is as follows). Figure 2 (As indicated by the arrow marked "16 bytes"). Similarly, counting 5 elements from the position where the data length information of list2, count=4, is stored in the binary file, the starting positions of the data elements n1, n2, n3, and n4 in list2 are (the pointer relationship is as follows). Figure 2 (As indicated by the arrow marked "20 bytes"), the three binary file elements following the starting positions of the data elements n1, n2, n3, and n4 in list2 respectively store the three elements n1, n2, n3, and n4 of list2.

[0110] Based on the above, the field identifiers in a data structure object can be used to indicate the data structure of the corresponding binary data. The data structure of the binary data is determined by the file listing program, and the determination process is as follows:

[0111] 1. Traverse the binary data to be stored.

[0112] 2. If binary data is a data structure based on a basic data type, then define a data structure with the same basic data type, such as integer, which is a data type that does not require a variable-length container when stored in a binary file.

[0113] 3. If binary data such as lists, strings, and mapping tables are stored in a variable-length container in a binary data structure, then define a corresponding data structure in the preset data structure type template.

[0114] 4. If the binary data is a custom data structure, then define a corresponding data structure based on that structure.

[0115] Having determined the binary data structure means that data structure objects in the code layer can store binary data. However, at the code layer, it's also necessary to unify the different representations of data structure objects. If the list data `list1 = {m1, m2, m3}` is clearly an exhaustive approach, which is ineffective when there are too many elements. From `list1`, we know it contains 3 elements. Similarly, `list2 = {n1, n2, n3, n4}` contains 4 elements. Assuming the binary file specifies 4 bytes for each element's storage location, then the storage of `list1` and `list2` will be as follows: Figure 2As shown, the data information for list1 is stored four elements after its own count and offset information, so the corresponding offset is 16. Similarly, the offset for list2 is 20. Therefore, at the code level, the binary data is serialized from list1 = {m1, m2, m3} and list2 = {n1, n2, n3, n4} to list1 = {count = 3, offset = 16} and list2 = {count = 4, offset = 20}. The serialized code-level data structure objects have a unified format, providing convenience for searching binary data in binary files.

[0116] like Figure 3 The diagram illustrating data structure conversion shows that in the example data structure, the list data is structured so that each element is defined as an integer variable with the data type int. This clearly does not meet the binary data storage requirements of this application. Therefore, the data structure can be converted using the steps described above. Figure 3 As shown, for basic data types, the data structure remains unchanged. However, for data types requiring variable-length containers for storage, the data structure is converted using the method preset in this application. Figure 3 std::list <int>The data structure in the form of a list is converted into a BinArray <int>list form.

[0117] After determining the data structure of the data to be stored, the following steps are needed for serialization, loading it into memory:

[0118] 1. Pre-allocate a large enough memory.

[0119] 2. Build a sequential memory allocator (i.e. memory is allocated sequentially) according to the pre-allocated memory.

[0120] 3. According to the memory allocator, allocate the current processing binary data to the memory space corresponding to its data structure type.

[0121] 4. After identifying the data type of the current processing binary data, if its data type is a basic data type, store the binary data directly into the memory space; if the binary data is a custom data structure, unify the data format of the binary data through the above-mentioned method; if the data type of the binary data is a data structure stored in the binary data using a variable-length container, store the binary data into the binary file through the steps 4.1-4.4 below;

[0122] 4.1: Calculate the number of elements count and the memory size occupied by each element of the binary data.

[0123] 4.2: Allocate a memory space of the size corresponding to each element through the memory allocator, and obtain an offset according to the memory address and the element number count.

[0124] 4.3: Write the offset and the element number into the memory.

[0125] 4.4: Traverse the elements in the container, serialize them into the array elements according to the element type of the container through the above-mentioned serialization steps for the corresponding data type, and when all elements are converted, obtain the allocated memory and its size through the memory allocator, and write the binary data of the corresponding size into the file.

[0126] The binary data stored in the above way can be directly read according to the above-mentioned serialization data structure type in the reading process, without any conversion during the reading process, solving the problem of time-consuming file content parsing. The data is organized with a certain data structure and algorithm to generate a configuration file, so that the configuration file can be used after loading, eliminating the file content parsing operation, improving the smoothness of the program running process, especially in the application scenario where there are a large number of configuration files, real-time loading of data from the disk is often required, eliminating the parsing operation, so that the data structure described by the file will not be time-consuming, making the program appear lag.

[0127] In practical applications, the updated data structure object is as follows: Figure 4 The data structure diagram shown illustrates that the basic container data structure consists of {count (quantity), offset (offset)} data. Count represents the number of container elements, and offset represents the offset of the container data in the binary file, which is the memory address obtained during data storage and retrieval. In the array (list) data structure definition, a proprietary array (list) template pre-defined in this application is used. Similarly, the mapping table data structure definition is similar to that of arrays (lists), using a pre-defined proprietary template for storage and retrieval.

[0128] This embodiment also provides an embodiment of a data reading method, which is described in detail below:

[0129] Figure 5 A flowchart of a data reading method provided in an embodiment of this application is shown, as follows: Figure 5 As shown, the specific implementation of the message reading method follows these steps:

[0130] Step S502: Obtain the field identifier.

[0131] This embodiment provides a data reading method corresponding to the above embodiment, which is used to read data after the data writing is completed using the above data processing method, and the specific implementation is as follows:

[0132] After the data is written, the binary data is saved in a binary file for later retrieval when needed. When reading the binary data, the updated data structure object in the data processing method is required. When reading the binary data in the binary file, the file list program receives the field identifier of the binary data to be read, finds the data structure object based on the field identifier, determines the data length and relative offset of the binary data to be read, and then determines the specific position of the binary data to be read in the binary file based on the data length and relative offset. Finally, the binary data is read from the binary file.

[0133] It is used in storage files to perform the writing and reading of binary data by a file listing program.

[0134] Specifically, during the reading of binary data, the file listing program needs to first obtain the field identifier corresponding to the binary data to be read, so as to find the data length and relative offset of the binary data in the data structure object.

[0135] Step S504: based on the field identifier, query the data structure object to obtain the data length and the relative offset corresponding to the field identifier.

[0136] Specifically, after receiving the character identifier of the binary data to be read, the position of the binary data to be read in the binary file needs to be located according to the character identifier. The correspondence between the character identifier of the binary data and the position information of the binary data in the binary file can be quickly learned through the updated data structure object in the data processing method.

[0137] Based on this, querying the field identifier in the data structure object avoids the case that the file list program directly searches in the binary file after receiving the character identifier of the binary data to be read. The position information of the binary data to be read in the binary file can be directly learned, and the file list program does not need to traverse the character information in the binary file, so that the message reading speed is greatly improved. Especially for multiple reading of discontinuous binary data to be read, the file list program can avoid traversing the character information in the binary file in each binary data reading process, saving the reading resources of the computing system.

[0138] Step S506: determining the target address in the binary file based on the relative offset.

[0139] Specifically, after the file list program obtains the data length and the relative offset corresponding to the field identifier of the binary data to be read, the specific reading position and the reading range of the corresponding binary data to be read in the binary file also need to be found according to the relative offset.

[0140] The target address is the specific position address of the data content of the binary data stored in the binary file.

[0141] Based on this, the file list program finds the specific position address of the data content of the corresponding binary data to be read stored in the binary file in the binary file according to the relative offset.

[0142] Further, after obtaining the relative offset of the binary data to be read, the file list program cannot directly determine the position of the binary data to be read in the binary file according to the relative offset, and further calculation is needed. In this embodiment, the specific method is as follows:

[0143] The field identifier to be queried corresponds to the segment address of the binary file. The segment address is added to the relative offset to obtain the target address of the target binary data.

[0144] The segment address is the address of the corresponding field stored in the binary file, and the target address is the address of the content of the binary data information stored in the binary file. For example, if the segment address is {number, offset 1}, the corresponding target address is {integer, …} stored in the field; if the segment address is {number, offset 2}, the corresponding target address is {character, …} stored in the field; and if the segment address is {number, offset 3}, the corresponding target address is {{integer, integer}, …} stored in the field. Figure 4

[0145] Specifically, after obtaining the character identifier of the binary data to be read, the file list program determines the data length and the relative offset of the binary data to be read in the data structure object, finds the address of the corresponding field stored in the binary file according to the field identifier of the binary data to be read, and adds the relative offset to the address to obtain the address of the content of the binary data information stored in the binary file.

[0146] For example, during the game running process, the player completes the first level of the game. The player adjusts the resolution of the game to the highest in the first level to improve the game performance. When jumping to the second level, the terminal needs to read the stored resolution data set by the player. After receiving the character identifier corresponding to the resolution data as Data.c, the file list program determines the data length and the relative offset corresponding to Data.c in the data structure object, finds the segment address of the corresponding stored field in the binary file, and adds the above-determined relative offset to obtain the address of the content of the resolution data Data.c[1] stored in the binary file.

[0147] In summary, the relative offset can be used to obtain the address information of the content of the binary data to be read, and the binary data can be quickly read during the reading process.

[0148] Step S508: reading the target binary data from the binary file according to the data length and the target address and loading the target binary data into the memory.

[0149] Specifically, after the file list program obtains the position information of the binary data to be read in the binary file, the content of the binary data stored in the position also needs to be read.

[0150] Therefore, the binary data to be read is successfully read by the file list program.

[0151] Further, after obtaining the target address and the data length of the binary data to be read, the binary data to be read can be read. In this embodiment, the specific method for solving the problem is as follows:​

[0152] add the target address and the data length to obtain a termination position of the target binary data; determine a byte range of the target binary data in the binary file based on the start position of the target binary data indicated by the target address and in combination with the termination position; sequentially read data content in the byte range to obtain the target binary data.

[0153] The start position of the binary data to be read is address information of a first byte of the binary data content stored in the binary file; and the termination position of the binary data to be read is address information of a last byte of the binary data content stored in the binary file.

[0154] Specifically, the target address obtained by the file list program is address information of a first byte of the binary data content stored in the binary file, and the file list program adds the target address and the data length to obtain address information of a last byte of the binary data content stored in the binary file. The two address information enclose a storage range of the binary data to be read in the binary file, and sequentially reading data content in the range obtains the binary data to be read.

[0155] In the above example, the file list program obtains a target address corresponding to the resolution character identifier Data.c in the binary file, i.e. address information A1 of a first byte of Data.c[1] stored in the binary file. At this time, the data length count is added to A1 to obtain address information A2 of a last byte of Data.c[1] stored in the binary file, and sequentially reading byte content in the range of A1-A2 obtains resolution data information.

[0156] In summary, the target address of the binary data to be read is a start address, the target address added to the data length is a tail address of the binary data to be read, and sequentially reading data content in the start and tail addresses can accurately read the binary data to be read.

[0157] Further, in the reading process of the binary data, not only single binary data reading but also continuous reading of multiple binary data often occurs, and after the reading of the first binary data is completed, subsequent binary data is sequentially read. In this embodiment, the specific manner of solving this problem is as follows:

[0158] save the target address and the data length of the target binary data; in case of receiving a continuous binary data reading request, determine a target address of adjacent binary data adjacent to the target binary data based on the saved target address and the data length; query the data structure object according to a field identifier of the adjacent binary data, and determine an adjacent data length of the adjacent binary data; read the adjacent binary data in the binary file based on the target address and the adjacent data length, and save the target address and the adjacent data length of the adjacent binary data.

[0159] wherein the adjacent binary data is binary data after the already read binary data, and the data length and the relative offset of the binary data after the already read binary data are an adjacent data length and an adjacent relative offset respectively.

[0160] Based on this, after reading the current binary data, the file list program saves the data length and the relative offset of the binary data, and after receiving a new binary data reading request, obtains a character identifier of the new binary data, and after detecting the data structure object based on the character identifier, finds that the new binary data is binary data after the adjacent already read binary data, adds the saved relative offset to a segment address of the already read binary data to obtain a target address of the already read binary data, adds the saved data length to the target address to obtain a termination position of the already read binary data, and then the next byte of the termination position is a starting byte of the new binary data to be read, and the data length of the new binary data to be read is obtained by querying the data structure object, and at this time, the data content of the new binary data can be read through the new binary data length and the starting byte.

[0161] For example, when the player completes the first level, the player sets the resolution to the highest and the brightness to the highest in the first level to improve the game performance, and jumps to the second level. At this time, the terminal reads the stored resolution data of the player setting, and also needs to read the brightness data stored after the resolution data. The file list program saves the data length count and the target address A1 of the resolution data. When the brightness data is read, the character identifier corresponding to the brightness data is Data.d, and the data length count_d corresponding to Data.d in the data structure object is determined. The terminal calculates the termination address A2 of the resolution data according to the count and A1, the starting position A3 of the brightness data is the next byte of A2, and the brightness data is read in the binary file according to A3 and count_d. The data length count_d of the brightness data and the target address A3 thereof are stored, so as to read subsequent binary data.

[0162] In summary, in the continuous binary data reading process, the termination position of the last read binary data is saved, so that the starting position of the next read binary data can be quickly obtained without calculation each time, thereby reducing the calculation pressure of the system when reading binary data and increasing the speed.

[0163] It should be noted that the data reading method provided by the embodiment corresponds to the data processing method provided by the above embodiment, and the same or corresponding description as in the above embodiment can be referred to the above embodiment, which will not be repeated here.

[0164] The data reading method provided by the application comprises the following steps: acquiring a field identifier of to-be-read data; acquiring a data length and a relative offset of to-be-read binary data in a data structure object according to the field identifier; and reading the to-be-read binary data from a binary file according to the data length and the relative offset. This reading method can directly load data into memory when the data is called, and the target data can be obtained without deserialization operation, so that data loading is more efficient, especially in the context of real-time data calculation, the system running efficiency is greatly increased without deserialization operation, and the problems of slow system running and lag are solved.

[0165] The embodiment combines Figure 6 The data processing method and the data reading method are further described by taking the whole process of storing and reading the configuration file in the scene by the file list program in the game as an example. Figure 6 The processing flowchart of the data processing method and the data reading method applied to the data reading and storage scene provided by an embodiment of the application comprises the following steps.

[0166] Step S602: acquiring to-be-stored data.

[0167] Specifically, during the game running, the player turns on the vertical synchronization, and the terminal needs to store the vertical synchronization data set by the player. The file list program receives the vertical synchronization data set by the player.

[0168] Step S604: detecting whether the to-be-stored data is binary data.

[0169] Specifically, after the file list program receives the vertical synchronization data set by the player, the data type of the vertical synchronization data is judged.

[0170] If it is detected that the vertical synchronization data is binary data, step S606 is executed.

[0171] If it is detected that the vertical synchronization data is not binary data, step S608 is executed.

[0172] Step S606: writing into a binary file.

[0173] Specifically, the vertical synchronization data whose data type is detected as binary data is written into a binary file by the file list program.

[0174] Step S608: converting the to-be-stored data into the to-be-stored binary data and writing into a binary file.

[0175] Specifically, if it is detected that the vertical synchronization data is in the form of 16 hexadecimal VARCHAR and is not binary data, a conversion code corresponding to the 16 hexadecimal VARCHAR is selected to convert the vertical synchronization data into binary data code, and then the converted vertical synchronization data is stored into a binary file.

[0176] Step S610: determining the data length and relative offset of the to-be-stored binary data.

[0177] Specifically, the file list program counts the number of bytes of the vertical synchronization data in the process of writing the binary vertical synchronization data into a binary file, and obtains the data length count of the vertical synchronization data.

[0178] The file list program traverses the binary file, selects a block of storage space with a rated size in the idle storage space at the front end of the binary file, and the size of the space is the number of bytes occupied by the character identifier Data.c storing the data length count and relative offset offset of the vertical synchronization data.

[0179] The first bit of the storage address of the data content of the binary vertical synchronization data in the binary file is defined as a target address A2, and then the address of the space of Data.c stored in the binary file is defined as a segment address A1, and the distance between A1 and A2 is a relative offset offset.

[0180] Step S612: updating the data structure object corresponding to the binary file based on the data length and the relative offset.

[0181] Specifically, the character identifier Data.c obtained in step S608 and the corresponding data length count and relative offset offset are updated in the data structure object, the field storing the vertical synchronization data content is defined as the character identifier Data.c[1], and the data type of the vertical synchronization data content is also updated in the data structure object. The character identifier is displayed in the field identifier of the data structure object, and the corresponding data structure and data type are stored in the file layout information.

[0182] After the vertical synchronization data is written into the binary file, a specific data reading part is further included, in this embodiment, the data is read by the file queue program, and the specific reading content is as follows:

[0183] Step S614: obtaining the field identifier.

[0184] Specifically, when the player plays the game, the scene of the game is switched at this time, and after the transition animation ends, the game interface is newly loaded, at this time, the data configured by the player needs to be read to adjust the properties of the newly loaded interface, which includes the vertical synchronization data. The file queue program obtains a request for reading the vertical synchronization data, and the request carries the field identifier Data.c of the vertical synchronization data.

[0185] Step S616: querying the data structure object based on the field identifier to obtain the data length and the relative offset corresponding to the field identifier.

[0186] Specifically, the file list program finds the corresponding data length count and relative offset offset in the data structure object according to the field identifier Data.c.

[0187] Step S618: determining a target address in the binary file based on the relative offset.

[0188] Specifically, the file list program determines that the segment address of the storage position corresponding to the field identifier Data.c in the binary file is A1, adds the relative offset offset to the segment address A1 to obtain the first byte address of the field storing the data content of the vertical synchronization data, and this address is the target address A2.

[0189] Step S620: reading target binary data in the binary file according to the data length and the target address and loading to the memory.

[0190] Specifically, the file list program adds the target address A2 and the data length count to obtain the position address A3 of the last byte of the vertical synchronization data stored in the binary file, and then reads the bytes between the addresses A2-A3 in the binary file, and the stored byte content is the binary data of the vertical synchronization data. The file list program reads the vertical synchronization data and places it in the memory for the system to read. At this time, the system does not need to deserialize the read-out vertical synchronization, but can directly apply it to construct the game screen after the scene conversion.

[0191] Step S622: saving the target address and the data length of the target binary data.

[0192] Specifically, the file list data saves the target address A2 and the data length count of the vertical synchronization data after reading the vertical synchronization data.

[0193] Step S624: sequentially reading the binary data after based on the saved target address and data length.

[0194] Specifically, after reading the vertical synchronization data, the file list program also needs to read the anti-aliasing data saved in the binary file, and the character identifier of the anti-aliasing data is Data.d.

[0195] After the file list program obtains the character identifier Data.d, it is found in the data structure object to obtain the data length count_d of the anti-aliasing data, and it is known that the storage position of the anti-aliasing data in the binary file is after the vertical synchronization data.

[0196] Since the file list program saves the target address A2 and the data length count of the vertical synchronization data, the position address A3 of the last byte of the vertical synchronization data stored in the binary file is known, and the next byte of A3 is the storage position of the first byte of the anti-aliasing data. Then the file list data points the pointer to the next bit of A3 and reads a distance of count_d bytes to read out the anti-aliasing data. The anti-aliasing data is placed in the memory to enable the system to construct the game screen after the scene conversion.

[0197] In summary, the target data can be directly loaded into the memory when called without deserialization operation, making the data loading more efficient, especially in the context of real-time data calculation, without the deserialization operation, the system running efficiency is greatly increased, solving the problem of slow system running and lag.

[0198] Corresponding to the method embodiments, the application further provides data processing device embodiments, Figure 7 A structure diagram of a data processing device is shown. As shown in the figure, Figure 7 The device comprises:

[0199] The acquisition module 702 is configured to acquire binary data to be stored and write the binary data into a binary file;

[0200] The determination module 704 is configured to determine the data length of the binary data to be stored and the relative offset corresponding to the binary data to be stored; wherein the relative offset is the distance between the target address of the binary data to be stored in the binary file and the segment address of the field where the binary data to be stored is located;

[0201] The update module 706 is configured to update the data structure object corresponding to the binary file based on the data length and the relative offset.

[0202] In an optional embodiment, the acquisition module 702 is further configured to:

[0203] Acquire data to be stored and detect the type of the data to be stored; in the case that the data to be stored is of binary data type, take the data to be stored as the binary data to be stored and write the data into the binary file; in the case that the data to be stored is not of binary data type, determine a data conversion method according to the type of the data to be stored and the data structure, convert the data to be stored into the binary data to be stored based on the data conversion method, and write the data into the binary file.

[0204] In an optional embodiment, the determination module 704 is further configured to:

[0205] Detect the number of bytes of the binary data to be stored; determine the data length of the binary data to be stored based on the number of bytes.

[0206] In an optional embodiment, the determination module 704 is further configured to:

[0207] Determine the position field corresponding to the binary data to be stored in the binary file, take the address information of the position field as a segment address; determine the target address of the binary data based on the address information of the field where the data content of the binary data to be stored is stored in the binary file; calculate the distance between the segment address and the target address, and determine the relative offset according to the calculation result.

[0208] In an optional embodiment, the updating module 706 is further configured to:

[0209] The data structure object comprises file layout information and field identification; the file layout information comprises a data structure of data stored by a field in the binary file; and the field identification comprises a field name of the field in the binary file, used to distinguish each field in the binary file.

[0210] The data processing apparatus provided in the application obtains the binary data to be stored, writes the binary data into a binary file, determines the data length of the binary data to be stored according to the number of bytes occupied by the binary data to be stored, determines the relative offset between the target address of the binary data to be stored in the binary file and the segment address of the segment where the binary data to be stored is located, and updates the data structure object according to the data length of the binary data to be stored and the relative offset. The data stored in this form can be directly loaded into the memory when called, and the target data can be obtained without deserialization operation, so that the data loading is more efficient, especially in the context of real-time data calculation, the operation of deserialization is not required, the system running efficiency is greatly increased, and the problems of slow system running and lag are solved.

[0211] It should be noted that the technical scheme of the data processing apparatus is the same as the technical scheme of the data processing method described above, and the details of the technical scheme of the data processing apparatus that are not described in detail can be referred to the description of the technical scheme of the data processing method. In addition, each component in the apparatus embodiment should be understood as a functional module that must be established to realize each step of the program flow or each step of the method. Each functional module is not an actual functional division or separation limitation. The apparatus claim defined by such a group of functional modules should be understood as a functional module architecture for realizing the solution of the computer program mainly by the description, and should not be understood as an entity apparatus for realizing the solution mainly by hardware.

[0212] Corresponding to the method embodiments described above, the application also provides data reading apparatus embodiments, Figure 8 A structure schematic diagram of a data reading apparatus provided by an embodiment of the application is shown. As shown in the figure, Figure 8 The apparatus comprises:

[0213] An identification obtaining module 802 is configured to obtain field identification;

[0214] A querying module 804 is configured to query a data structure object based on the field identification, and obtain a data length and a relative offset corresponding to the field identification;

[0215] An address confirming module 806 is configured to determine a target address in a binary file based on the relative offset.

[0216] The reading module 808 is configured to read target binary data in the binary file according to the data length and the target address and load the target binary data into the memory.

[0217] In an optional embodiment, the address confirmation module 806 is further configured to:

[0218] determine that the field identifier queried corresponds to a segment address of the binary file; and add the segment address to the relative offset to obtain the target address of the target binary data.

[0219] In an optional embodiment, the reading module 808 is further configured to:

[0220] add the target address to the data length to obtain a termination position of the target binary data; determine a byte range of the target binary data in the binary file based on a start position of the target binary data indicated by the target address and in combination with the termination position; and sequentially read data content in the byte range to obtain the target binary data.

[0221] In an optional embodiment, the data reading apparatus further includes:

[0222] an adjacent data reading module configured to save the target address and the data length of the target binary data; in a case where a continuous binary data reading request is received, determine an adjacent target address of adjacent binary data adjacent to the target binary data based on the saved target address and the data length; query the data structure object according to a field identifier of the adjacent binary data to determine an adjacent data length of the adjacent binary data; and read the adjacent binary data in the binary file based on the adjacent target address and the adjacent data length, and save the target address and the adjacent data length of the adjacent binary data.

[0223] The data reading apparatus provided in the application can obtain a field identifier of to-be-read data, and then obtain a data length and a relative offset of to-be-read binary data in a data structure object according to the field identifier, and read the to-be-read binary data in a binary file according to the data length and the relative offset. This reading method can directly load data into the memory when the data is called, and the target data can be obtained without deserialization operation, so that data loading is more efficient, and in particular, in a situation where data needs to be calculated in real time, the operation of deserialization is not needed, so that the system running efficiency is greatly increased, and the problems of slow system running and lag are solved.

[0224] It should be noted that the technical scheme of the data reading device is the same as the technical scheme of the data reading method described above, and the details of the technical scheme of the data reading device that are not described in detail can be seen from the description of the technical scheme of the data reading method. In addition, each component in the device embodiment should be understood as a functional module that must be established to implement each step of the program flow or each step of the method. Each functional module is not limited by actual functional division or separation. The device claim defined by such a set of functional modules should be understood as a functional module architecture for implementing the solution by the computer program mainly recorded in the specification, and should not be understood as an entity device for implementing the solution mainly by hardware.

[0225] Figure 9 A structural block diagram of a computing device 900 according to an embodiment of the present application is shown. The components of the computing device 900 include, but are not limited to, a memory 910 and a processor 920. The processor 920 is connected to the memory 910 through a bus 930, and a database 950 is used to save data.

[0226] The computing device 900 also includes an access device 940, which enables the computing device 900 to communicate via one or more networks 960. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 940 can include one or more of any type of network interface (e.g., a network interface card (NIC)) such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like, either wired or wireless.

[0227] In an embodiment of the present application, the above-mentioned components of the computing device 900 and other components not shown in the Figure 9 may be connected to each other, for example, through a bus. It should be understood that Figure 9 The structural block diagram of the computing device shown is only for the purpose of example, and is not a limitation on the scope of the present application. Those skilled in the art can add or replace other components as needed.

[0228] The computing device 900 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. The computing device 900 can also be a mobile or stationary server.

[0229] The processor 920 is configured to implement the data processing method and the data reading method when executing the computer executable instructions.

[0230] The above is a schematic solution of the computing device of the embodiment. It should be noted that the technical solution of the computing device belongs to the same concept as the technical solution of the data processing method and the data reading method described above, and the details of the technical solution of the computing device that are not described in detail can be referred to the description of the technical solution of the data processing method and the data reading method.

[0231] An embodiment of the present application further provides a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to implement the data processing method or the data reading method.

[0232] The above is a schematic solution of the computer readable storage medium of the embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the data processing method or the data reading method described above, and the details of the technical solution of the storage medium that are not described in detail can be referred to the description of the technical solution of the data processing method or the data reading method.

[0233] An embodiment of the present application further provides a chip, which stores a computer program, and the computer program is executed by the chip to implement the steps of the data processing method or the data reading method.

[0234] The specific embodiments of the present application are described above. Other embodiments are within the scope of the appended claims. In some cases, operations or steps described in the claims can be performed in different order than described in the embodiments and still achieve desirable results. Additionally, the process depicted in the figures can not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0235] The computer readable medium can include any entity or apparatus capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium can include appropriate additions or subtractions according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0236] It should be noted that for the foregoing method embodiments, the descriptions are expressed as a combination of a series of actions for the sake of simplicity, but those skilled in the art should know that the present application is not limited by the order of the described actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0237] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0238] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not describe all the details and do not limit the present application to the specific embodiments described. Obviously, according to the content of the present application, many modifications and changes can be made. The present application selects and describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and utilize the present application. The present application is limited by the claims and their entire scope and equivalents.< / int> < / int>

Claims

1. A data processing method, characterized by, The method comprises: acquiring binary data to be stored and writing the binary data to a binary file; determining a data length of the binary data to be stored and a relative offset corresponding to the binary data to be stored; wherein the relative offset indicates a relationship between an address at which the binary data stores the data length and the relative offset and an actual address at which the binary data content is stored; the determination of the relative offset corresponding to the binary data to be stored comprises: determining a position field corresponding to the binary data to be stored in the binary file, taking address information of the position field as a segment address; determining a target address of the binary data based on address information of a field in which data content of the binary data to be stored is stored in the binary file; calculating a distance between the segment address and the target address, and determining the relative offset according to a calculation result; updating a data structure object corresponding to the binary file based on the data length and the relative offset, and reading stored binary data according to information of the data length and the relative offset when reading the stored binary data, wherein the data structure object is used to record layout information and field identification of binary data stored in the binary file.

2. The method of claim 1, wherein, The acquiring of the binary data to be stored and the writing of the binary data to the binary file comprises: acquiring data to be stored and detecting a type of the data to be stored; in a case where the data to be stored is of a binary data type, taking the data to be stored as the binary data to be stored and writing the binary data to the binary file; in a case where the data to be stored is not of the binary data type, determining a data conversion method according to the type of the data to be stored and a data structure, converting the data to be stored into the binary data to be stored based on the data conversion method, and writing the binary data to the binary file.

3. The method of claim 1, wherein, The determination of the data length of the binary data to be stored comprises: detecting a number of bytes of the binary data to be stored; determining the data length of the binary data to be stored based on the number of bytes.

4. The method of claim 1, wherein, The file layout information comprises a data structure of data stored in a field in the binary file; the field identification comprises a field name of the field in the binary file, and is used to distinguish between the fields in the binary file.

5. A data reading method characterized by, The method comprises: acquiring field identification; identifying a query data structure object based on the field identifier, obtaining a data length and a relative offset corresponding to the field identifier, wherein the data structure object is used to record structure information and field identifiers of binary data stored in a binary file, the relative offset indicates a relationship between an address at which the binary data stores the data length and the relative offset and an actual address at which the binary data content is stored, and the relative offset is determined by the following process: determining a position field corresponding to the binary data to be stored in the binary file, taking address information of the position field as a segment address; determining a target address of the binary data based on address information of a field in which data content of the binary data to be stored is stored in the binary file; and calculating a distance between the segment address and the target address, and determining the relative offset according to a calculation result; determining that the field identifier to be queried corresponds to a segment address of the binary file; adding the segment address to the relative offset to obtain the target address of the target binary data; reading the target binary data from the binary file according to the data length and the target address and loading the target binary data into a memory.

6. The method of claim 5, wherein, After the target binary data is read from the binary file according to the data length and the target address and loaded into the memory, the method further includes: saving the target address and the data length of the target binary data; in a case where a continuous binary data reading request is received, determining a neighboring target address of neighboring binary data adjacent to the target binary data based on the saved target address and the data length; querying the data structure object according to a field identifier of the neighboring binary data, and determining a neighboring data length of the neighboring binary data; reading the neighboring binary data in the binary file based on the neighboring target address and the neighboring data length, and saving the target address and the neighboring data length of the neighboring binary data.

7. The method of claim 5, wherein, The reading of the target binary data from the binary file according to the data length and the target address includes: adding the target address and the data length to obtain a termination position of the target binary data; determining a byte range of the target binary data in the binary file based on a start position of the target binary data indicated by the target address and the termination position; sequentially reading data content in the byte range to obtain the target binary data.

8. A data processing apparatus, characterized by, The method includes: an obtaining module configured to obtain binary data to be stored and write the binary data to a binary file; The determining module is configured to determine a data length of the binary data to be stored and a relative offset corresponding to the binary data to be stored; the relative offset indicates a relationship between an address at which the binary data stores the data length and the relative offset and an actual address at which the binary data content is stored; the determining module is further configured to determine a position field corresponding to the binary data to be stored in the binary file, take address information of the position field as a segment address, determine a target address of the binary data based on address information of a field in which data content of the binary data to be stored is stored in the binary file, calculate a distance between the segment address and the target address, and determine the relative offset according to a calculation result. The updating module is configured to update a data structure object corresponding to the binary file based on the data length and the relative offset, and when reading the stored binary data, index and read the stored binary data according to information of the data length and the relative offset; the data structure object is used to record layout information and a field identifier of the binary data stored in the binary file.

9. A data reading device, characterized by The method comprises the following steps: The identifier obtaining module is configured to obtain a field identifier. The querying module is configured to query a data structure object based on the field identifier, and obtain a data length and a relative offset corresponding to the field identifier; the data structure object is used to record layout information and a field identifier of the binary data stored in the binary file; the relative offset indicates a relationship between an address at which the binary data stores the data length and the relative offset and an actual address at which the binary data content is stored; the relative offset is determined by the following process: determining a position field corresponding to the binary data to be stored in the binary file, taking address information of the position field as a segment address, determining a target address of the binary data based on address information of a field in which data content of the binary data to be stored is stored in the binary file, calculating a distance between the segment address and the target address, and determining the relative offset according to a calculation result. The address confirming module is configured to determine a segment address of the binary file corresponding to the queried field identifier, and add the segment address to the relative offset to obtain the target address of the target binary data. The reading module is configured to read the target binary data from the binary file according to the data length and the target address and load the target binary data into a memory.

10. A computing device, comprising: The method comprises the following steps: A memory and a processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to implement steps of the data processing method or the data reading method in any one of claims 1-4 or 5-7.

11. A computer-readable storage medium storing computer instructions, wherein, The instructions are executed by the processor to implement steps of the data processing method or the data reading method in any one of claims 1-4 or 5-7.

12. A computer program product, characterised in that, Computer program comprising computer instructions which, when executed by a processor, implement the steps of the data processing method or data reading method according to any one of claims 1-4 or 5-7.

Images