Method and apparatus for generating data to be read, electronic device, and storage medium

Abstract

The application provides a data generation method and device, electronic equipment and storage medium, and relates to the technical field of data processing. The method comprises the following steps: obtaining preset field information of pre-stored binary data corresponding to a data reading requirement, wherein the binary data comprises a plurality of fields; determining the respective head length and null flag length of the plurality of fields according to the preset field information, and determining the respective data selection information of the plurality of fields according to the head length, the null flag length and the preset field information; determining the data to be converted from the binary data based on the data selection information, and generating a corresponding deserialization function according to the data to be converted and a preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted. The application generates the deserialization function in a targeted manner, thereby improving the data reading efficiency in a high-concurrency scenario.

Description

Technical Field

[0001] This application relates to the field of data processing technology, such as a method, apparatus, electronic device, and storage medium for generating data to be read. Background Technology

[0002] With the advent of the big data era, data storage, management, and efficient processing have become key challenges in the field of information technology. Key-value stores, as an efficient storage model, are widely used in large-scale distributed systems, such as caching systems, databases, and search engines, due to their simple data structure, flexible scalability, and high-performance data query capabilities. By associating keys with values, key-value stores enable fast lookup, update, and delete operations, significantly improving system response speed.

[0003] In related technologies, key-value storage systems rely on serialization and deserialization techniques to store and retrieve data in binary form. This is particularly true in distributed systems, where the transmission and storage efficiency of binary data is far superior to that of text formats. However, as data volumes continue to increase, the performance requirements for serialization and deserialization become increasingly stringent. Especially in high-concurrency scenarios, each read task requires multiple checks, calls, and other branching processes, leading to lower deserialization efficiency and consequently, lower data retrieval efficiency. Summary of the Invention

[0004] This application aims to provide a method, apparatus, electronic device, and storage medium for generating data to be read.

[0005] According to one aspect of this application, a method for generating data to be read is proposed, comprising: obtaining preset field information of pre-stored binary data corresponding to the data reading requirement, wherein the binary data includes multiple fields; determining the header length and null flag length corresponding to each of the multiple fields according to the preset field information, and determining the data selection information corresponding to each of the multiple fields according to the header length, null flag length and preset field information; determining the data to be converted from the binary data based on the data selection information, and generating a corresponding deserialization function according to the data to be converted and a preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted.

[0006] According to one aspect of this application, a data generation apparatus is provided, comprising:

[0007] The information acquisition module is used to acquire preset field information of pre-stored binary data corresponding to the data reading requirements. The binary data includes multiple fields.

[0008] The information selection module is used to determine the header length and null flag length of each of the multiple fields according to the preset field information, and to determine the data selection information corresponding to each of the multiple fields according to the header length, null flag length and preset field information.

[0009] The deserialization module is used to determine the data to be converted from binary data based on data selection information, and to generate the corresponding deserialization function according to the data to be converted and the preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted.

[0010] According to one aspect of this application, an electronic device is provided, comprising: a processor; and a memory storing a computer program that, when executed by the processor, causes the processor to perform the method described above.

[0011] According to one aspect of this application, a non-transitory computer-readable medium is proposed, on which readable instructions are stored, which, when executed by a processor, cause the processor to perform the method described above.

[0012] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application.

[0013] Beneficial effects:

[0014] The embodiments provided in this application dynamically generate deserialization functions tailored to specific data reading needs, avoiding the impact on task processing accuracy caused by performing the same processing on each reading task in high-concurrency scenarios. Based on the header length and null flag of the binary data, jump instructions are directly generated, achieving zero-copy data parsing. This is particularly suitable for high-frequency key-value storage scenarios, thus improving overall data reading efficiency. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings, without exceeding the scope of protection claimed by this application.

[0016] Figure 1 A flowchart illustrating the method for generating data to be read provided in this application embodiment;

[0017] Figure 2 This is a schematic diagram illustrating the composition of field information provided in the embodiments of this application;

[0018] Figure 3 This is a schematic diagram of the field composition provided in the embodiments of this application;

[0019] Figure 4 A block diagram of a data generation apparatus provided in an embodiment of this application;

[0020] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0022] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0023] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0024] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0025] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this application. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.

[0026] For specific implementation details, please refer to the following examples.

[0027] Figure 1 A flowchart illustrating a method for generating data to be read, as provided in an embodiment of this application. Figure 1 As shown, the method includes steps S10, S11 and S12.

[0028] In step S10, the preset field information of the pre-stored binary data corresponding to the data reading requirement is obtained, wherein the binary data includes multiple fields.

[0029] In this application, data retrieval requirements can be used to characterize the data a user wants to read. In a key-value storage scenario, data is stored in binary form, relying on serialization. When storing this data, it can be stored as multiple fields, and the field information (i.e., preset field information) of each field can be pre-defined, including field type, whether the field can be nullable, and field length.

[0030] In some implementations, the data to be read can be determined based on data reading requirements, and this data can be treated as binary data. Preset field information of the binary data can then be retrieved.

[0031] In other implementations, a vector can be defined. <metawithjit>`metaWithJits` is used to represent the storage location of the deserialization function; its structure is referenced in [reference needed]. Figure 2 The custom MetaWithJit structure contains: a reference to a Meta object, which stores predefined field information, including field type, binary data offset, size, and nullability. `jitted_deserialize_func` is a pointer to a function generated by JIT (Just-In-Time) compilation, i.e., the storage method for the deserialization function. It aims to extract field values ​​at runtime, and its parameter is of type Value, used to store the extracted result.

[0032] In step S11, based on the preset field information, the header length and null flag length of each of the multiple fields are determined, and based on the header length, null flag length and preset field information, the data selection information of each of the multiple fields is determined.

[0033] In this application, for new data to be read, it is necessary to determine and store the deserialization function used for its corresponding binary data. The header length can be used to characterize the data length of the header information for each field, and the null flag length can be the data length of the null flag for each field. The control flag length can be pre-calculated and stored in preset field information, or it can be calculated in real-time based on preset field information and a preset calculation function. Data selection information can be used to characterize the selection method of the data that actually needs to be deserialized, i.e., the start and end positions of the selection.

[0034] In some implementations, the header length and null flag length for each of the multiple fields can be extracted from preset field information. (See reference.) Figure 3 Each field can consist of header information, a null flag, and serialized data. The preset field information can include the field length, and data selection information can be calculated based on the field length, header information, null flag length, and a preset data selection calculation function.

[0035] In step S12, based on the data selection information, the data to be converted is determined from the binary data, and a corresponding deserialization function is generated according to the data to be converted and the preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted.

[0036] In this application, information can be selected based on the data, and standard library functions such as memcpy can be called to determine the data to be converted from the binary data. A compilation function, such as the functionality of a C++ front-end compiler, can be pre-set to compile the data to be converted into an LLVM (Low Level Virtual Machine, compiler infrastructure) intermediate representation (IR) file, i.e., a deserialization function, which is in machine code form. The data to be converted is then substituted into the deserialization function for processing to obtain the data to be read, which is then sent to the user.

[0037] In some implementations, LLVM's Just-In-Time (JIT) compilation generates a function `jitted_deserialize` for each field to extract its value. This function extracts the value from the data structure based on the field type and calls the previously defined deserialization function for the actual data extraction. In practice, a JIT-compiled code generation instance `llvmCodeGen` can be obtained first, and an IR file containing the deserialization function can be loaded. This instructs the LLVM compiler to always inline the deserialization function during code generation to optimize performance.

[0038] Create an LLVM IRBuilder instance `builder` to construct the IR and create a new JIT-compiled LLVM function prototype `jitted_deserialize`, which will contain the calling logic for the deserialization function. Define the parameters of the deserialization function; `arg0` represents the pointer type of the return value, i.e., `Value`. Use the `getIntConstant` method of `llvmCodeGen` (a commonly used helper method in the LLVM code generation `CodeGen` process) to generate corresponding integer constants for the predefined field information, representing information such as field type, nullability, offset in the data, and size. Use `builder.CreateCall` (a core method of the IRBuilder class in LLVM, used to create function call instructions in the generated LLVM IR) to call the predefined `deserialize`, passing in the relevant parameters (including `arg0` and the constant values ​​of various fields). At this point, `deserialize` will extract the field values ​​from the given binary data and store the extraction results in `arg0`. The generated deserialization function is handed over to the JIT compiler to be compiled into directly executable machine code. The function pointer of the machine code is stored in the jitted_deserialize_func pointer of the corresponding field in metaWithJits (i.e., the storage location of the deserialization function) for efficient subsequent calls.

[0039] In other implementations, when a user wants to read certain data on a platform / site (data read request), the platform / site can write this data into the function generation server of this application in binary data form. The function generation server will generate a corresponding deserialization function based on the data read request to convert the binary data for the user to view. If the user's data read request is an old request, i.e., it has been written to the function generation server before, then the current read can directly call the already generated deserialization function.

[0040] This application avoids the impact of performing the same processing on each read task in high-concurrency scenarios by dynamically generating deserialization functions tailored to specific data reading needs. Based on the header length of the binary data (e.g., a 4-byte length prefix) and the null flag (e.g., a 1-bit Nullable flag), jump instructions are directly generated, achieving zero-copy data parsing. This is particularly suitable for high-frequency key-value storage scenarios, significantly improving overall data reading efficiency.

[0041] According to some embodiments, the header length and null value status of each of the multiple fields can be extracted from the preset field information; when the field null value status is nullable, the number of nullable fields corresponding to each of the multiple fields is determined according to the preset field information, and the corresponding null value flag length is determined according to the preset type field processing function and the number of nullable fields; when the field null value status is non-nullable, the corresponding null value flag length is determined to be 0; and the data selection information corresponding to each of the multiple fields is determined according to the header length, null value flag length, and preset field information.

[0042] In this application, the field null value status can be used to characterize whether a field can be nullable, that is, whether the location in the field storing serialized data can be empty. The type field processing function can be a predefined function named deserialize, whose main purpose is to process various type fields in binary data structures, including integers, strings, floating-point numbers, and dates.

[0043] In some implementations, the header length and null value status of multiple fields can be extracted from preset field information. If the field null value status is nullable, the number of nullable fields corresponding to each field can be obtained from the preset field information, i.e., the number of fields controlled by the serialization function. This number of nullable fields is then substituted into the type field processing function for calculation to obtain the null flag length. Based on the header length, null flag length, and preset field information, the data selection information corresponding to each of the multiple fields can be determined.

[0044] If the field's null value status is non-nullable, the null value flag length is 0, so that the data selection information corresponding to each of the multiple fields can be determined based on the header length, the null value flag length, and the preset field information.

[0045] This application can further determine the null value status of a field, and determine the corresponding null value flag length for different field null value statuses, which facilitates the determination of subsequent data selection information and improves the accuracy of the data.

[0046] According to some embodiments, the preset field information includes the field type. The field offset and the length of the binary data can be extracted from the preset field information; the header length, null flag length, and field offset are summed to determine the offset of the serialized data in the binary data; the corresponding data selection length is determined according to the field type; and data selection information is determined based on the offset of the serialized data and the data selection length.

[0047] In this application, the preset field information may include field offsets, i.e., the position of the header of any field relative to the entire serialized data. The sum of the header length, null flag length, and field offset is used as the offset of the serialized data in the binary data, i.e., the starting position of the data to be read. The data selection length corresponding to different field types can be preset. For example, for integer (int) types, 4 bytes are directly copied from the calculated data offset and interpreted as an integer value, i.e., the data selection length is 4 bytes. Therefore, the preset data selection length can be found based on the field type.

[0048] Starting from the beginning position corresponding to the offset of the serialized data, the data selection length can be moved to obtain the end position of the binary data to be converted. The offset (start position), end position, and data selection length of the serialized data can be used as the data selection information.

[0049] This application can accurately calculate the actual offset of serialized data in binary data by summing the header length, null flag length, and field offset extracted from preset field information. This operation effectively eliminates interference from non-business data such as header information and null flags on the location of serialized data. An appropriate data selection length is selected based on the field type, and the data selection information is determined based on the calculated offset of the serialized data and the data selection length. This process clearly and rationally integrates complex data location and selection logic, forming an efficient and accurate data selection mechanism. In practical applications, it can quickly and accurately extract the serialized data corresponding to the required fields from binary data, greatly improving data processing efficiency, reducing unnecessary calculations and data reading operations, and saving system resources.

[0050] According to some embodiments, when the field null value state is nullable, the null flag in the preset field information can be detected to determine whether multiple fields are null; if any field among the multiple fields is null, it is determined that any field does not meet the data reading requirements; the number of nullable fields corresponding to the non-nullable fields among the multiple fields is obtained; and the corresponding null value flag length is determined according to the preset type field processing function and the number of nullable fields.

[0051] In this application, after determining that the field's nullable state is nullable, it is further determined whether the field is nullable.

[0052] In some implementations, it is possible to detect, such as Figure 3 The null flag shown is used to determine whether each field is empty. Different null flags (e.g., 0 or 1) can be preset to correspond to whether a field is empty or not. If a field is empty, it indicates that the field does not meet the data reading requirements and will not be processed further. For non-empty fields, the corresponding preset controllable number of fields can be obtained. Then, the number of nullable fields can be substituted into the type field processing function for calculation to obtain the null flag length.

[0053] This application, under the condition that fields are nullable, determines whether multiple fields are null by detecting null flags in preset field information, accurately identifying which fields in the data may contain null values. For subsequent data processing, it clarifies data boundaries and the range of valid data, avoiding the misinterpretation of null fields as valid data in calculations, storage, or analysis, effectively improving the accuracy of data processing and reducing data errors or logical confusion caused by null value interference. When any field among multiple fields is detected to be null, it is determined that the field does not meet the data reading requirements, and invalid data is filtered out in a timely manner. Fields that do not meet the requirements are excluded in the early stages of the data processing flow, reducing the burden of processing invalid data in subsequent processes. By obtaining the number of nullable fields corresponding to the non-nullable fields among multiple fields, and accurately grasping the number of non-nullable fields, the storage space used for marking null values ​​in the data structure can be planned more precisely, ensuring the rationality and efficiency of the data structure.

[0054] According to some embodiments, the corresponding data to be converted can be determined from binary data based on the data selection function and data selection information; the data to be converted can be compiled based on the compilation function to generate the corresponding machine code, and the machine code can be determined as the deserialization function; the data to be converted can be deserialized according to the deserialization function to generate the data to be read.

[0055] In this application, following the method described above, a data selection function is called based on the data selection information to determine the function to be converted from the binary data. Then, a compilation function is used to compile the data to be converted, generating machine code that can be understood and executed by the server, which serves as the deserialization function. The deserialization function is then used to deserialize the data to be converted, generating the data to be read. For example, for fixed-length string fields, the selection function can truncate the string to a fixed length; for variable-length structure fields, the selection function can accurately extract the data based on a specific length identifier or terminator.

[0056] This application utilizes a data selection function in conjunction with data selection information determined based on key data such as header length, null flag length, and field offset to accurately locate and select the data to be converted from binary data. This avoids blindly searching through massive amounts of binary data or over-reading irrelevant data, significantly reducing resource waste during data processing. Based on a pre-defined compilation function, the data to be converted is compiled to generate corresponding machine code, which is then used as the deserialization function. This dynamic compilation method can flexibly generate matching deserialization functions according to different data structures, data types, or business requirements. It eliminates the need to pre-write a large amount of fixed-pattern deserialization code, greatly enhancing adaptability to different data formats and business changes.

[0057] According to some embodiments, field types can be extracted from preset field information; based on the field types, data selection functions corresponding to each of the multiple fields can be determined.

[0058] In this application, field types can include certificate types, string types, and other complex types such as DATE, TIME, and DATETIME. Different field types are handled differently. For example, for integer types, four bytes are directly copied from the calculated starting position and interpreted as an integer value. For string types, the function reads the string's offset and length to construct a corresponding string object. Therefore, different data selection functions can be used.

[0059] In some implementations, data selection functions corresponding to different field types can be pre-defined. After extracting the field types from the preset field information, the data selection function for each field can be matched accordingly.

[0060] This application, by accurately extracting field types from preset field information, can determine the data category represented by each field. Different processing strategies can be adopted based on the characteristics of different field types, avoiding a "one-size-fits-all" approach and significantly enhancing the targeting and professionalism of data processing. Based on the extracted field types, corresponding data selection functions are determined for each field. Different types of data have different storage formats, lengths, alignment methods, and other characteristics in binary. By matching a specific data selection function to each field type, data can be accurately extracted according to the storage rules of that data type.

[0061] According to some embodiments, the deserialization function can be stored in a preset function storage location in preset field information.

[0062] You can refer to this. Figure 2 The storage location of this preset function can correspond to Figure 2 The location of the JIT function in the code.

[0063] This application achieves unified management of deserialization functions through a standardized design of preset function storage locations. The fixed storage location design reduces function lookup time and optimizes the execution efficiency of deserialization operations.

[0064] The following describes an apparatus embodiment of this application, which can be used to perform the method embodiment of this application. For details not disclosed in the apparatus embodiment of this application, please refer to the method embodiment of this application.

[0065] Figure 4 A block diagram of a data generation apparatus provided in an embodiment of this application. (e.g.) Figure 4 As shown, the data generation device 400 includes an information acquisition module 401, an information selection and determination module 402, and a deserialization processing module 403.

[0066] The information acquisition module 401 is used to acquire preset field information of pre-stored binary data corresponding to the data reading requirements, wherein the binary data includes multiple fields;

[0067] The selection information determination module 402 is used to determine the header length and null value flag length of each of the multiple fields according to the preset field information, and to determine the data selection information corresponding to each of the multiple fields according to the header length, null value flag length and preset field information;

[0068] The deserialization processing module 403 is used to determine the data to be converted from binary data based on data selection information, and generate a corresponding deserialization function according to the data to be converted and a preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted.

[0069] Optionally, the selection information determination module 402 is specifically used for:

[0070] Extract the header length and null value status of each of the preset fields from the preset field information;

[0071] When a field is nullable, the number of nullable fields corresponding to each field is determined based on the preset field information, and the length of the corresponding null flag is determined based on the preset type field processing function and the number of nullable fields.

[0072] When the field's null value status is non-nullable, the length of the corresponding null value flag is set to 0;

[0073] Based on the header length, null flag length, and preset field information, determine the data selection information corresponding to each of the multiple fields.

[0074] Optionally, the preset field information includes the field type; when the selection information determination module 402 determines the data selection information corresponding to each of the multiple fields based on the header length, the null flag length, and the preset field information, it is specifically used for:

[0075] Extract the field offset and the length of the binary data from the preset field information;

[0076] Sum the header length, null flag length, and field offset to determine the offset of the serialized data in the binary data;

[0077] Determine the corresponding data selection length based on the field type;

[0078] The data selection information is determined based on the offset of the serialized data and the data selection length.

[0079] Optionally, when the field null value determination module 402 determines the number of nullable fields corresponding to each of the multiple fields based on preset field information, and determines the corresponding null value flag length based on preset type field processing functions and the number of nullable fields, it is specifically used for:

[0080] When a field is nullable, check the null flag in the preset field information to determine whether multiple fields are null.

[0081] If any of the multiple fields is empty, then that field is determined to be unsuitable for data retrieval.

[0082] Get the number of nullable fields corresponding to the non-nullable fields among multiple fields;

[0083] The length of the null flag is determined based on the preset type field processing function and the number of nullable fields.

[0084] Optionally, the deserialization module 403 is specifically used for:

[0085] Based on the data selection function and data selection information, determine the corresponding data to be converted from the binary data;

[0086] Based on the compilation function, the data to be converted is compiled to generate the corresponding machine code, and the machine code is used as the deserialization function.

[0087] The deserialization function is used to deserialize the data to be converted, in order to generate the data to be read.

[0088] Optionally, the data generation device 400 further includes a function determination module 404, used for:

[0089] Extract field types from preset field information;

[0090] Based on the field type, determine the data selection function corresponding to each of the multiple fields.

[0091] Optionally, the data generation device 400 further includes a function storage module 405, used for:

[0092] Store the deserialization function in the preset function storage location in the preset field information.

[0093] The device performs functions similar to those described above; other functions are described in the preceding descriptions and will not be repeated here.

[0094] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 5 As shown, the electronic device 500 of this embodiment may include a memory 501 and a processor 502.

[0095] The memory 501 stores a computer program, which, when executed by the processor 502, causes the processor 502 to perform the method described in the above embodiments.

[0096] The processor 502 and the memory 501 are connected, for example, via a bus.

[0097] Optionally, the electronic device 500 may also include a transceiver. It should be noted that in practical applications, the transceiver is not limited to one, and the structure of the electronic device 500 does not constitute a limitation on the embodiments of this application.

[0098] Processor 502 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 502 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0099] A bus can include a pathway for transmitting information between the aforementioned components. The bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one thick line is used in the diagram, but this does not imply that there is only one bus or one type of bus.

[0100] The memory 501 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0101] The memory 501 is used to store application code that executes the solution of this application, and its execution is controlled by the processor 502. The processor 502 is used to execute the application code stored in the memory 501 to implement the content shown in the foregoing method embodiments.

[0102] Electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Servers can also be included. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0103] The electronic device in this embodiment can be used to execute the method of any of the above embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0104] This application also provides a non-transitory computer-readable storage medium storing computer-readable instructions thereon, which, when executed by a processor, cause the processor to perform the method as described in the above embodiments.

[0105] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a non-transitory computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0106] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. Furthermore, any changes or modifications made by those skilled in the art based on the ideas of this application, and on the specific implementation methods and application scope of this application, are all within the scope of protection of this application. Therefore, the content of this specification should not be construed as a limitation of this application.< / metawithjit>

Claims

1. A method for generating data to be read, characterized in that, include: Obtain preset field information of pre-stored binary data corresponding to the data reading requirement, wherein the binary data includes multiple fields; Based on the preset field information, determine the header length and null flag length corresponding to each of the multiple fields, and based on the header length, the null flag length, and the preset field information, determine the data selection information corresponding to each of the multiple fields. Based on the data selection information, the data to be converted is determined from the binary data, and a corresponding deserialization function is generated according to the data to be converted and a preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted; The preset field information includes field types; wherein, determining the data selection information corresponding to each of the plurality of fields based on the header length, the null value flag length, and the preset field information includes: Extract the field offset and the length of the binary data from the preset field information; The header length, the null flag length, and the field offset are summed to determine the offset of the serialized data in the binary data; Determine the corresponding data selection length based on the field type; The data selection information is determined based on the offset of the serialized data and the data selection length.

2. The method according to claim 1, characterized in that, The step of determining the header length and null flag length corresponding to each of the plurality of fields based on the preset field information, and determining the data selection information corresponding to each of the plurality of fields based on the header length, the null flag length, and the preset field information, includes: Extract the header length and field null value status of each of the multiple fields from the preset field information; When the field is nullable, the number of nullable fields corresponding to each of the multiple fields is determined according to the preset field information, and the length of the corresponding null flag is determined according to the preset type field processing function and the number of nullable fields. When the field is in a non-nullable state, the length of the corresponding null value flag is set to 0. Based on the header length, the null value flag length, and the preset field information, determine the data selection information corresponding to each of the multiple fields.

3. The method according to claim 2, characterized in that, When the field's null value state is nullable, the number of nullable fields corresponding to each of the multiple fields is determined according to the preset field information, and the corresponding null value flag length is determined according to the preset type field processing function and the number of nullable fields, including: When the field is nullable, the null flag in the preset field information is detected to determine whether the multiple fields are null. If any of the multiple fields is empty, then it is determined that any field does not meet the data reading requirements; Obtain the number of nullable fields corresponding to the non-nullable fields among the multiple fields; The length of the null value flag is determined based on the preset type field processing function and the number of nullable fields.

4. The method according to claim 1, characterized in that, The process of determining the data to be converted from the binary data based on the data selection information, and generating a corresponding deserialization function based on the data to be converted and a preset compilation function, to generate data to be read based on the deserialization function and the data to be converted, includes: Based on the data selection function and the data selection information, the corresponding data to be converted is determined from the binary data; Based on the compilation function, the data to be converted is compiled to generate corresponding machine code, and the machine code is determined as the deserialization function; The data to be converted is deserialized according to the deserialization function to generate the data to be read.

5. The method according to claim 4, characterized in that, Also includes: Extract the field type from the preset field information; Based on the field type, determine the data selection function corresponding to each of the multiple fields.

6. The method according to claim 1, characterized in that, Also includes: The deserialization function is stored in the preset function storage location in the preset field information.

7. A device for generating data to be read, characterized in that, include: The information acquisition module is used to acquire preset field information of pre-stored binary data corresponding to the data reading requirement, wherein the binary data includes multiple fields; The selection information determination module is used to determine the header length and null value flag length corresponding to each of the multiple fields according to the preset field information, and to determine the data selection information corresponding to each of the multiple fields according to the header length, the null value flag length and the preset field information; The deserialization processing module is used to determine the data to be converted from the binary data based on the data selection information, and to generate a corresponding deserialization function according to the data to be converted and a preset compilation function, so as to generate data to be read according to the deserialization function and the data to be converted; The preset field information includes field types; wherein, when the selection information determination module determines the data selection information corresponding to each of the multiple fields based on the header length, the null value flag length, and the preset field information, it is specifically used for: Extract the field offset and the length of the binary data from the preset field information; The header length, the null flag length, and the field offset are summed to determine the offset of the serialized data in the binary data; Determine the corresponding data selection length based on the field type; The data selection information is determined based on the offset of the serialized data and the data selection length.

8. An electronic device, characterized in that, include: processor; A memory storing a computer program that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-6.

Images