Generation method and device of data to be read, electronic equipment and storage medium

Abstract

The invention provides a to-be-read data generation method and device, electronic equipment and a storage medium, and relates to the technical field of data processing. The method comprises the steps that preset field information of pre-stored binary data corresponding to a data reading requirement is obtained, and the binary data comprises a plurality of fields; according to preset field information, determining head lengths and null value mark lengths corresponding to the multiple fields, and according to the head lengths, the null value mark lengths and the preset field information, determining data selection information corresponding to the multiple fields; and based on the data selection information, determining to-be-converted data from the binary data, and generating a corresponding deserialization function according to the to-be-converted data and a preset compiling function, so as to generate to-be-read data according to the deserialization function and the to-be-converted data. According to the method, the deserialization function is generated in a targeted manner, so that the data reading efficiency in a high-concurrency scene is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, for example, to a method, device, electronic device and storage medium for generating data to be read. Background Art

[0002] With the advent of the big data era, data storage, management, and efficient processing have become key challenges in the information technology industry. Key-value storage, as an efficient storage model, is widely used in large-scale distributed systems such as cache systems, databases, and search engines due to its simple data structure, flexible scalability, and efficient data query performance. By associating keys with values, key-value storage enables fast lookup, update, and delete operations, significantly improving system responsiveness.

[0003] Key-value storage systems rely on serialization and deserialization technologies to store and read data in binary form. Particularly in distributed systems, binary data transmission and storage efficiency is far higher than that of text formats. However, as data scale continues to expand, the performance requirements for serialization and deserialization are becoming increasingly stringent. Especially in high-concurrency scenarios, each read task requires multiple branches of processing, including multiple judgments and calls. This results in low deserialization efficiency, which in turn leads to low data reading efficiency. Summary of the Invention

[0004] The present application aims to provide a method, device, electronic device and storage medium for generating data to be read.

[0005] According to one aspect of the present application, a method for generating data to be read is proposed, including: obtaining preset field information of pre-stored binary data corresponding to data reading requirements, wherein the binary data includes multiple fields; determining the header length and null value flag length corresponding to each of the multiple fields according to the preset field information, and determining the data selection information corresponding to each of the multiple fields according to the header length, null value flag length and the preset field information; based on the data selection information, determining the data to be converted from the binary data, and generating a corresponding deserialization function according to the data to be converted and a preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted.

[0006] According to one aspect of the present application, a device for generating data to be read is provided, comprising: An information acquisition module, configured to acquire preset field information of pre-stored binary data corresponding to a data reading requirement, wherein the binary data includes a plurality of fields; a selection information determination module, configured to determine, based on preset field information, a header length and a null value flag length corresponding to each of the plurality of fields, and determine, based on the header length, the null value flag length, and the preset field information, data selection information corresponding to each of the plurality of fields; The deserialization processing module is used to determine the data to be converted from the binary data based on the data selection information, and generate a corresponding deserialization function according to the data to be converted and the preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted.

[0007] According to one aspect of the present application, an electronic device is provided, comprising: a processor; and a memory storing a computer program, wherein when the computer program is executed by the processor, the processor executes the method as described above.

[0008] According to one aspect of the present application, a non-transitory computer-readable medium is provided, on which readable instructions are stored. When the instructions are executed by a processor, the processor is caused to execute the method described above.

[0009] It should be understood that the foregoing general description and the following detailed description are merely illustrative and are not restrictive of the present application.

[0010] Beneficial effects: The above-described embodiments of this application dynamically generate deserialization functions tailored to specific data read requirements, thus avoiding the impact of identical processing on task accuracy in high-concurrency scenarios. Jump instructions are directly generated based on the header length and null flag of binary data, enabling zero-copy data parsing. This approach is particularly suitable for high-frequency key-value storage scenarios, improving overall data reading efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without exceeding the scope of protection required by this application.

[0012] Figure 1 A flowchart of a method for generating data to be read provided in an embodiment of the present application; Figure 2 A schematic diagram of the composition of field information provided in an embodiment of the present application; Figure 3 A schematic diagram of the field composition provided in the embodiment of the present application; Figure 4 A block diagram of a device for generating data to be read provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. Like reference numerals in the drawings represent like or similar parts, and thus repetitive description thereof will be omitted.

[0014] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0015] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0016] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0017] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Thus, the first component discussed below could be referred to as the second component without departing from the teachings of the present invention. As used herein, the term "and / or" includes any one and all combinations of one or more of the associated listed items.

[0018] For specific implementation methods, please refer to the following embodiments.

[0019] Figure 1 This is a flow chart of a method for generating data to be read provided in an embodiment of the present application. Figure 1 As shown, the method includes: step S10, step S11 and step S12.

[0020] In step S10, preset field information of pre-stored binary data corresponding to a data reading requirement is obtained, wherein the binary data includes a plurality of fields.

[0021] In this application, a data read requirement can be used to represent the data a user wants to read. In a key-value store, data is stored in binary form, relying on serialization. When storing this data, it can be stored as multiple fields, and field information for each field can be pre-set (i.e., preset field information), which can include the field type, whether the field can be nullable, and the field length.

[0022] In some implementations, the data to be read may be determined according to data reading requirements, and the data may be treated as binary data. Preset field information of the binary data may be obtained.

[0023] In other implementations, you can define a vector vector <metawithjit>metaWithJits, used to indicate the storage location of the deserialization function, its structure reference Figure 2 The custom MetaWithJit structure contains: a reference to the Meta object, which is used to store preset field information, including attributes such as field type, binary data offset, size, and nullability. jitted_deserialize_func: A pointer to a function compiled by JIT (Just-In-Time, dynamically compiling code at runtime), which is the storage method of the deserialization function. It is designed to extract field values at runtime. The parameter is of type Value and is used to store the extraction result.

[0024] In step S11, the header length and null value flag length corresponding to each of the multiple fields are determined according to the preset field information, and the data selection information corresponding to each of the multiple fields is determined according to the header length, null value flag length and the preset field information.

[0025] In this application, for new data to be read, it is necessary to determine the deserialization function used for the corresponding binary data and store it. The header length can be used to represent the data length of the header information of each field, and the null value flag length can be the data length of the null value flag of each field. The control flag length can be pre-calculated and stored in the preset field information, or it can be calculated in real time based on the preset field information and the preset calculation function. The data selection information can be used to represent the selection method of the data that actually needs to be deserialized, that is, the starting position and the ending position of the selection.

[0026] In some implementations, the header length and null value flag length corresponding to each of the multiple fields can be extracted from the preset field information. Figure 3 Each field can be composed of header information, a null value flag, and serialized data. The preset field information can include the field length of the field. Data selection information can be calculated based on the field length, header information, null value flag length, and a preset data selection calculation function.

[0027] In step S12, based on the data selection information, the data to be converted is determined from the binary data, and a corresponding deserialization function is generated according to the data to be converted and a preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted.

[0028] In this application, based on data selection information, standard library functions such as memcpy can be called to determine the data to be converted from the binary data. A compilation function can be pre-set, such as the function of a C++ front-end compiler, to compile the data to be converted into an LLVM (Low Level Virtual Machine, compiler infrastructure) intermediate representation (IR) file, i.e., a deserialization function, which is in machine code form. The data to be converted is substituted into the deserialization function for processing to obtain the data to be read, and the data to be read can be sent to the user.

[0029] In some implementations, LLVM's just-in-time (JIT) compiler generates a function called jitted_deserialize for each field to extract the field value. This function extracts the value from the data structure based on the field type and calls the previously defined deserialization function to perform the actual data extraction. In a specific implementation, you can first obtain the JIT-compiled code generation instance llvmCodeGen and load the IR file containing the deserialization function. This instructs the LLVM compiler to always inline the deserialization function when generating code to optimize performance.

[0030] Create an LLVM IRBuilder instance, builder, to build the IR and create a new JIT-compiled LLVM function prototype, jitted_deserialize, which will contain the call logic for the deserialization function. Define the deserialization function's parameters, with arg0 representing the pointer type of the return value (Value). Use llvmCodeGen's getIntConstant method (a common helper method in the LLVM CodeGen process) to generate integer constants corresponding to the preset field information, representing the field type, nullability, offset in the data, and size. Use builder.CreateCall (a core method of the LLVM IRBuilder class, used to create a function call instruction in the generated LLVM IR) to call the predefined deserialize function, passing in the relevant parameters (including arg0 and constant values for various fields). At this point, deserialize extracts the field values from the given binary data and stores the extracted results in arg0. The generated deserialization function is handed over to the JIT compiler and compiled into directly executable machine code, and the function pointer of the machine code is stored in the jitted_deserialize_func pointer of the corresponding field in metaWithJits (that is, the storage location of the deserialization function) for subsequent efficient calls.

[0031] In other implementations, if a user wants to read certain data on a certain platform / site (data reading requirement), the platform / site can write this data in the form of binary data to the function generation server of this application. The function generation server will generate a corresponding deserialization function based on the data reading requirement to convert the binary data for the user to view. If the user's data reading requirement is an outdated requirement, that is, it has been written to the function generation server before, then this read can directly call the already generated deserialization function.

[0032] This application dynamically generates deserialization functions tailored to specific data read requirements, avoiding the impact of identical processing on task accuracy in high-concurrency scenarios. Based on the header length of the binary data (e.g., a 4-byte length prefix) and the null flag (e.g., a 1-bit Nullable flag), jump instructions are directly generated, enabling zero-copy data parsing. This approach is particularly well-suited for high-frequency key-value storage scenarios, improving overall data read efficiency.

[0033] According to some embodiments, the header length and field null value status corresponding to each of the multiple fields can be extracted from the preset field information; when the field null value status is nullability, the number of nullable fields corresponding to each of the multiple fields is determined according to the preset field information, and the corresponding null value flag length is determined according to the preset type field processing function and the number of nullable fields; when the field null value status is non-nullability, the corresponding null value flag length is determined to be 0; based on the header length, the null value flag length and the preset field information, the data selection information corresponding to each of the multiple fields is determined.

[0034] In this application, the field null value status can be used to indicate whether the field is nullable, that is, whether the location storing serialized data in the field can be empty. The type field processing function can be a predefined function named deserialize, whose main purpose is to process various types of fields in binary data structures, including integers, strings, floating-point numbers, and dates.

[0035] In some implementations, the header length and field null status corresponding to each of the multiple fields can be extracted from the preset field information. If the field null status is nullable, the number of nullable fields corresponding to each of the multiple fields can be obtained from the preset field information, that is, the number of fields controlled by the serialization function. The number of nullable fields is substituted into the type field processing function for calculation to obtain the null flag length. Based on the header length, null flag length, and the preset field information, the data selection information corresponding to each of the multiple fields can be determined.

[0036] If the field null value status is non-nullable, the null value flag length is 0, so as to determine the data selection information corresponding to each of the multiple fields based on the header length, the null value flag length and the preset field information.

[0037] This application can further determine the null value status of the field, and determine the corresponding null value flag length for different field null value statuses, which facilitates the determination of subsequent data selection information and improves the accuracy of the data.

[0038] According to some embodiments, the preset field information includes a field type. A field offset and a length of binary data can be extracted from the preset field information; the header length, the null value flag length, and the field offset are summed to determine an offset of serialized data in the binary data; a corresponding data selection length is determined based on the field type; and data selection information is determined based on the offset of the serialized data and the data selection length.

[0039] In this application, the preset field information may include a field offset, that is, the position of the header of any field relative to the entire serialized data. The header length, the null value flag length, and the field offset are summed, and the resulting value is used as the offset of the serialized data in the binary data, that is, the starting position of the data to be read. The data selection length corresponding to different field types can be preset. For example, for an integer (int) type, 4 bytes are directly copied from the calculated data offset and interpreted as an integer value, that is, the data selection length is 4 bytes. Therefore, the preset data selection length can be found based on the field type.

[0040] You can start from the starting position corresponding to the offset of the serialized data, move the data selection length, and get the end position of the binary data to be converted. The offset (starting position), end position, and data selection length of the serialized data can be used as data selection information as a whole.

[0041] This application can accurately calculate the actual offset of serialized data in binary data by summing the header length, the null value flag length and the field offset extracted from the preset field information. This operation effectively eliminates the interference of non-business data such as header information and null value flags on the positioning of serialized data. The appropriate data selection length is selected by the field type, and the data selection information is determined based on the offset of the serialized data and the data selection length calculated above. This process clearly and reasonably integrates the complex data positioning and selection logic to form a set of efficient and accurate data selection mechanism. In actual applications, the serialized data corresponding to the required fields can be extracted quickly and accurately from binary data, which greatly improves the efficiency of data processing, reduces unnecessary calculations and data reading operations, and saves system resources.

[0042] According to some embodiments, when the field null value status is nullability, the empty flag in the preset field information can be detected to determine whether multiple fields are empty; if any field among the multiple fields is empty, it is determined that any field does not meet the data reading requirements; the number of nullable fields corresponding to the fields that are not empty among the multiple fields is obtained; and the corresponding null value flag length is determined based on the preset type field processing function and the number of nullable fields.

[0043] In this application, when it is determined that the empty value state of the field is nullable, it is further determined whether the field is empty.

[0044] In some implementations, it is possible to detect Figure 3 The null flag shown in the is used to determine whether each field is empty. Different null flag values (e.g., 0 or 1) can be pre-set to correspond to whether a field is empty or not. If a field is empty, it is identified as not meeting the data reading requirements and is not processed further. For fields that are not empty, the corresponding preset number of controllable fields can be obtained. The number of nullable fields can then be substituted into the type field processing function for calculation to obtain the null value flag length.

[0045] This application, under the condition that the field null value state has nullability, determines whether multiple fields are empty by detecting the empty flag in the preset field information, and can accurately identify which fields in the data may have null values. For subsequent data processing, the data boundary and valid data range are clarified, avoiding the mistaken use of null value fields as valid data for calculation, storage or analysis and other operations, effectively improving the accuracy of data processing, and reducing data errors or logical confusion caused by null value interference. When it is detected that any field among multiple fields is empty, it is determined that the field does not meet the data reading requirements, and the invalid data is filtered in time. The fields that do not meet the requirements are excluded in the early stage of the data processing process, reducing the processing burden of invalid data in the subsequent processing process. The number of nullable fields corresponding to the non-empty fields in multiple fields is obtained. By accurately grasping the number of non-empty fields, the storage space for marking null values in the data structure can be more accurately planned to ensure the rationality and efficiency of the data structure.

[0046] According to some embodiments, the corresponding data to be converted can be determined from the binary data based on the data selection function and the data selection information; based on the compilation function, the data to be converted is compiled to generate the corresponding machine code, and the machine code is determined as a deserialization function; according to the deserialization function, the data to be converted is deserialized to generate the data to be read.

[0047] In this application, the data selection function can be called in accordance with the data selection information in the above manner to determine the function to be converted from the binary data, and then the compilation function can be used to compile the data to be converted to generate machine code that can be understood and executed by the server as a deserialization function. The deserialization function is used to deserialize the data to be converted to generate the data to be read. For example, for a fixed-length string field, the selection function can truncate it according to a fixed length; for a variable-length structure field, the selection function can accurately extract it based on a specific length identifier or terminator.

[0048] This application works in conjunction with the data selection function and the data selection information determined based on key information such as the header length, the null value flag length, and the field offset, to accurately locate and select the data to be converted from the binary data. This avoids blindly searching or excessively reading irrelevant data in huge binary data, significantly reducing the waste of resources in the data processing process. Based on the preset compilation function, the data to be converted is compiled to generate the corresponding machine code, and it is determined as the deserialization function. This dynamic compilation method can flexibly generate a matching deserialization function based on different data structures, data types, or business requirements to be converted. There is no need to pre-write a large amount of fixed-pattern deserialization code, which greatly enhances the adaptability to different data formats and business changes.

[0049] According to some embodiments, the field type may be extracted from the preset field information; and based on the field type, the data selection functions corresponding to the plurality of fields are determined.

[0050] In this application, field types can include certificate types, string types, and other complex types such as DATE, TIME, and DATETIME. Different types of fields are handled differently. For example, for integer types, 4 bytes are directly copied from the calculated starting position and interpreted as an integer value. For string types, the function reads the offset and length of the string to construct a corresponding string object. Therefore, different data selection functions can be used. In some implementations, data selection functions corresponding to different field types may be preset. After the field type is extracted from the preset field information, the data selection function for each field may be matched accordingly.

[0051] This application can accurately extract the field type from the preset field information to determine the data category represented by each field. Differentiated processing strategies can be adopted according to the characteristics of different field types to avoid "one-size-fits-all" extensive processing, significantly enhancing the pertinence and professionalism of data processing. Based on the extracted field type, corresponding data selection functions are determined for each of the multiple fields. Different types of data have different characteristics such as storage format, length, and alignment in binary. By matching a dedicated data selection function for each field type, data extraction can be performed accurately according to the storage rules of that type of data.

[0052] According to some embodiments, the deserialization function may be stored in a preset function storage location in the preset field information.

[0053] You can refer to Figure 2 , the preset function storage location can correspond to Figure 2 The location of the JIT function in .

[0054] This application achieves unified management of deserialization functions through the standardized design of preset function storage locations. The design of fixed storage locations reduces function search time and optimizes the execution efficiency of deserialization operations.

[0055] The following describes an apparatus embodiment of the present application, which can be used to perform the method embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, reference can be made to the method embodiment of the present application.

[0056] Figure 4 This is a block diagram of a device for generating data to be read provided in an embodiment of the present application. Figure 4 As shown, the device 400 for generating data to be read includes an information acquisition module 401 , a selected information determination module 402 and a deserialization processing module 403 .

[0057] The information acquisition module 401 is used to acquire preset field information of pre-stored binary data corresponding to the data reading requirement, wherein the binary data includes multiple fields; A selection information determination module 402 is configured to determine a header length and a null value flag length corresponding to each of the plurality of fields based on the preset field information, and to determine data selection information corresponding to each of the plurality of fields based on the header length, the null value flag length, and the preset field information; The deserialization processing module 403 is used to determine the data to be converted from the binary data based on the data selection information, and generate a corresponding deserialization function according to the data to be converted and a preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted.

[0058] Optionally, the selection information determination module 402 is specifically configured to: Extract the header length and field null value status corresponding to each of the multiple fields from the preset field information; When the field null value state is nullable, the number of nullable fields corresponding to each of the multiple fields is determined according to the preset field information, and the corresponding null value flag length is determined according to the preset type field processing function and the number of nullable fields; When the field null value state is non-nullable, the corresponding null value flag length is determined to be 0; According to the header length, the null value flag length and the preset field information, the data selection information corresponding to each of the multiple fields is determined.

[0059] Optionally, the preset field information includes a field type; when the selection information determination module 402 determines the data selection information corresponding to each of the multiple fields based on the header length, the null value flag length, and the preset field information, it is specifically configured to: Extract the field offset and length of binary data from the preset field information; The header length, the null flag length, and the field offset are summed to determine the offset of the serialized data in the binary data; Determine the corresponding data selection length according to the field type; The data selection information is determined based on the offset of the serialized data and the data selection length.

[0060] Optionally, when the field null value state is nullability, the selection information determination module 402 determines the number of nullable fields corresponding to each of the multiple fields according to preset field information, and determines the corresponding null value flag length according to a preset type field processing function and the number of nullable fields, specifically for: When the field null value state is nullable, the null flag in the preset field information is detected to determine whether multiple fields are null; If any of the multiple fields is empty, it is determined that any field does not meet the data reading requirements; Get the number of nullable fields corresponding to the non-null fields in multiple fields; Determine the corresponding null value flag length based on the preset type field processing function and the number of nullable fields.

[0061] Optionally, the deserialization processing module 403 is specifically configured to: Determine corresponding data to be converted from the binary data according to the data selection function and the data selection information; Based on the compilation function, compile the data to be converted to generate corresponding machine code, and determine the machine code as a deserialization function; The data to be converted is deserialized according to the deserialization function to generate the data to be read.

[0062] Optionally, the device 400 for generating data to be read further includes a function determination module 404, configured to: Extract field type from preset field information; According to the field type, determine the data selection function corresponding to multiple fields.

[0063] Optionally, the device 400 for generating data to be read further includes a function storage module 405 for: The deserialized function is stored in the preset function storage location in the preset field information.

[0064] The device performs functions similar to the method provided above. For other functions, please refer to the previous description and will not be repeated here.

[0065] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 5 As shown, the electronic device 500 of this embodiment may include: a memory 501 and a processor 502.

[0066] The memory 501 stores a computer program. When the computer program is executed by the processor 502 , the processor 502 executes the method in the above embodiment.

[0067] The processor 502 and the memory 501 are connected, for example, via a bus.

[0068] Optionally, the electronic device 500 may further include a transceiver. It should be noted that in actual applications, the number of transceivers is not limited to one, and the structure of the electronic device 500 does not constitute a limitation on the embodiments of the present application.

[0069] Processor 502 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. Processor 502 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, or a combination of a DSP and a microprocessor.

[0070] A bus includes a path that transmits information between the components mentioned above. Examples include a PCI (Peripheral Component Interconnect) bus and an EISA (Extended Industry Standard Architecture) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the diagram uses a single thick line, but this does not imply a single bus or type of bus.

[0071] The memory 501 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0072] The memory 501 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 502. The processor 502 is used to execute the application code stored in the memory 501 to implement the content shown in the above method embodiment.

[0073] Electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. They may also include servers, etc. Figure 5 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0074] The electronic device of this embodiment can be used to execute the method of any of the above embodiments. The implementation principles and technical effects are similar and will not be described in detail here.

[0075] The present application also provides a non-transitory computer-readable storage medium having computer-readable instructions stored thereon. When the aforementioned instructions are executed by a processor, the processor executes the method in the above embodiment.

[0076] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a non-transitory computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0077] The embodiments of the present application are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only intended to help understand the method and core ideas of the present application. At the same time, changes or modifications made by those skilled in the art based on the ideas of the present application, the specific implementation methods, and the scope of application of the present application, all fall within the scope of protection of the present application. In summary, the contents of this specification should not be construed as limiting the present application.< / metawithjit>

Claims

1. A method for generating data to be read, characterized in that: include: Acquire preset field information of pre-stored binary data corresponding to a data reading requirement, wherein the binary data includes a plurality of fields; Determine, according to the preset field information, a header length and a null value flag length corresponding to each of the multiple fields, and determine, according to the header length, the null value flag length, and the preset field information, data selection information corresponding to each of the multiple fields; Based on the data selection information, the data to be converted is determined from the binary data, and a corresponding deserialization function is generated according to the data to be converted and a preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted.

2. The method according to claim 1, characterized in that The step of determining, based on the preset field information, a header length and a null value flag length corresponding to each of the multiple fields, and determining, based on the header length, the null value flag length, and the preset field information, data selection information corresponding to each of the multiple fields, includes: Extracting the header length and field null value status corresponding to each of the multiple fields from the preset field information; In the case where the field null value state is nullability, determining the number of nullable fields corresponding to each of the multiple fields according to the preset field information, and determining the corresponding null value flag length according to a preset type field processing function and the number of nullable fields; When the null value state of the field is non-nullable, the corresponding null value flag length is determined to be 0; According to the header length, the null value flag length and the preset field information, data selection information corresponding to each of the multiple fields is determined.

3. The method according to claim 1 or 2, characterized in that The preset field information includes field type; The step of determining the data selection information corresponding to each of the plurality of fields according to the header length, the null value flag length, and the preset field information includes: Extracting the field offset and the length of the binary data from the preset field information; Summing the header length, the null flag length, and the field offset to determine an offset of serialized data in the binary data; Determine the corresponding data selection length according to the field type; The data selection information is determined based on the offset of the serialized data and the data selection length.

4. The method according to claim 2, characterized in that In the case where the field null value state is nullability, determining the number of nullable fields corresponding to each of the multiple fields according to the preset field information, and determining the corresponding null value flag length according to a preset type field processing function and the number of nullable fields, including: In a case where the field null value state is nullability, detecting a null flag in the preset field information to determine whether the multiple fields are null; If any of the multiple fields is empty, determining that the field does not meet the data reading requirement; Obtain the number of nullable fields corresponding to the non-null fields in the multiple fields; The corresponding null value flag length is determined according to the preset type field processing function and the number of nullable fields.

5. The method according to claim 1, wherein The method of determining the data to be converted from the binary data based on the data selection information, and generating a corresponding deserialization function according to the data to be converted and a preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted, includes: Determining the corresponding data to be converted from the binary data according to a data selection function and the data selection information; Compiling the data to be converted based on the compilation function to generate corresponding machine code, and determining the machine code as the deserialization function; The data to be converted is deserialized according to the deserialization function to generate the data to be read.

6. The method according to claim 5, characterized in that Also includes: Extracting a field type from the preset field information; The data selection functions corresponding to each of the multiple fields are determined according to the field type.

7. The method according to claim 1, characterized in that Also includes: The deserialization function is stored in a preset function storage location in the preset field information.

8. A device for generating data to be read, characterized in that: include: An information acquisition module, configured to acquire preset field information of pre-stored binary data corresponding to a data reading requirement, wherein the binary data includes a plurality of fields; a selection information determination module, configured to determine, based on the preset field information, a header length and a null value flag length corresponding to each of the plurality of fields, and determine, based on the header length, the null value flag length, and the preset field information, data selection information corresponding to each of the plurality of fields; The deserialization processing module is used to determine the data to be converted from the binary data based on the data selection information, and generate a corresponding deserialization function according to the data to be converted and a preset compilation function, so as to generate the data to be read according to the deserialization function and the data to be converted.

9. An electronic device, characterized in that: include: processor; A memory storing a computer program, which, when executed by the processor, causes the processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that Computer-readable instructions are stored thereon, and when the instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 7.

Images