Code generation method and device, equipment and storage medium

Abstract

The invention discloses a code generation method and device, equipment and a storage medium, and relates to the field of the Internet of Things, and the method comprises the steps: reading an initial data structure definition file and an initial mapping file; performing topological sorting based on the dependency relationship data object to obtain a target sorting result, and performing preprocessing operation on the initial mapping file to generate a processed mapping file; obtaining a preset coding code generation template set and a preset decoding code generation template set according to a preset coding demand; analyzing the target sorting result to obtain a data type, and determining a target coding code generation template and a target decoding code generation template according to the data type; filling the target sorting result and the processed mapping file into a code generation template by utilizing a preset template engine to obtain a current coding and decoding code; and generating a target test code, and verifying the current coding and decoding code through the target test code to determine a target coding and decoding code. And efficient, reliable and safe MessagePack coding and decoding are provided for an embedded system.

Description

Technical Field

[0001] This invention relates to the field of the Internet of Things, and in particular to a code generation method, apparatus, device, and storage medium. Background Technology

[0002] In the fields of IoT and embedded systems, data exchange between devices and between devices and the cloud is crucial. Serialization is the process of converting internal device data structures, such as sensor readings and device status, into a standard format for network transmission or storage. MessagePack is widely used in embedded systems due to its advantages of being more compact and faster to parse than JSON.

[0003] However, embedded systems typically have stringent resource constraints:

[0004] 1. Limited memory: Random access memory and flash memory typically have very small capacities.

[0005] 2. Weak processing power: The computing power of microcontrollers is far lower than that of general-purpose processors.

[0006] 3. High reliability requirements: Many embedded systems, such as automotive electronics and industrial control, require stable operation over a long period of time and cannot allow system crashes due to memory leaks, fragmentation, or abnormal behavior.

[0007] In conclusion, how to provide efficient and reliable secure MessagePack encoding and decoding for embedded systems is an urgent problem to be solved. Summary of the Invention

[0008] In view of this, the purpose of this invention is to provide a code generation method, apparatus, device, and storage medium capable of providing efficient, reliable, and secure MessagePack encoding and decoding for embedded systems. The specific solution is as follows:

[0009] Firstly, this application provides a code generation method applied to an encoding / decoding code generator, comprising:

[0010] Read the initial data structure definition file and initial mapping file input by the user; the data structure definition file includes the definition of fields, data types, dependencies and organization methods of each data object, and the initial mapping file is a mapping file between keys and enumerations;

[0011] Based on the aforementioned dependencies, a topological sorting operation is performed on the data objects in the initial data structure definition file to obtain the target sorting result, and a preprocessing operation is performed on the initial mapping file to generate a processed mapping file;

[0012] A set of preset encoding code generation templates is obtained based on preset encoding requirements, and a set of preset decoding code generation templates is obtained based on preset decoding requirements using zero-copy technology and preset memory pool allocation strategy; wherein, the template set includes templates corresponding to each preset language;

[0013] The target sorting result is parsed to obtain the data type of the data object in the target sorting result. Based on the data type, the target encoding code generation template and the target decoding code generation template are determined from the preset encoding code generation template set and the preset decoding code generation template set.

[0014] The target sorting result and the processed mapping file are filled into the target encoding code generation template and the target decoding code generation template using a preset template engine to obtain the current encoding and decoding code;

[0015] Target test code is generated based on the data objects used for testing in the target sorting result. The current codec is verified using the target test code, and the target codec is determined based on the verification result.

[0016] Optionally, the step of performing a topological sorting operation on the data objects in the initial data structure definition file based on the dependency relationship to obtain the target sorting result, and performing a preprocessing operation on the initial mapping file to generate a processed mapping file, includes:

[0017] Parse the initial data structure definition file to obtain the data structure definition information of each data object defined in the initial data structure definition file;

[0018] Construct a dependency graph between the data objects based on the dependencies in the definition information of each data structure.

[0019] Perform a topological sorting operation on the dependency graph;

[0020] If the result indicates that the topological sorting operation was successfully executed, then the target sorting result is generated.

[0021] If the operation result indicates that the topology sorting operation has failed, the preset topology sorting operation is aborted and a preset error message is sent to the user terminal.

[0022] Iterate through the fields of each data object in the initial data structure definition file and read the initial mapping file to determine new keys that exist in the initial data structure definition file but are not defined in the initial mapping file;

[0023] Assign a corresponding target enumeration value to the new key, and update the initial mapping file based on the new key and the corresponding target enumeration value to obtain the processed mapping file.

[0024] Optionally, the step of parsing the target sorting result to obtain the data type of the data object in the target sorting result, and determining the target encoding code generation template and the target decoding code generation template from the preset encoding code generation template set and the preset decoding code generation template set according to the data type, includes:

[0025] Parse the target sorting result to determine the data type of the corresponding data object;

[0026] An intermediate structure is defined based on the data type of the data object; wherein, the intermediate structure is used to map the serialized format data type to a static structure representation;

[0027] Based on the intermediate structure, the corresponding target encoding code generation template and target decoding code generation template are selected from the preset encoding code generation template set and the preset decoding code generation template set.

[0028] Optionally, the target test code includes binary code test files and unit test code;

[0029] Accordingly, generating target test code based on the data objects used for testing in the target sorting result includes:

[0030] Determine whether the target sorting result contains a data object for testing;

[0031] If it exists, the current encoding code in the generated current encoding / decoding code is used to serialize the preset test data corresponding to the data object to generate a binary code test file;

[0032] Based on the decoding and verification requirements of the binary code test file, corresponding unit test code is generated; wherein the unit test code includes at least one test case.

[0033] Optionally, the step of verifying the current codec using the target test code and determining the target codec based on the verification result includes:

[0034] The target test code is used to read the binary code test file, and the current decoding function in the current encoding / decoding code is called to obtain the corresponding decoding result;

[0035] The decoding result is compared and verified with the preset test data;

[0036] If the verification result indicates that the decoding result is consistent with the preset test data, then the current codec is determined to be successfully verified, so that the user terminal can identify the current codec as the target codec.

[0037] If the verification result indicates that the decoding result is inconsistent with the preset test data, the current codec verification is determined to be unsuccessful. The user terminal then modifies the files and code templates related to the current codec to obtain a new current codec, and jumps back to the step of generating target test code based on the target sorting result, and verifying the current codec through the target test code, until the verification result indicates that the decoding result is consistent with the preset test data.

[0038] Optionally, the step of obtaining a preset decoding code generation template set based on preset decoding requirements using zero-copy technology and a preset memory pool allocation strategy includes:

[0039] The preset memory pool allocation logic for each template in the preset decoding code generation template set is determined based on the preset memory pool allocation strategy; wherein, the preset memory pool allocation logic is to select a preset memory pool for memory management during the decoding process, and the preset memory pool includes an internal static memory pool or an external memory pool.

[0040] The data type decoding logic of each template in the preset decoding code generation template set is determined according to the zero-copy technology; wherein, the data type decoding logic includes decoding logic for string or binary data types, array decoding logic, and unknown key decoding logic using zero-copy technology;

[0041] Based on the preset memory pool allocation logic and the data type decoding logic, a preset decoding code generation template set is determined.

[0042] Optionally, the step of determining the data type decoding logic of each template in the preset decoding code generation template set according to the zero-copy technology includes:

[0043] Based on the memory in the preset memory pool, read the key-value pairs of the target codec from the target codec;

[0044] Determine the type of the key in the target decoding code based on the reading results;

[0045] If the key is of type string or binary data, then a preset in-situ read function is called based on zero-copy technology, and the pointer is directly pointed to the position corresponding to the string or binary data in the input buffer;

[0046] If the key is of type array, then the corresponding memory is allocated from the preset memory pool to decode the corresponding field;

[0047] If the key is of unknown type, then the corresponding field is skipped.

[0048] Secondly, this application provides a code generation apparatus for use in an encoding / decoding code generator, comprising:

[0049] The file reading module is used to read the initial data structure definition file and the initial mapping file input by the user. The data structure definition file includes the definition of the fields, data types, dependencies and organization methods of each data object. The initial mapping file is a mapping file between keys and enumerations.

[0050] The result and file acquisition module is used to perform a topological sorting operation on the data objects in the initial data structure definition file based on the dependency relationship to obtain the target sorting result, and to perform a preprocessing operation on the initial mapping file to generate a processed mapping file;

[0051] The template set acquisition module is used to acquire a preset encoding code to generate a template set based on preset encoding requirements, and to acquire a preset decoding code to generate a template set based on preset decoding requirements using zero-copy technology and a preset memory pool allocation strategy; wherein, the template set includes templates corresponding to each preset language;

[0052] The template determination module is used to parse the target sorting result to obtain the data type of the data object in the target sorting result, and determine the target encoding code generation template and the target decoding code generation template from the preset encoding code generation template set and the preset decoding code generation template set according to the data type;

[0053] The code acquisition module is used to fill the target encoding code generation template and the target decoding code generation template with the target sorting result and the processed mapping file using a preset template engine, so as to obtain the current encoding and decoding code;

[0054] The code determination module is used to generate target test code based on the data objects used for testing in the target sorting result, verify the current codec code using the target test code, and determine the target codec code based on the verification result.

[0055] Thirdly, this application provides an electronic device, comprising:

[0056] Memory, used to store computer programs;

[0057] A processor for executing the computer program to implement the code generation method as described above.

[0058] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the code generation method described above.

[0059] In summary, this application reads an initial data structure definition file and an initial mapping file input by the user. The data structure definition file includes definitions of fields, data types, dependencies, and organization methods for each data object. The initial mapping file is a mapping file between keys and enumerations. Based on the dependencies, a topological sorting operation is performed on the data objects in the initial data structure definition file to obtain the target sorting result. A preprocessing operation is then performed on the initial mapping file to generate a processed mapping file. A preset encoding code generation template set is obtained according to preset encoding requirements, and a preset decoding code generation template set is obtained based on preset decoding requirements using zero-copy technology and a preset memory pool allocation strategy. The template set includes various... A template corresponding to a preset language is used; the target sorting result is parsed to obtain the data type of the data object in the target sorting result; based on the data type, a target encoding code generation template and a target decoding code generation template are determined from the preset encoding code generation template set and the preset decoding code generation template set; the target sorting result and the processed mapping file are used to fill the target encoding code generation template and the target decoding code generation template using a preset template engine to obtain the current encoding / decoding code; target test code is generated based on the data object used for testing in the target sorting result; the current encoding / decoding code is verified using the target test code; and the target encoding / decoding code is determined based on the verification result. As described above, this application first reads and preprocesses the initial data structure definition file and the initial mapping file to obtain the target sorting result and the processed mapping file. Then, based on the preset encoding and decoding requirements, it obtains the corresponding code generation template set by combining zero-copy technology and memory pool allocation strategy. The target encoding and decoding template is selected by parsing the data object type in the topology sorting result. Subsequently, the template engine is used to fill the template with relevant data to generate the current encoding and decoding code. Simultaneously, test code is generated based on the test objects in the topology sorting result to complete verification. Finally, the target encoding and decoding code is determined based on the verification result. In this way, by completely replacing string keys with compact integer enumeration values, data redundancy is fundamentally eliminated, and the serialized data volume is greatly compressed, thus significantly saving transmission bandwidth and storage space. At the same time, all encoding and decoding codes are automatically generated from a unified, human-readable data structure definition file, and topology sorting and other technologies ensure that even complex nested structures can generate correct, compilable code, completely freeing developers from tedious and error-prone manual coding. Furthermore, unit test code and data matching the business code are automatically generated, realizing definition-as-test, greatly reducing the cost of writing and maintaining test code. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0061] Figure 1 This is a flowchart of a code generation method disclosed in this application;

[0062] Figure 2 This is a flowchart of a specific document preprocessing method disclosed in this application;

[0063] Figure 3 This is a schematic diagram of a specific decoding code logic disclosed in this application;

[0064] Figure 4 This is a system framework diagram of a specific code generation method disclosed in this application;

[0065] Figure 5 This is a schematic diagram of a specific source code and verification code generation process disclosed in this application;

[0066] Figure 6 This is a schematic diagram of the structure of a code generation device disclosed in this application;

[0067] Figure 7 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] Currently, data exchange between devices and between devices and the cloud is crucial in the fields of IoT and embedded systems. Serialization is the process of converting internal device data structures, such as sensor readings and device status, into a standard format for network transmission or storage. MessagePack is widely used in the embedded field due to its advantages of being more compact and faster to parse than JSON. However, embedded systems typically have stringent resource constraints: limited memory (RAM and flash memory capacities are usually very small); weak processing power (microcontrollers have much lower computing power than general-purpose processors); and high reliability requirements (many embedded systems, such as automotive electronics and industrial control, require stable operation over long periods and cannot tolerate system crashes due to memory leaks, fragmentation, or abnormal behavior). To address these technical problems, this application discloses a code generation method, apparatus, device, and storage medium that can provide efficient, reliable, and secure MessagePack encoding and decoding for embedded systems.

[0070] See Figure 1 As shown, this embodiment of the invention discloses a code generation method applied to an encoding / decoding code generator, comprising:

[0071] Step S11: Read the initial data structure definition file and initial mapping file input by the user terminal; the data structure definition file includes the definition of fields, data types, dependencies and organization methods of each data object, and the initial mapping file is a mapping file between keys and enumerations.

[0072] In this embodiment, the initial data structure definition file (schema.json) and the initial mapping file (key_enums.json) are first obtained. It's important to know that the initial data structure definition file is a JSON file used to describe the data structure, including field names and types, and can provide `default_data` for automatic test generation. The initial mapping file is a mapping file between keys and enumerations, storing the mapping relationship between all field names and their corresponding unique integers. The code generator automatically maintains this file.

[0073] Step S12: Perform a topological sorting operation on the data objects in the initial data structure definition file based on the dependency relationship to obtain the target sorting result, and perform a preprocessing operation on the initial mapping file to generate a processed mapping file.

[0074] In this embodiment, as Figure 2As shown, the initial data structure definition file is parsed to obtain the data structure definition information of each data object defined in the initial data structure definition file; a dependency graph between the data objects is constructed based on the dependencies in each of the data structure definition information; a topological sorting operation is performed on the dependency graph; if the operation result indicates that the topological sorting operation was successful, a target sorting result is generated; if the operation result indicates that the topological sorting operation failed, the preset topological sorting operation is aborted and a preset error message is sent to the user terminal; the fields of each data object in the initial data structure definition file are traversed, and the initial mapping file is read to determine new keys that exist in the initial data structure definition file but are not defined in the initial mapping file; a corresponding target enumeration value is assigned to the new key, and the initial mapping file is updated based on the new key and the corresponding target enumeration value to obtain a processed mapping file. Specifically, the initial data structure definition file is parsed to extract the specific structure information of each data object, including field definitions and dependencies between objects. A dependency graph is constructed based on these dependencies, where nodes represent data objects and edges represent dependencies between objects. For example, if the field type of TVehicleData is TPressureInfo, a dependency edge is established from TVehicleData to TPressureInfo. Then, to ensure that when generating the C header file, structures such as TpressureInfo that are dependent on it are always defined before structures such as TvehicleData that depend on it, a topological sorting operation is performed on the graph to verify the acyclicity of dependencies. This fundamentally solves compilation errors caused by improper definition order in C, especially for complex nested structures. If the sorting is successful, an ordered target sorting result is generated, indicating the processing order; if the sorting fails, such as due to circular dependencies in the dependency graph, the operation is immediately stopped and a preset error message is sent to the user. Afterwards, the fields of all data objects in the initial data structure definition file are traversed, and the initial mapping file is read and compared to identify new keys that appear in the data structure definition but are not defined in the mapping file. For these new keys, corresponding target enumeration values ​​are automatically assigned, and these key-value pairs are added to the mapping file, thus updating the initial mapping file and finally generating the processed mapping file. The entire process ensures the synchronization and consistency between the data structure and the mapping file.

[0075] Step S13: Obtain a set of preset encoding code generation templates according to preset encoding requirements, and obtain a set of preset decoding code generation templates based on preset decoding requirements according to zero-copy technology and preset memory pool allocation strategy; wherein, the template set includes templates corresponding to each preset language.

[0076] In this embodiment, a set of code templates for generating encoding functions is obtained based on preset encoding requirements. Simultaneously, to meet the requirements of high-performance data transmission, zero-copy technology and a preset memory pool allocation strategy are used as key design principles, and another set of code templates specifically for generating decoding function code is obtained based on preset decoding requirements. It should be noted that, in order to determine the preset decoding code generation template set, such as... Figure 3 As shown, the preset memory pool allocation logic for each template in the preset decoding code generation template set is determined based on a preset memory pool allocation strategy. The preset memory pool allocation logic involves selecting a preset memory pool for memory management during the decoding process. The preset memory pool includes an internal static memory pool or an external memory pool. The data type decoding logic for each template in the preset decoding code generation template set is determined according to the zero-copy technology. The data type decoding logic includes decoding logic for string or binary data types using zero-copy technology, array decoding logic, and unknown key decoding logic. The preset decoding code generation template set is determined based on the preset memory pool allocation logic and the data type decoding logic. Specifically, firstly, the memory management logic is determined based on the preset memory pool allocation strategy. A dedicated preset memory pool allocation logic is predefined for each template in the decoding template set. When decoding data that requires storage space, such as arrays or nested objects, the decoding function obtains memory from a memory pool (Arena). Specific options typically fall into two categories: one is an internal static memory pool, where the decoding function internally links to a static, fixed-size global memory pool (g_default_arena_buffer). If the caller does not provide a custom memory pool (i.e., the arena parameter is NULL), the decoding function will automatically enable this default pool and reset it to 0 using a pointer at the beginning of each top-level decoding call. Alternatively, there is an external memory pool. The caller can also create and manage its own memory pool and pass the address of the memory pool as the arena parameter to the decoding function. This is crucial in multi-threaded environments or scenarios that require finer memory control, providing great flexibility.

[0077] In addition, to determine the data type decoding logic, key-value pairs of the target decoding code are read from the memory in the preset memory pool. The type of the key in the target decoding code is determined based on the reading result. If the key type is a string or binary data, a preset in-place read function is called based on zero-copy technology, directly pointing the pointer to the position corresponding to the string or binary data in the input buffer. If the key type is an array, corresponding memory is allocated from the preset memory pool to decode the corresponding field. If the key type is an unknown key, the corresponding field is skipped. Specifically, the key-value pair data processed by the target decoding code is first read from the preset memory pool. The specific type of the key to be decoded is analyzed and determined based on the reading result. Next, drastically different decoding strategies are adopted according to different key types. When decoding string or binary data, the generated code achieves true zero-copy by calling the `mpack_read_bytes_inplace` function. It does not allocate new memory and copy data, but directly points a pointer to the position of the data in the input buffer, while recording its length. This fundamentally eliminates unnecessary memory allocation and data copying overhead. When the key is an array, the decoding process requires creating a separate data structure for the array itself and its elements. In this case, a contiguous block of memory is allocated from the selected memory pool to hold the decoded array content. When an unknown key is encountered—that is, when the key is not in the current mapping table or schema definition—the entire field associated with this key, including its corresponding value, is skipped without any decoding operation or memory allocation, and subsequent data processing continues.

[0078] Step S14: Parse the target sorting result to obtain the data type of the data object in the target sorting result, and determine the target encoding code generation template and the target decoding code generation template from the preset encoding code generation template set and the preset decoding code generation template set according to the data type.

[0079] In this embodiment, to map the flexible map and nested array types in MessagePack to static C language structures, the generator automatically creates temporary, internally used intermediate structures. The target sorting result is parsed to determine the data type of the corresponding data object; an intermediate structure is defined based on the data type of the data object; wherein, the intermediate structure is used to map the serialized format data type to a static structure representation; according to the intermediate structure, the corresponding target encoding code generation template and target decoding code generation template are selected from the preset encoding code generation template set and the preset decoding code generation template set. Specifically, the target sorting result obtained after topological sorting is first parsed to identify the specific data type corresponding to each data object. This step clarifies which types of data need to be encoded and decoded. To achieve efficient code generation, temporary intermediate structures are defined for dynamic composite types in serialization formats such as MessagePack, such as map or nested array. For example, for map...<uint8,bool> The type generates a TMapU8Bool structure containing uint8_tkey_ and boolvalue_ members. This allows C code to represent a map using an array of structures. The intermediate structure acts as a conversion bridge, mapping the flexible key-value pair set in the serialized data to a well-defined, static array of structures in C, enabling subsequent encoding and decoding operations to be performed on the defined type. After defining these intermediate structures, a clear, static C language type representation corresponding to the original data type is obtained. Based on these finalized types, templates matching the intermediate structures are precisely selected from the previously prepared sets of preset encoding and decoding code generation templates. These selected templates become the target encoding and decoding code generation templates.

[0080] Step S15: Use a preset template engine to fill the target sorting result and the processed mapping file into the target encoding code generation template and the target decoding code generation template to obtain the current encoding and decoding code.

[0081] In this embodiment, all the core information obtained in the previous steps—namely, the target sorting result representing the correct generation order and the processed mapping file containing the latest key-value mapping relationships—is provided as data input to a preset template engine, such as the Inja template engine. The template engine fills these two parts of data into carefully selected target encoding code generation templates and target decoding code generation templates, respectively. Variables and logical control structures in the templates are instantiated based on the input data, such as lists of data objects, enumeration values, and field types, thereby generating specific, compilable C / C++ code text. Finally, this process produces a complete current encoding / decoding code file. As shown in the example, the output includes a header file (_mpack.h) containing structure and function declarations, a C language implementation file (_mpack.c), and a C++ wrapper file (_mpack.hpp).

[0082] Step S16: Generate target test code based on the data objects used for testing in the target sorting result, verify the current codec using the target test code, and determine the target codec based on the verification result.

[0083] In this embodiment, to verify the generated current codec file, target test code needs to be generated first. The target test code includes a binary code test file (*.msgpack) and unit test code (_test.cpp). It is determined whether a data object for testing exists in the target sorting result; if so, the preset test data corresponding to the data object is serialized using the current encoding code in the generated current codec file, generating a binary code test file. Based on the decoding verification requirements of the binary code test file, corresponding unit test code is generated; the unit test code includes at least one test case. Specifically, if the message definition in schema.json defines a data object `default_data` for testing, the encoding logic of its current codec file is called to serialize this default data into a standard MessagePack binary file, such as `TVehicleData.msgpack`. Simultaneously, a C++ unit test file, such as `generated_mpack_test.cpp`, is created, containing one test case. This test case automatically reads the generated `.msgpack` file, calls the decoding function of the generated current codec file, and then compares the decoded result with the content of `default_data` one by one.

[0084] Next, the target test code is used to read the binary code test file, and the current decoding function in the current codec is called to obtain the corresponding decoding result. The decoding result is then compared and verified with the preset test data. If the verification result indicates that the decoding result is consistent with the preset test data, the current codec is deemed to have been successfully verified, so that the user terminal can identify the current codec as the target codec. If the verification result indicates that the decoding result is inconsistent with the preset test data, the current codec is deemed to have failed to verify, so that the user terminal can modify the files and code templates related to the current codec to obtain a new current codec, and then jump back to the step of generating the target test code according to the target sorting result, and verifying the current codec using the target test code, until the verification result indicates that the decoding result is consistent with the preset test data. Specifically, the generated target test code is first used to read the binary code test file containing serialized data. By calling the current decoding function in the current codec, the binary file is decoded to obtain a decoding result. Subsequently, the decoding result is compared and verified item by item with the original data object default_data. If the comparison results are completely consistent, the current codec verification is considered successful. If the comparison reveals inconsistencies, the current codec verification is considered unsuccessful. At this point, the user or developer will be guided to modify the data structure definition file, mapping file, or code template corresponding to the current codec, generating a new, corrected codec. The process will then return to the step of generating the target test code, using the updated code to re-execute the complete read, decode, and comparison verification process. This process will repeat until a verification is successful and the decoding result is completely consistent with the preset test data, thus ensuring that the final output code is a tested and verified, functionally correct target codec.

[0085] As described above, this embodiment first reads and preprocesses the initial data structure definition file and the initial mapping file to obtain the target sorting result and the processed mapping file. Then, based on the preset encoding and decoding requirements, it obtains the corresponding code generation template set by combining zero-copy technology and memory pool allocation strategy. The target encoding and decoding template is selected by parsing the data object type in the topology sorting result. Subsequently, the template engine is used to fill the relevant data into the template to generate the current encoding and decoding code. Simultaneously, test code is generated based on the test objects in the topology sorting result to complete verification. Finally, the target encoding and decoding code is determined based on the verification result. In this way, by completely replacing string keys with compact integer enumeration values, data redundancy is fundamentally eliminated, and the serialized data volume is greatly compressed, thus significantly saving transmission bandwidth and storage space. At the same time, all encoding and decoding codes are automatically generated from a unified, human-readable data structure definition file, and topology sorting and other technologies ensure that even complex nested structures can generate correct, compilable code, completely freeing developers from tedious and error-prone manual coding. Furthermore, unit test code and data matching the business code are automatically generated, realizing definition-as-test, greatly reducing the cost of writing and maintaining test code.

[0086] As can be seen from the previous embodiment, this application discloses a code generation method that can provide efficient and reliable secure MessagePack encoding and decoding for embedded systems. Suppose we are developing a communication system for an IoT device. In this system, sensor nodes need to package and send collected data such as temperature, GPS location, and device status to a central server. To ensure communication efficiency and data consistency, a set of efficient encoding and decoding code is needed to serialize and deserialize this structured data. Next, using the example of generating encoding and decoding code for a communication system, we will address issues such as... Figure 4 The code generation method shown is explained in detail.

[0087] First, the developers write two input files. The initial data structure definition file (sensor.msg) defines all the data objects to be transmitted. For example, the SensorData structure contains a floating-point number (temperature), a nested GPS structure (location), and an integer (timestamp) field, and explicitly defines that SensorData depends on GPS. The initial mapping file (key_map.yaml) defines the mapping between key names and enumeration values, for example, mapping the field name "temp" to the enumeration value 1.

[0088] Next, the code generator reads these two files, analyzes the dependencies between data structures (for example, GPS must be defined before SensorData, which contains it), and performs a topological sort to obtain an ordered, non-cyclically dependent target sort result, such as [GPS, SensorData]. Simultaneously, as... Figure 5 As shown, the system scans all fields in the data structure definition. If a new key is found, such as the new field "humidity", which does not exist in the initial mapping file, a new enumeration value will be automatically assigned to it, and the generated processed mapping file will be updated.

[0089] Then, based on the requirements of high-performance communication, two types of code generation templates are prepared in advance. The preset encoding code generation template set contains code frameworks for converting C structures into compact binary formats such as MessagePack. The preset decoding code generation template set is constructed strictly according to zero-copy technology and a preset memory pool allocation strategy. The sorting results are parsed to identify the specific data types of GPS and SensorData. For complex nested types like location, a temporary intermediate structure (TMapStringDouble) is automatically created to map the flexible MessagePack map to a fixed array of C language structures. Then, based on the determined data types, including the intermediate structure, the most suitable target encoding code generation template and target decoding code generation template are precisely selected from the two template sets respectively.

[0090] Finally, as Figure 5 As shown, the preset template engine uses the target sorting results and the processed mapping file as data to populate the selected target encoding and target decoding templates. This process dynamically generates the final C / C++ source code files, such as sensor_mpack.c and sensor_mpack.h, which are the current encoding and decoding code.

[0091] In addition, to ensure the generated code is error-free, target test code is automatically generated based on the data objects marked as test objects in the sorting results. The target test code uses the generated decoding function to decode a known, correct binary test file and compares the decoding result with the original test data. If they match perfectly, the verification is successful, and this current codec is determined as the final usable target codec and integrated into the firmware and server software of the IoT device. If they do not match, the verification fails, and developers need to check and modify the data structure definitions, mapping files, or codec templates, repeating the entire process until valid code is generated.

[0092] See Figure 6As shown, an embodiment of the present invention discloses a code generation apparatus applied to an encoding / decoding code generator, comprising:

[0093] The file reading module 11 is used to read the initial data structure definition file and the initial mapping file input by the user terminal; the data structure definition file includes the definition of the fields, data types, dependencies and organization methods of each data object, and the initial mapping file is a mapping file between keys and enumerations;

[0094] The result and file acquisition module 12 is used to perform a topological sorting operation on the data objects in the initial data structure definition file based on the dependency relationship to obtain the target sorting result, and to perform a preprocessing operation on the initial mapping file to generate a processed mapping file;

[0095] The template set acquisition module 13 is used to acquire a preset encoding code to generate a template set according to preset encoding requirements, and to acquire a preset decoding code to generate a template set based on preset decoding requirements according to zero-copy technology and preset memory pool allocation strategy; wherein, the template set includes templates corresponding to each preset language;

[0096] Template determination module 14 is used to parse the target sorting result to obtain the data type of the data object in the target sorting result, and determine the target encoding code generation template and the target decoding code generation template from the preset encoding code generation template set and the preset decoding code generation template set according to the data type;

[0097] The code acquisition module 15 is used to fill the target encoding code generation template and the target decoding code generation template with the target sorting result and the processed mapping file using a preset template engine, so as to obtain the current encoding and decoding code;

[0098] The code determination module 16 is used to generate target test code based on the data objects used for testing in the target sorting result, verify the current codec code through the target test code, and determine the target codec code based on the verification result.

[0099] As described above, this application first reads and preprocesses the initial data structure definition file and the initial mapping file to obtain the target sorting result and the processed mapping file. Then, based on the preset encoding and decoding requirements, it obtains the corresponding code generation template set by combining zero-copy technology and memory pool allocation strategy. The target encoding and decoding template is selected by parsing the data object type in the topology sorting result. Subsequently, the template engine is used to fill the template with relevant data to generate the current encoding and decoding code. Simultaneously, test code is generated based on the test objects in the topology sorting result to complete verification. Finally, the target encoding and decoding code is determined based on the verification result. In this way, by completely replacing string keys with compact integer enumeration values, data redundancy is fundamentally eliminated, and the serialized data volume is greatly compressed, thus significantly saving transmission bandwidth and storage space. At the same time, all encoding and decoding codes are automatically generated from a unified, human-readable data structure definition file, and topology sorting and other technologies ensure that even complex nested structures can generate correct, compilable code, completely freeing developers from tedious and error-prone manual coding. Furthermore, unit test code and data matching the business code are automatically generated, realizing definition-as-test, greatly reducing the cost of writing and maintaining test code.

[0100] In some specific implementation methods, the result and file acquisition module 12 may specifically include:

[0101] An information acquisition unit is used to parse the initial data structure definition file to obtain the data structure definition information of each data object defined in the initial data structure definition file.

[0102] A relationship graph construction unit is used to construct a dependency graph between the data objects based on the dependency relationships in the definition information of each data structure.

[0103] A topology sorting unit for performing a topology sorting operation on the dependency graph;

[0104] The first operation result determination unit is used to generate the target sorting result if the obtained operation result indicates that the topological sorting operation was successfully executed.

[0105] The second operation result determination unit is used to stop the preset topology sorting operation and send a preset error message to the user terminal if the operation result indicates that the topology sorting operation has failed.

[0106] The key determination unit is used to traverse the fields of each data object in the initial data structure definition file and read the initial mapping file to determine new keys that exist in the initial data structure definition file but are not defined in the initial mapping file;

[0107] The mapping file acquisition unit is used to assign a corresponding target enumeration value to the new key, and update the initial mapping file based on the new key and the corresponding target enumeration value to obtain the processed mapping file.

[0108] In some specific implementation methods, the template determination module 14 may specifically include:

[0109] A data type determination unit is used to parse the target sorting result and determine the data type of the corresponding data object.

[0110] A structure definition unit is used to define a corresponding intermediate structure based on the data type of the data object; wherein, the intermediate structure is used to map serialized format data types to static structure representations;

[0111] The template selection unit is used to select the corresponding target encoding code generation template and target decoding code generation template from the preset encoding code generation template set and the preset decoding code generation template set according to the intermediate structure.

[0112] In some specific implementation methods, the target test code includes binary code test files and unit test code;

[0113] Accordingly, the code determination module 16 may specifically include:

[0114] A data object selection unit is used to determine whether there is a data object for testing in the target sorting result;

[0115] The test file generation unit is used, if it exists, to serialize the preset test data corresponding to the data object using the current encoding code in the generated current encoding / decoding code, and generate a binary code test file.

[0116] The test code generation unit is used to generate corresponding unit test code based on the decoding and verification requirements of the binary code test file; wherein the unit test code includes at least one test case.

[0117] In some specific implementation methods, the code determination module 16 may specifically include:

[0118] The decoding result acquisition unit is used to read the binary code test file using the target test code and call the current decoding function in the current encoding / decoding code to obtain the corresponding decoding result;

[0119] The result verification unit is used to compare and verify the decoding result with the preset test data;

[0120] The first result determination unit is used to determine that the current codec verification is successful if the obtained verification result indicates that the decoding result is consistent with the preset test data, so that the user terminal can determine the current codec as the target codec.

[0121] The second result determination unit is used to determine that the current codec verification fails if the obtained verification result indicates that the decoding result is inconsistent with the preset test data. This allows the user terminal to modify the files and code templates related to the current codec to obtain a new current codec, and then jump back to the step of generating target test code based on the target sorting result and verifying the current codec using the target test code, until the obtained verification result indicates that the decoding result is consistent with the preset test data.

[0122] In some specific implementation methods, the template set acquisition module 13 may specifically include:

[0123] A memory pool allocation logic generation unit is used to determine the preset memory pool allocation logic of each template in the preset decoding code generation template set based on a preset memory pool allocation strategy; wherein, the preset memory pool allocation logic is to select a preset memory pool for memory management during the decoding process, and the preset memory pool includes an internal static memory pool or an external memory pool.

[0124] The decoding logic determining unit is used to determine the data type decoding logic of each template in the preset decoding code generation template set according to the zero-copy technology; wherein, the data type decoding logic includes decoding logic for string or binary data types using zero-copy technology, array decoding logic, and unknown key decoding logic;

[0125] The template set unit is used to determine a preset decoding code generation template set based on the preset memory pool allocation logic and the data type decoding logic.

[0126] In some specific implementation methods, the decoding logic determining unit may specifically include:

[0127] The key-value pair reading subunit is used to read the key-value pairs of the target decoded code in the target codec based on the memory in the preset memory pool;

[0128] The type determination subunit is used to determine the type of the key in the target decoded code based on the read result;

[0129] The first data decoding subunit is used to call a preset in-situ read function based on zero-copy technology when the key type is string or binary data, so that the pointer directly points to the position corresponding to the string or binary data in the input buffer;

[0130] The second data decoding subunit is used to allocate corresponding memory from the preset memory pool to decode the corresponding field if the key is of type array.

[0131] The third data decoding subunit is used to skip the corresponding field if the key type is an unknown key.

[0132] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0133] Figure 7 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the code generation method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0134] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0135] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0136] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the code generation method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0137] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed code generation method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0138] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0139] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0140] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0141] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0142] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A code generation method, characterized in that, Applications in code generators include: Read the initial data structure definition file and initial mapping file input by the user; the data structure definition file includes the definition of fields, data types, dependencies and organization methods of each data object, and the initial mapping file is a mapping file between keys and enumerations; Based on the aforementioned dependencies, a topological sorting operation is performed on the data objects in the initial data structure definition file to obtain the target sorting result, and a preprocessing operation is performed on the initial mapping file to generate a processed mapping file; A set of preset encoding code generation templates is obtained based on preset encoding requirements, and a set of preset decoding code generation templates is obtained based on preset decoding requirements using zero-copy technology and preset memory pool allocation strategy; wherein, the template set includes templates corresponding to each preset language; The target sorting result is parsed to obtain the data type of the data object in the target sorting result. Based on the data type, the target encoding code generation template and the target decoding code generation template are determined from the preset encoding code generation template set and the preset decoding code generation template set. The target sorting result and the processed mapping file are filled into the target encoding code generation template and the target decoding code generation template using a preset template engine to obtain the current encoding and decoding code; Target test code is generated based on the data objects used for testing in the target sorting result. The current codec is verified using the target test code, and the target codec is determined based on the verification result.

2. The code generation method according to claim 1, characterized in that, The step of performing a topological sorting operation on the data objects in the initial data structure definition file based on the dependency relationship to obtain the target sorting result, and performing a preprocessing operation on the initial mapping file to generate a processed mapping file, includes: Parse the initial data structure definition file to obtain the data structure definition information of each data object defined in the initial data structure definition file; Construct a dependency graph between the data objects based on the dependencies in the definition information of each data structure. Perform a topological sorting operation on the dependency graph; If the result indicates that the topological sorting operation was successfully executed, then the target sorting result is generated. If the operation result indicates that the topology sorting operation has failed, the preset topology sorting operation is aborted and a preset error message is sent to the user terminal. Iterate through the fields of each data object in the initial data structure definition file and read the initial mapping file to determine new keys that exist in the initial data structure definition file but are not defined in the initial mapping file; Assign a corresponding target enumeration value to the new key, and update the initial mapping file based on the new key and the corresponding target enumeration value to obtain the processed mapping file.

3. The code generation method according to claim 1, characterized in that, The step of parsing the target sorting result to obtain the data type of the data object in the target sorting result, and determining the target encoding code generation template and the target decoding code generation template from the preset encoding code generation template set and the preset decoding code generation template set based on the data type, includes: Parse the target sorting result to determine the data type of the corresponding data object; An intermediate structure is defined based on the data type of the data object; wherein, the intermediate structure is used to map the serialized format data type to a static structure representation; Based on the intermediate structure, the corresponding target encoding code generation template and target decoding code generation template are selected from the preset encoding code generation template set and the preset decoding code generation template set.

4. The code generation method according to any one of claims 1 to 3, characterized in that, The target test code includes binary code test files and unit test code; Accordingly, generating target test code based on the data objects used for testing in the target sorting result includes: Determine whether the target sorting result contains a data object for testing; If it exists, the current encoding code in the generated current encoding / decoding code is used to serialize the preset test data corresponding to the data object to generate a binary code test file; Based on the decoding and verification requirements of the binary code test file, corresponding unit test code is generated; wherein the unit test code includes at least one test case.

5. The code generation method according to claim 4, characterized in that, The step of verifying the current codec using the target test code and determining the target codec based on the verification result includes: The target test code is used to read the binary code test file, and the current decoding function in the current encoding / decoding code is called to obtain the corresponding decoding result; The decoding result is compared and verified with the preset test data; If the verification result indicates that the decoding result is consistent with the preset test data, then the current codec is determined to be successfully verified, so that the user terminal can identify the current codec as the target codec. If the verification result indicates that the decoding result is inconsistent with the preset test data, the current codec verification is determined to be unsuccessful. The user terminal then modifies the files and code templates related to the current codec to obtain a new current codec, and jumps back to the step of generating target test code based on the target sorting result, and verifying the current codec through the target test code, until the verification result indicates that the decoding result is consistent with the preset test data.

6. The code generation method according to claim 1, characterized in that, The step of obtaining a set of preset decoding code generation templates based on preset decoding requirements using zero-copy technology and a preset memory pool allocation strategy includes: The preset memory pool allocation logic for each template in the preset decoding code generation template set is determined based on the preset memory pool allocation strategy; wherein, the preset memory pool allocation logic is to select a preset memory pool for memory management during the decoding process, and the preset memory pool includes an internal static memory pool or an external memory pool. The data type decoding logic of each template in the preset decoding code generation template set is determined according to the zero-copy technology; wherein, the data type decoding logic includes decoding logic for string or binary data types, array decoding logic, and unknown key decoding logic using zero-copy technology; Based on the preset memory pool allocation logic and the data type decoding logic, a preset decoding code generation template set is determined.

7. The code generation method according to claim 6, characterized in that, The data type decoding logic for determining each template in the preset decoding code generation template set based on the zero-copy technology includes: Based on the memory in the preset memory pool, read the key-value pairs of the target codec from the target codec; Determine the type of the key in the target decoding code based on the reading results; If the key is of type string or binary data, then a preset in-situ read function is called based on zero-copy technology, and the pointer is directly pointed to the position corresponding to the string or binary data in the input buffer; If the key is of type array, then the corresponding memory is allocated from the preset memory pool to decode the corresponding field; If the key is of unknown type, then the corresponding field is skipped.

8. A code generation device, characterized in that, Applications in code generators include: The file reading module is used to read the initial data structure definition file and the initial mapping file input by the user. The data structure definition file includes the definition of the fields, data types, dependencies and organization methods of each data object. The initial mapping file is a mapping file between keys and enumerations. The result and file acquisition module is used to perform a topological sorting operation on the data objects in the initial data structure definition file based on the dependency relationship to obtain the target sorting result, and to perform a preprocessing operation on the initial mapping file to generate a processed mapping file; The template set acquisition module is used to acquire a preset encoding code to generate a template set based on preset encoding requirements, and to acquire a preset decoding code to generate a template set based on preset decoding requirements using zero-copy technology and a preset memory pool allocation strategy; wherein, the template set includes templates corresponding to each preset language; The template determination module is used to parse the target sorting result to obtain the data type of the data object in the target sorting result, and determine the target encoding code generation template and the target decoding code generation template from the preset encoding code generation template set and the preset decoding code generation template set according to the data type; The code acquisition module is used to fill the target encoding code generation template and the target decoding code generation template with the target sorting result and the processed mapping file using a preset template engine, so as to obtain the current encoding and decoding code; The code determination module is used to generate target test code based on the data objects used for testing in the target sorting result, verify the current codec code using the target test code, and determine the target codec code based on the verification result.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the code generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the code generation method as described in any one of claims 1 to 7.

Images