Data serialization and deserialization methods, electronic equipment and storage medium

Abstract

The invention provides a data serialization method, a data deserialization method, electronic equipment and a storage medium, memory pool allocation is performed according to node types in the serialization process of an abstract syntax tree, a serialization storage table is generated on the basis of the memory pool allocation, and only each node in the abstract syntax tree is recorded in the serialization storage table. According to the method, the pointing relation of the pointer is recorded only in the node recording process, so that decoupling between the nodes can be better achieved in the serialization process or the subsequent deserialization process, other nodes are difficult to influence when one node is recorded or recovered, the corresponding cost can be further reduced, and the reliability of the system is improved. And meanwhile, the recorded content is simplified, so that the transmission efficiency is improved, and the overall efficiency and the user experience are finally improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data serialization and deserialization method, an electronic device, and a storage medium. Background Art

[0002] The Abstract Syntax Tree (AST) is generated by compiling the source code of the Device Under Test (DUT) through a digital simulator, and is mainly used to generate executable files later. At the same time, the debugging tool also needs to apply the AST when debugging the object under test. Since the digital simulator and the debugging tool are two sets of tools, the transmission of the AST from the digital simulator to the debugging tool involves the transmission and reconstruction of the AST structure, that is, serialization and deserialization.

[0003] In the context of the increasing size of the objects to be tested, the transmission cost and time consumption of the abstract syntax tree in related technologies are becoming increasingly high, which has seriously affected the overall efficiency and user experience. Summary of the invention

[0004] In view of this, the present application proposes a data serialization and deserialization method, an electronic device and a storage medium to solve or partially solve the above problems.

[0005] Based on the above objectives, in a first aspect, the present application provides a data serialization method, comprising:

[0006] Acquire an abstract syntax tree; wherein the abstract syntax tree includes a plurality of nodes and pointers for indicating pointing relationships between nodes;

[0007] Determine at least one node type of the abstract syntax tree, and allocate memory pools of different node types according to the at least one node type;

[0008] Determine the number of nodes within any node type, and generate a serialized storage table based on the number of nodes and attribute information of the nodes within any node type; wherein the serialized storage table at least includes the memory pool allocation position of any node and the pointing relationship of the pointer starting from any node.

[0009] In a second aspect of the present application, the present application also provides a data deserialization method, comprising:

[0010] Obtain a serialized storage table; wherein the serialized storage table is formed by the method described in the first aspect;

[0011] Determine at least one node in the serialized storage table, create the at least one node from the memory, and record the current address of the at least one node;

[0012] Determine the pointing relationship of any node according to the serialized storage table, and update the pointer starting from any node according to the pointing relationship and the current address;

[0013] Build an abstract syntax tree based on the updated nodes and pointers.

[0014] In a third aspect of the present application, an electronic device is provided, comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the program includes instructions for executing the method described in the first aspect.

[0015] In a fourth aspect of the present application, a non-volatile computer-readable storage medium containing a computer program is provided. When the computer program is executed by one or more processors, the processors execute the method described in the first aspect.

[0016] As can be seen from the above, the present application provides a data serialization and deserialization method, an electronic device and a storage medium. The present application allocates a memory pool according to the node type during the serialization process of the abstract syntax tree, and generates a serialization storage table based on this, and only records each node in the abstract syntax tree in the serialization storage table, and only records the pointing relationship of the pointer in the process of recording the node, so that whether in the serialization process or the subsequent deserialization process, the decoupling between nodes can be better achieved, and recording or restoring a node is unlikely to affect other nodes, thereby reducing the corresponding cost. At the same time, due to the simplification of the recorded content, it is also beneficial to the transmission efficiency, and ultimately improves the overall efficiency and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of a host according to an embodiment of the present application is shown.

[0019] Figure 2 A schematic diagram of a digital simulator and a debugging tool according to an embodiment of the present application is shown.

[0020] Figure 3 A schematic flow chart of an exemplary data serialization method according to an embodiment of the present application is shown.

[0021] Figure 4 A schematic flow chart of an exemplary data deserialization method according to an embodiment of the present application is shown.

[0022] Figure 5 A schematic diagram showing the effect of comparing the data serialization and deserialization method according to an embodiment of the present application with related technologies. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of this specification more clear, this specification is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0024] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements, objects or method steps appearing before the word cover the elements, objects or method steps listed after the word and their equivalents, without excluding other elements, objects or method steps. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0025] As described in the background technology section, an abstract syntax tree can be generated by a digital simulator and can be modified and adjusted. At the same time, the debugging tool needs to be applied to the abstract syntax tree during debugging, which involves the problem of abstract syntax tree transmission. The abstract syntax tree is usually stored and restored with the help of serialization and deserialization processes. Digital simulators and debugging tools generally require corresponding electronic equipment for support.

[0026] Figure 1 The schematic diagram of the structure of a host 100 provided in an embodiment of the present application is shown. The host 100 may include: a processor 102, a memory 104, a network interface 106, a peripheral interface 108 and a bus 110. The processor 102, the memory 104, the network interface 106 and the peripheral interface 108 are connected to each other through the bus 110 in the device.

[0027] Processor 102 may be a central processing unit (CPU), an image processor, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or one or more integrated circuits. Processor 102 may be used to perform functions related to the technology described in this application. In some embodiments, processor 102 may also include multiple processors integrated into a single logical component. Figure 1 As shown, the processor 102 may include a plurality of processors 102a, 102b, and 102c.

[0028] The memory 104 may be configured to store data (eg, instruction sets, computer codes, intermediate data, etc.). Figure 1 As shown, the stored data may include program instructions (e.g., program instructions for implementing the technical solution of the present application) and data to be processed. The processor 102 may also access the stored program instructions and data, and execute the program instructions to operate the data to be processed. The memory 104 may include a volatile storage device or a non-volatile storage device. In some embodiments, the memory 104 may include a random access memory (RAM), a read-only memory (ROM), an optical disk, a magnetic disk, a hard disk, a solid-state drive (SSD), a flash memory, a memory stick, etc.

[0029] The network interface 106 can be configured to provide communication with other external devices to the host 100 via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC) etc.), a cellular network, the Internet or a combination thereof. It is understood that the type of network is not limited to the above specific examples. In some embodiments, the network interface 106 can include any number of network interface controllers (NICs), radio frequency modules, transceivers, modems, routers, gateways, adapters, cellular network chips, etc., in any combination.

[0030] The peripheral interface 108 can be configured to connect the host 100 to one or more peripheral devices to achieve information input and output. For example, the peripheral devices can include input devices such as a keyboard, a mouse, a touch pad, a touch screen, a microphone, and various sensors, and output devices such as a display, a speaker, a vibrator, and an indicator light.

[0031] The bus 110 may be configured to transmit information between various components of the host 100 (e.g., the processor 102, the memory 104, the network interface 106, and the peripheral interface 108), such as an internal bus (e.g., a processor-memory bus), an external bus (USB port, PCI-E bus), etc.

[0032] It should be noted that, although the above device only shows the processor 102, the memory 104, the network interface 106, the peripheral interface 108 and the bus 110, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiment of the present application, and does not necessarily include all the components shown in the figure.

[0033] Figure 2 FIG. 2 is a schematic diagram of a digital simulator 202 and a debugging tool 200 according to an embodiment of the present application. The digital simulator 202 and the debugging tool 200 may be computer programs running on the host 100 .

[0034] In the field of chip design, a design can usually be simulated using a digital simulator 202. The digital simulator 202 can be, for example, the GalaxSim simulation tool produced by Xinhuazhang Technology Co., Ltd. Figure 2 The exemplary digital simulator 202 shown may include a compiler 120 and a simulator 220. The compiler 120 may compile the design into a target code 204 (e.g., DUT source code), and the simulator 220 may perform simulation according to the target code 204, and output a simulation result 206. In this embodiment, the simulation result 206 may be an abstract syntax tree (AST). In one example, when the digital simulator 202 compiles the DUT source code, it may perform lexical analysis and syntactic analysis, and convert the source code into an abstract syntax tree AST. This process helps to identify errors in the code, optimize it, and generate machine executable code. It can be seen that since the transmission process of the simulation result 206 is involved, the abstract syntax tree AST as the simulation result 206 needs to undergo a serialization process to make it data that can be fixed to the corresponding hard disk storage space.

[0035] Afterwards, the debugging tool 200 can read the simulation result 206. The debugging tool 200 can be, for example, the Fusion Debug debugging tool produced by Xinhuazhang Technology Co., Ltd. For example, the debugging tool 200 can read the abstract syntax tree AST in serialized form, and deserialize it for recovery, so that it can be smoothly applied to the debugging tool 200 to debug the DUT. Afterwards, the debugging tool 200 can also generate various graphical interfaces (for example: debugging windows) to facilitate the user's debugging work. The user can issue a debugging command 208 to the debugging tool 200, and the debugging tool 200 applies the debugging command 208 to the digital simulator 202 for corresponding execution. The debugging tool 200 can also read a log file. The log file may include various information of the simulation process, including information of the simulation error, the line number of the error, the time when the simulation error occurred, etc.

[0036] It is understandable that, in addition to being connected to the digital simulator 202 , the debugging tool 200 may also be connected to a hardware simulator (emulator).

[0037] Before the logic system design is put into actual operation, the compiled logic system design needs to be run and debugged using the aforementioned debugging tool 200. The debugging tool 200 can receive user instructions 210 from the user and make corresponding debugging actions. In some embodiments, the user can issue user instructions 210 by clicking the mouse in the graphical interface; in some embodiments, the user can enter a command line in the command line interface to issue a user instruction 210. If an abnormality is found in the logic system design during the debugging process, it is necessary to further diagnose the abnormality found, find out the cause and the specific source location of the abnormality, and make corrections to ensure the robustness of the logic system design.

[0038] In the field of integrated circuits, simulation can refer to compiling a logical system design and then running it with a simulation tool to simulate and test various functions of the design. It is used for the design of application-specific integrated circuits (ASIC) or system-on-chips (SOC). Therefore, the design tested or verified in the simulation can also be called the design under test (DUT).

[0039] As can be seen from the above, the generation and reading of the simulation result 206 involves the serialization process and deserialization process of the abstract syntax tree AST. First of all, the abstract syntax tree AST is essentially a tree structure, mainly composed of nodes and edges (also called pointers in this scenario), and is mainly used to represent the source code structure of a programming language, such as the aforementioned DUT source code. It converts the structures in the source code (such as variable declarations, control flow statements, function definitions, etc.) into a tree form, where each node represents a grammatical structure in the source code.

[0040] Afterwards, in some embodiments, for serialization AST, after determining a node, the child node of the node will be determined according to the pointer (or edge) of the node recorded in AST, and then the data structure of the child node will be embedded in the data structure of the parent node, and then the node connection process will be performed, and finally serialized data will be formed. By analyzing the above embodiment, it can be found that in the serialization process of a node, other nodes will inevitably be involved, resulting in the difficulty of decoupling between nodes when serialization is performed, and it is necessary to keep other node data unchanged. However, in the daily development of chip design, the source code of the design DUT to be tested will be continuously modified and iterated, and then the digital simulator will also be continuously adjusted in the process of compiling and generating an abstract syntax tree, which has caused the node data, especially the edge data, to be difficult to keep stable for a long time. In combination with the above situation, it can be seen that in the case of conflict between the two, it is inevitable that the transmission cost, time-consuming, etc. of the abstract syntax tree serialization and deserialization process are increasingly high.

[0041] In view of this, another embodiment of the present application provides a data serialization and deserialization method, an electronic device and a storage medium. By allocating a memory pool according to the node type during the serialization of the abstract syntax tree, and generating a serialization storage table based on this, and only recording each node in the abstract syntax tree in the serialization storage table, and only recording the pointing relationship for the pointer, whether in the serialization process or the subsequent deserialization process, the decoupling between nodes can be better achieved, and recording or restoring a node is unlikely to affect other nodes, thereby reducing the corresponding cost. At the same time, due to the simplification of the recorded content, it is also beneficial to the transmission efficiency, and ultimately improves the overall efficiency and user experience.

[0042] Specifically, Figure 3 The flowchart of the exemplary data serialization method provided in the embodiment of the present application is shown. Figure 1 The method may be implemented by the host 100 and may further include the following steps.

[0043] Step 302: Obtain an abstract syntax tree; wherein the abstract syntax tree includes a plurality of nodes and pointers for indicating pointing relationships between nodes.

[0044] In this step, as described in the above-mentioned embodiment, the abstract syntax tree AST is essentially a tree structure, mainly composed of nodes and edges (also called pointers in this scenario).

[0045] Step 304: determine at least one node type of the abstract syntax tree, and allocate memory pools of different node types according to the at least one node type.

[0046] In this step, each node of the abstract syntax tree AST belongs to a different node type, and the specific node types include function definition, expression, block, return statement, conditional statement, loop statement, etc. Here, the nodes of the abstract syntax tree can be divided into N types, which can be recorded as N types of nodes: kind1, kind2, ..., kind N Each node type contains at least one specific node. Then, for different node types, a corresponding memory pool can be allocated, namely pool1, pool2, ..., pool N , where pool1 can correspond to kind1, pool2 can correspond to kind2, and so on.

[0047] In some more specific scenarios, the data structure of each node recorded in each memory pool may mainly include Node{Flags;Containers <pointer>; pointer;}. Node means a node, Flags means some attribute information of the node itself, and pointer means the pointer of the node. If a node has one pointer, pointer is used. If a node has multiple pointers, the pointer can be recorded in the form of containers, that is, using Containers. <pointer>.

[0048] Step 306, determine the number of nodes within any node type, and generate a serialized storage table based on the number of nodes and the attribute information of the nodes within any node type; wherein the serialized storage table at least includes the memory pool allocation position of any node, and the pointing relationship of the pointer starting from any node.

[0049] In this embodiment, one of the node types is used as an example for explanation, and other node types can repeat the corresponding process to complete the serialization of the entire abstract syntax tree AST. For any node type, since it generally contains multiple nodes, the number of nodes it specifically contains can be counted. For example, in some embodiments, after counting the number of nodes, each node can be numbered accordingly, so that for a node, it can be located according to the node type it belongs to and the number in the node type, which is convenient for searching.

[0050] After that, on this basis, a serialized storage table can be generated in combination with the attribute information of the node. Here, the serialized storage table mainly needs to record the node's allocation position in the memory pool, for example, by counting the specific memory pool in which the node is located and its specific number, the node's allocation position in the memory pool can be determined. After that, for the attributes of the node itself, since it will not affect other nodes, it can be directly recorded. After that, for the pointers starting from the node, only the pointing relationship of these pointers can be recorded, without paying attention to the content of the specific object pointed to by the pointer. For example, node A has a pointer that points to node B. Then, when recording node A, only node A is recorded as having a pointer to node B, and the specific content of node B is not paid attention to. It can be seen that this recording method is more like recording the node's own attributes, and does not require additional resources to understand the pointed object, which also facilitates the recording of data and the decoupling between nodes. In this way, in the serialization process, since the serialization of each node is only for its own node, it does not affect other nodes, thereby achieving complete decoupling between nodes, which can reduce the corresponding cost. At the same time, due to the simplification of the recorded content, it is also beneficial to the transmission efficiency.

[0051] In some embodiments, in order to more clearly determine the position of the node in the memory pool, the nodes can be numbered based on the number of nodes of a node, and a serialized storage table can be generated according to the numbering sequence. That is, in some embodiments, after determining the number of nodes in any node type, the method further includes: according to the number of nodes, numbering the nodes in any node type, so as to generate a serialized storage table according to the numbering sequence.

[0052] In other embodiments, according to the foregoing, a node itself will have certain attribute data, which is the basic attribute here. Then, according to the abstract syntax tree, the pointing relationship of the pointer starting from the node can be determined, and the combination of the basic attribute and the pointing relationship is the attribute information of the node. Similarly, by determining the node type and the corresponding number of a node, the memory pool allocation position of the node can be accurately determined. Finally, when the serialized storage table is generated, the attribute information and the memory pool allocation position can be recorded as the value of a node in the table. That is, in some embodiments, the generation of the serialized storage table includes: determining the basic attributes and pointing relationship of any node according to the abstract syntax tree, thereby generating the attribute information; determining the node type and the corresponding number corresponding to any node, generating the memory pool allocation position; recording the attribute information and the memory pool allocation position as the address of any node in the serialized storage table.

[0053] In a more specific scenario, for the serialization process, you can traverse pool1, pool2, ..., pool N , and record the number of nodes num in each pool i After that, you can write the value of any node in memory directly to disk, such as the file node i , record the current node address, node type and node index in the memory pool to node index Among them, Node_index: [{ptr 1, ID XCMODULE, pool index 1}, {ptr 2, ID XCINST, pool index 5}, ...]. Further, if there is a container in the node, the container element number and element pointer are written to the disk. Finally, the node index Persist to the index file to complete the serialization process.

[0054] In some embodiments, the serialization process can be accelerated by multithreading. More importantly, according to the aforementioned process, during the serialization process, since each node does not affect each other, decoupling between each other is achieved, and each node will only be processed once during the serialization process, and then when performing multithreading, it can be completely lock-free, and lock-free is bound to be more efficient than a locked process. Of course, for the specific process of multithreading implementation, according to the aforementioned, since each node does not affect each other, it can be allocated according to the node type, for example, the first thread processes node types 1 to 5, the second thread processes node types 6 to 10, and so on. Of course, specific nodes can also be directly allocated in these threads, for example, the first thread processes 100 nodes, the second thread processes another 100 nodes, and so on. That is, in some embodiments, the determining of the number of nodes within any node type and generating a serialized storage table based on the number of nodes and attribute information of the nodes within any node type further include: determining the number of threads in response to supporting multi-threaded operations; allocating tasks to at least one node type or allocating tasks to nodes in any node type based on the number of threads, thereby performing multi-threaded lock-free serialization.

[0055] after, Figure 4 The flowchart of the exemplary data deserialization method provided in the embodiment of the present application is shown. Figure 1 The method may be implemented by the host 100 and may further include the following steps.

[0056] Step 402, obtaining a serialized storage table; wherein the serialized storage table is formed by the data serialization method described in any of the aforementioned embodiments.

[0057] In this step, the serialized storage table is formed by the data serialization method in the aforementioned embodiment, and this embodiment mainly performs a deserialization process based on the serialized storage table.

[0058] Step 404: determine at least one node in the serialized storage table, create the at least one node from the memory, and record the current address of the at least one node.

[0059] In this step, a node in the serialized storage table is first determined. Since deserialization is the process of reading the serialized storage table into the memory, each node will be assigned a new memory address during deserialization. This memory address is generally different from the original memory address of the node when the serialized storage table was generated, and thus needs to be recorded.

[0060] In some embodiments, in order to facilitate subsequent data replacement and the recording of current data, before updating the address of the node, a first mapping and a second mapping can be created according to the serialized storage table. The first mapping can be in the form of a hash table, which is used to record node attributes and the pointing relationship of pointers starting with the node, specifically the address of the node's internal pointer and the node type, etc.; the second mapping can be used to clearly distinguish each node and record node data, for example, the primary key of the mapping represents different node types, the secondary key represents the number, and then each node can find a unique corresponding unit according to the node type and number it belongs to, which is used to record the address of the node in the current memory. Then, for each node, its corresponding new memory address can be updated to the second mapping. That is, in some embodiments, before determining at least one node in the serialized storage table, the method also includes: generating a first mapping for recording node attributes and the pointing relationship of pointers starting with the node according to the serialized storage table; generating a second mapping for recording each node according to the serialized storage table; wherein the primary key of the second mapping represents the node type to which any node belongs, and the secondary key represents the number of any node in the corresponding node type. In some embodiments, recording the current address of at least one node includes: determining the current address of any node in the serialized storage table in the memory after deserialization, and recording the current address at the corresponding position of any node in the second mapping.

[0061] In a more specific scenario, during the deserialization process, you can first index Restore from disk to memory. First create a hash table: Ptr->(XC CLASS ID, pool index), that is, create the first mapping first. Here we take the hash table as an example. Then, create a two-dimensional matrix N*pool num, new.nade.Row:XC CLASS ID, Column:pool index, value:new address of the node, that is, create the second mapping. The rows represent different node types and the columns represent different numbers. Then, from node i The node is restored to the memory. At this time, since a new memory address is allocated, the address storage of the node can be updated in the two-dimensional matrix.

[0062] Step 406: determine the pointing relationship of any node according to the serialized storage table, and update the pointer starting from any node according to the pointing relationship and the current address.

[0063] In this step, according to step 404, each node will obtain a new memory address during the deserialization process, so each node will update the address. As for the pointer, the pointer points to the address of the serialized node, which has become invalid during the deserialization process. Therefore, the pointer needs to be updated. Since the serialization storage table stores the pointing relationship of the pointer, for a pointer, the node type pointed to by the internal pointer of any node, the location information in the memory pool, etc. can be obtained to determine the node pointed to, and then the new address of the node pointed to can be found to complete the update of the pointer.

[0064] In some embodiments, since the first mapping and the second mapping are created, the type of node pointed to by a node pointer can be directly queried through the first mapping, and then the current address of the corresponding node can be found using the second mapping to complete the update of the pointer. That is, in some embodiments, the updating of the pointer starting with any node includes: determining the attribute information of the target node pointed to by the pointing relationship of any node according to the first mapping; determining the updated address of the target node in the second mapping according to the attribute information, and updating the pointer starting with any node according to the updated address of the target node. The target node is the node pointed to by the pointer.

[0065] In a more specific scenario, after completing the address update of each node itself, further update the pointers of the elements in the container of all nodes or the pointers in the nodes (that is, update all kinds of pointers). The update method can be to query the node index table or hash table, know the type of the node, and the corresponding mem pool The index in the second matrix new can be node Find the address of the new node and replace the old address with it.

[0066] Step 408: Create an abstract syntax tree based on the updated nodes and pointers.

[0067] In this step, based on the aforementioned updated information, the deserialization of the abstract syntax tree can be finally completed.

[0068] In some embodiments, for the deserialization process, similar to serialization, it can also be processed with the help of multi-threading. Similarly, due to the independent relationship between nodes, it can achieve process lock-free. Similarly, for multi-threaded task allocation, since they do not affect each other, they can also be directly allocated according to nodes or node types. That is, in some embodiments, after obtaining the serialized storage table, the method also includes: determining the number of threads in response to supporting multi-threaded operations; and assigning tasks to at least one node according to the number of threads, thereby performing multi-threaded lock-free deserialization.

[0069] Finally, if Figure 5 As shown, it is a schematic diagram comparing the effects of serialization and deserialization of the abstract syntax tree by the method of this embodiment and other methods. Wherein bm represents different abstract syntax trees for testing, V1 represents the results of serialization and deserialization in the manner of the relevant technology, V2_1 represents the results of serialization and deserialization by a 4-thread method in the manner of the embodiment of the present application, and RBS represents the results of serialization and deserialization by a 4-thread locked method of the relevant technology. Afterwards, save represents serialization, V1_save is the time of serialization in the V1 method, in seconds, V2_1_save and RBS_save are the same, save_V1 / V2_1 represents the ratio of V1_save to V2_1_save, and RBS_V1 / V2_1 is the same. Load represents deserialization, similar to save, V1_load is the time of deserialization in the V1 method, in seconds, and so on. DB indicates the size of the generated serialized file in megabytes (M). Similar to save or load, V1_DB is the size of the serialized file generated by the V1 method, and so on. Finally, AVG is the average value. It can be seen that compared with V1, the serialization efficiency of V2_1 has increased by nearly 7 times, and the deserialization efficiency has increased by more than 2 times; compared with RBS, the serialization efficiency of V2_1 has increased by nearly 8 times, and the deserialization efficiency has increased by more than 2 times.

[0070] From the above, it can be seen that the embodiment of the present application provides a data serialization and deserialization method, which allocates a memory pool according to the node type during the serialization of the abstract syntax tree, and generates a serialization storage table based on this. In the serialization storage table, only each node in the abstract syntax tree is recorded, and for the pointer, only the pointing relationship is recorded in the process of recording the node. In this way, whether in the serialization process or the subsequent deserialization process, the nodes can be better decoupled, and recording or restoring a node is unlikely to affect other nodes, thereby reducing the corresponding cost. At the same time, due to the simplification of the recorded content, it is also beneficial to the transmission efficiency, and ultimately improves the overall efficiency and user experience.

[0071] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of the embodiment of the present application can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0072] It should be noted that the above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0073] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device. The electronic device may be Figure 1 The host 100. The electronic device may include a memory for storing a set of instructions; and at least one processor configured to execute the set of instructions to enable the data serialization method or the data deserialization method.

[0074] The electronic device of the above-mentioned embodiment is used to implement the corresponding data serialization method or data deserialization method in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0075] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-volatile computer-readable storage medium containing a computer program, wherein the non-volatile computer-readable storage medium containing a computer program stores computer instructions, and the computer instructions are used to enable the computer to execute the data serialization method or the data deserialization method described in any of the above embodiments.

[0076] The computer-readable storage media of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0077] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the data serialization method or data deserialization method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0078] Based on the same inventive concept, corresponding to the data serialization method or data deserialization method described in any of the above embodiments, the present application also provides a computer program product, which includes a computer program. In some embodiments, the computer program is executable by one or more processors so that the processor executes the data serialization method or data deserialization method. Corresponding to the execution subject corresponding to each step in each embodiment of the data serialization method or data deserialization method, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0079] The computer program product of the above embodiment is used to enable the processor to execute the data serialization method or data deserialization method described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0080] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0081] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0082] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0083] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.< / pointer> < / pointer>

Claims

1. A data serialization method, comprising: Acquire an abstract syntax tree; wherein the abstract syntax tree includes a plurality of nodes and pointers for indicating pointing relationships between nodes; Determine at least one node type of the abstract syntax tree, and allocate memory pools of different node types according to the at least one node type; Determine the number of nodes within any node type, and generate a serialized storage table based on the number of nodes and attribute information of the nodes within any node type; wherein the serialized storage table at least includes the memory pool allocation position of any node and the pointing relationship of the pointer starting from any node.

2. The method according to claim 1, wherein: After determining the number of nodes in any node type, the method further includes: The nodes in any node type are numbered according to the number of nodes, so as to generate a serialized storage table according to the numbering sequence.

3. The method according to claim 2, wherein: The generating of the serialized storage table comprises: Determine the basic attributes and pointing relationships of any node according to the abstract syntax tree, so as to generate the attribute information; Determine the node type and the corresponding number corresponding to any node, and generate the memory pool allocation position; The attribute information and the memory pool allocation position are recorded as the address of any node in the serialized storage table.

4. The method according to claim 1, wherein: The determining the number of nodes in any node type and generating a serialized storage table according to the number of nodes and attribute information of the nodes in any node type further includes: Responsive to supporting multithreaded operation, determining a number of threads; According to the number of threads, tasks are allocated to the at least one node type, or tasks are allocated to nodes in any node type, so as to perform multi-threaded lock-free serialization.

5. A data deserialization method, comprising: Obtain a serialized storage table; wherein the serialized storage table is formed by the method according to any one of claims 1 to 4; Determine at least one node in the serialized storage table, create the at least one node from the memory, and record the current address of the at least one node; Determine the pointing relationship of any node according to the serialized storage table, and update the pointer starting from any node according to the pointing relationship and the current address; Build an abstract syntax tree based on the updated nodes and pointers.

6. The method according to claim 5, wherein: Before determining at least one node in the serialized storage table, the method further includes: Generate a first mapping for recording node attributes and pointing relationships of pointers starting with the node according to the serialized storage table; A second mapping for recording each node is generated according to the serialized storage table; wherein the primary key of the second mapping represents the node type to which any node belongs, and the secondary key represents the number of any node in the corresponding node type.

7. The method according to claim 6, wherein: The recording of the current address of the at least one node comprises: Determine the current address of any node in the serialized storage table in the memory after deserialization, and record the current address at a corresponding position of any node in the second mapping.

8. The method according to claim 7, wherein: The updating of the pointer starting from any node includes: Determining, according to the first mapping, attribute information of a target node pointed to by the pointing relationship of any one of the nodes; An updated address of the target node is determined in the second mapping according to the attribute information, and a pointer starting from any node is updated according to the updated address of the target node.

9. The method according to claim 5, wherein: After obtaining the serialized storage table, the method further includes: Responsive to supporting multithreaded operation, determining a number of threads; According to the number of threads, tasks are allocated to the at least one node, so as to perform multi-threaded lock-free deserialization.

10. An electronic device comprising one or more processors, a memory; and one or more programs; wherein: The one or more programs are stored in the memory and executed by the one or more processors, the programs including instructions for executing the method according to any one of claims 1 to 9.

11. A non-volatile computer-readable storage medium containing a computer program, which, when executed by one or more processors, causes the processors to perform the method of any one of claims 1 to 9.

Images