A high-performance data model access method and device based on shared memory pool

Abstract

The present invention relates to a high-performance data model access method and device based on a shared memory pool. The method primarily includes: applying for shared memory and mapping the shared memory to a fixed virtual address of a process; traversing XML to create a deduplicated symbol table and placing it in the shared memory; parsing the XML and converting it into a tree structure based on the shared memory pool to form an XML spanning tree; providing a data access interface and defining input parameters so that the input parameters can be used to search for corresponding memory nodes in the XML spanning tree, thereby reading and writing their contents. The present invention can be used to solve the problem of fast access to XML configuration data by multiple processes in a terminal device, reducing memory usage and improving access efficiency. It also addresses the use of pointers in shared memory and the dynamic allocation and release of space.

Description

Technical Field

[0001] The present invention relates to the field of data model access technology, and in particular to a high-performance data model access method and device based on a shared memory pool. Background Art

[0002] Existing data model access technology is not convenient for using pointers in shared memory, because the address space of each process is different, and the shared memory address seen by each process is different. Therefore, when process A stores an address pointing to a certain area of shared memory, it is invalid for process B to access this address. Therefore, it is difficult to use dynamic data structures in shared memory. Generally, when using shared memory to transmit data, the data format and size are relatively fixed and single. The sender writes the data to the shared memory and then notifies the receiver to take it out of the shared memory. There is no modification of the data in the shared memory in the middle, nor is it related to the change of the data structure, so there is no allocation and release of shared memory space.

[0003] To solve this problem, existing technologies use a solution based on a shared file directory tree to implement data model access. The implementation method and existing problems are summarized as follows:

[0004] The first step is initialization. XML (Extensible Markup Language) is converted into an in-memory file directory tree according to its structure. A corresponding directory is created at each level. The attributes and values of the leaf nodes are saved in the files in the corresponding directory. All these directories and files are created in the in-memory file system to speed up access.

[0005] Then there's the implementation of the data access interface. XML node search relies on finding a specific file using the file path, which involves system calls. File locks are then needed to achieve mutual exclusion and synchronization, which again involves system calls. Opening, reading, and writing files also require system calls, and all of these operations are time-consuming. When there are a large number of read and write operations, this means a large number of file operations, which consumes system resources and slows down the system, while also reducing interface access speed. Furthermore, because multiple parameter nodes for an XML object are stored in the same file, even if only one parameter node is accessed at a time, the entire file must be read, and then the content to be accessed must be searched for. This consumes memory and is inefficient. Other methods for multi-process access to XML data models may include directly reading and writing XML files, converting XML files into relational databases, etc. These methods also involve file operations and are less efficient.

[0006] In view of this, how to overcome the defects of the existing technology and solve the problem of terminal devices quickly accessing XML configuration data through multiple processes is a difficult problem to be solved in this technical field. Summary of the Invention

[0007] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides a high-performance data model access method and device based on a shared memory pool, which is used to solve the problem of rapid access of XML configuration data by multiple processes of terminal devices, reduce memory usage and improve access efficiency, and at the same time solve the problem of pointer usage in shared memory and the problem of dynamic space allocation and release.

[0008] The embodiment of the present invention adopts the following technical solutions:

[0009] In a first aspect, the present invention provides a high-performance data model access method based on a shared memory pool, comprising:

[0010] Apply for shared memory and map the shared memory to the fixed virtual address of the process;

[0011] Traverse the XML, create a deduplicated symbol table and place it in shared memory;

[0012] Parse XML and convert it into a tree structure based on a shared memory pool to form an XML spanning tree;

[0013] Provides a data access interface and defines input parameters to search for the corresponding memory node in the XML generation tree through the input parameters, thereby reading and writing its content.

[0014] Furthermore, when applying for shared memory, the shared memory is divided into three areas, wherein:

[0015] The first area is a shared memory context information storage area, which is used to store various basic information, including: one or more of the shared memory pool start address, XML spanning tree root node address, and symbol table address size;

[0016] The second area is a static symbol table storage area, which is used to store the symbol table after deduplication;

[0017] The third area is the shared memory pool manager management area, which is used to parse the XML original file into a tree structure and manage the memory allocation and release in the process of creating and operating the XML generated tree.

[0018] Furthermore, when the created deduplicated symbol table is placed in the shared memory, the deduplicated symbol table is stored in the second area of the shared memory, and the basic information of the symbol table is also stored in the first area of the shared memory, so as to realize the symbol table search function through the stored symbol table basic information, wherein the stored symbol table basic information includes one or more of the starting address, the ending address, and the total number of fixed character strings.

[0019] Furthermore, the shared memory pool is used to implement management of the third shared memory area, and the shared memory pool includes shared memory pool context header information, free blocks, and allocated blocks, wherein:

[0020] The shared memory pool context header information includes one or more of the length of the adjacent front free block, the length of the current block, a forward pointer, and a backward pointer. When applying for memory, the free block search starts from the forward pointer and the backward pointer. The shared memory pool context header information also includes statistical information, and the statistical information includes one or more of the current total number of allocated bytes, the current number of allocated bytes, and the current number of released bytes, for checking the current usage of the shared memory pool.

[0021] The free block includes one or more of the length of the adjacent previous free block, the length of the current block, a forward pointer, a backward pointer, and a free area. The free blocks are connected in series using the pointers to form a free list. If there are adjacent free blocks, they are merged. When a free block is allocated space, it needs to be removed from the free list, and its forward pointer, backward pointer, and free area are allocated to the user for use. After that, the free block becomes an allocated block.

[0022] The allocated blocks include one or more of the length of the adjacent front free block, the current block length, and user data. These allocated blocks are removed from the free list after allocation, and the forward pointer and the backward pointer are removed. When the user releases the corresponding space, the forward pointer and the backward pointer of the corresponding allocated block are re-enabled, and the allocated block is appended to the end of the free list. If it can be merged, the merge operation is performed, and then the allocated block becomes a free block.

[0023] Furthermore, when converting XML into a tree structure based on a shared memory pool, the required memory space is allocated through the shared memory pool. During the conversion process, the nodes in the tree structure are connected in series using pointers, and the fixed strings in the tree structure are referenced to a deduplicated symbol table using pointers.

[0024] Furthermore, after converting the XML into a tree structure, locks are assigned to all nodes except the leaf nodes on the XML generated tree to achieve mutual exclusion and synchronization. Specifically:

[0025] Nodes are identified as dynamic nodes or static nodes based on their purpose. Dynamic nodes can be added or deleted, while static nodes cannot be added or deleted. Only the value or attribute of the node can be read and written. For static nodes, the end node of the static node is locked, and for dynamic nodes, the parent node of the dynamic node is locked, so that only one node needs to be locked in one operation.

[0026] Furthermore, the node modification process of the data access interface includes:

[0027] Try to acquire the lock of node A;

[0028] If the lock of node A is not successfully obtained, wait and try again until it is successfully obtained;

[0029] Determine whether the length of the written content exceeds the original length. If not, directly execute the operation of modifying node A and release the lock of node A;

[0030] If it exceeds the original length, try to acquire the lock of the shared memory pool;

[0031] If the shared memory pool lock is not successfully obtained, wait and try again until it is successfully obtained;

[0032] If the shared memory pool lock is acquired successfully, the original space is released and new space is requested according to the new length requirement. After success, the shared memory pool lock is released.

[0033] Execute the node modification operation and write new content to node A;

[0034] Release the lock of node A.

[0035] Furthermore, the reading node process of the data access interface includes:

[0036] Try to acquire the lock of node B;

[0037] If the lock of node B is not successfully obtained, wait and try again until it is successfully obtained;

[0038] Read the content of node B;

[0039] Release the lock of node B.

[0040] Furthermore, the method further comprises:

[0041] A log processing module is provided to save node logs modified through the data access interface and remove duplicate logs.

[0042] On the other hand, the present invention provides a high-performance data model access device based on a shared memory pool, specifically: including at least one processor and a memory, at least one processor and the memory are connected through a data bus, the memory stores instructions that can be executed by at least one processor, and after the instructions are executed by the processor, they are used to complete the high-performance data model access method based on the shared memory pool in the first aspect.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. The XML configuration data is stored in shared memory, so the interface access efficiency is obviously higher. Compared with the traditional method, the configuration of the present invention can effectively reduce memory usage and significantly improve data reading and writing efficiency.

[0045] 2. Convert XML into a tree structure, so that reading and writing operations are free from concerns about XML syntax format and can focus on the data itself.

[0046] 3. When modifying XML, there is no need to save the entire XML file. Only the local modification log needs to be saved, which reduces the flash read and write operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0048] Figure 1 A flowchart of a high-performance data model access method based on a shared memory pool provided in Example 1 of the present invention;

[0049] Figure 2 A schematic diagram of the shared memory area division provided in Example 1 of the present invention;

[0050] Figure 3 This is a flowchart of the overall implementation of the method provided in Example 1 of the present invention;

[0051] Figure 4 A schematic diagram of shared memory mapping provided in Example 2 of the present invention;

[0052] Figure 5 A schematic diagram of the symbol table structure provided in Example 2 of the present invention;

[0053] Figure 6 A schematic diagram of the overall solution of the shared memory pool provided in Example 2 of the present invention;

[0054] Figure 7 A schematic diagram of an idle list provided in Example 2 of the present invention;

[0055] Figure 8 A schematic diagram of the detailed structure of the shared memory pool provided in Example 2 of the present invention;

[0056] Figure 9 A schematic diagram of a tree structure provided in Example 2 of the present invention;

[0057] Figure 10 Schematic diagram of the interface access process provided in Example 2 of the present invention;

[0058] Figure 11 A schematic structural diagram of a high-performance data model access device based on a shared memory pool provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0060] The present invention is an architecture of a specific functional system. Therefore, the specific embodiments mainly illustrate the functional logical relationship between the various structural modules, and do not limit the specific software and hardware implementation methods.

[0061] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0062] Example 1:

[0063] An embodiment of the present invention provides a high-performance data model access method based on a shared memory pool, which is used to solve the problem of rapid access of XML configuration data by multiple processes of a terminal device, reduce memory usage and improve access efficiency, and at the same time solve the problem of pointer usage in shared memory and the problem of dynamic space allocation and release.

[0064] like Figure 1 As shown, the method provided in this embodiment includes the following steps.

[0065] Step 100: Apply for shared memory and map the shared memory to the fixed virtual address of the process. This step is to apply for a piece of shared memory and determine a fixed virtual address (to solve the problem of using pointers in shared memory) when the system is initialized, and map this virtual address to the shared memory.

[0066] Step 200: Traverse the XML, create a deduplicated symbol table, and store it in shared memory. This step is to save memory. Because XML contains many redundant and repeated fixed strings, it is necessary to traverse the XML to remove duplicates and then store the deduplicated symbol table in shared memory to implement the search function.

[0067] Step 300: Parse the XML and convert it into a tree structure based on the shared memory pool to form an XML spanning tree. This step allows for the design of an appropriate tree structure based on the specific characteristics of the XML. Accessing the interface to read and write data requires operating on this XML spanning tree. The tree structure is linked together using pointers, and linked list pointers are used to perform node searches, ultimately enabling data reading and writing. It should be noted that the shared memory in this embodiment refers to a block of memory requested from the system during initialization; the shared memory pool is the software module that manages this memory.

[0068] Step 400: Provide a data access interface and define input parameters. These parameters are used to search for the corresponding memory node in the XML spanning tree, allowing for reading and writing its contents. For example, for Tr069 (CPE WAN Management Protocol), a dot-separated string can be used as the access path for a node. Using this path as an input parameter, the memory node can be searched in the XML spanning tree, allowing for reading and writing its contents. Because configuration data resides in shared memory, interface access efficiency is significantly improved.

[0069] Since all reading and writing of XML nodes are done in shared memory, an additional module for power failure recovery is needed. The method is to save the modification log (use a module to remove duplicate logs to ensure the use of the least logs). This method reduces the reading and writing of flash. Based on the above requirements, refer to Figure 1 , it may also include step 500: providing a log processing module for saving the node logs modified through the data access interface and removing duplicate logs.

[0070] Through the above steps, the embodiment of the present invention can solve the problem of fast access to XML configuration data by multiple processes of terminal devices, reduce memory usage and improve access efficiency, and at the same time solve the problem of pointer usage in shared memory and dynamic space allocation and release.

[0071] Specifically, for step 100, this embodiment divides the shared memory into three areas when applying for shared memory, such as Figure 2The figure shows a schematic diagram of the shared memory area division. The first area is the shared memory context information storage area, which is used to store various basic information, including one or more of the following: the shared memory pool starting address, the XML spanning tree root node address, and the symbol table address size. The second area is the static symbol table storage area, which is used to store the deduplicated symbol table. This is to save memory. Fixed strings in XML, such as node names, are used to create a deduplicated symbol table and place it in this area. When parsing XML, only a pointer to the corresponding string needs to be stored. The third area is the shared memory pool manager management area, which is used to parse the original XML file into a tree structure and manage memory allocation and release during the creation and operation of the XML spanning tree. This area is managed by the shared memory pool module (to solve the problem of dynamic space allocation and release in shared memory). The original XML file is parsed into a tree structure, and a suitable tree structure can be designed based on the specific characteristics of XML. Memory allocation and release during the creation and operation of the XML spanning tree occur in this area. Access interfaces to read and write data must operate on this XML spanning tree. The tree structure is linked together using pointers, and linked list pointers are used to implement node search, ultimately achieving data reading and writing functions.

[0072] For step 200, when this embodiment creates a deduplicated symbol table and places it in the shared memory, the deduplicated symbol table is stored in the second area of the shared memory, and the basic information of the symbol table is also stored in the first area of the shared memory, so as to realize the symbol table search function through the stored symbol table basic information, wherein the stored symbol table basic information includes one or more of the starting address, the ending address, and the total number of fixed character strings.

[0073] In step 300, the shared memory pool of this embodiment is used to manage the third shared memory area and is responsible for allocating and releasing shared memory space. The XML is then parsed and converted into a tree structure. During this conversion process, the nodes in the tree structure are linked together using pointers, and fixed strings in the tree structure are referenced using pointers to the symbol table deduplicated in step 200. Furthermore, all memory space required to generate the tree structure is obtained from the shared memory pool.

[0074] Step 400 is used to implement the XML node data access interface. This interface can be implemented as a dynamic library to be provided to each business process for use. It mainly implements interfaces such as data addition, deletion, modification, and query, and can also be expanded to other interfaces as needed.

[0075] Based on the above configuration of this embodiment, the method of this embodiment can also provide an overall implementation process. Figure 3 As shown, the specific steps include the following steps.

[0076] Step 1: Apply for shared memory, determine a virtual address and map this address to the shared memory.

[0077] Step 2: Traverse the XML to generate a symbol table and store it in shared memory.

[0078] Step 3: Parse the XML and generate a tree structure based on the shared memory pool.

[0079] Step 4: Determine whether there is a data access request. If so, proceed to step 5; if not, proceed to step 8.

[0080] Step 5: The interface processes the access request.

[0081] Step 6: Determine whether to save the modification log to the flash. If yes, proceed to step 7; if no, proceed to step 8.

[0082] Step 7: The log processing module deduplicates and saves the data.

[0083] Step 8: Wait for the access request to be received, and return to step 4 to re-determine whether there is a data access request.

[0084] In summary, this preferred embodiment stores XML configuration data in shared memory, significantly improving interface access efficiency. Compared to traditional approaches, this embodiment effectively reduces memory usage while significantly improving data read and write efficiency. This embodiment converts XML into a tree structure, freeing read and write operations from concerns about XML syntax and formatting, allowing users to focus on the data itself. This embodiment eliminates the need to save the entire XML file for XML modification operations, only saving a local modification log, thus reducing flash read and write operations.

[0085] Example 2:

[0086] Based on the high-performance data model access method based on the shared memory pool provided in Example 1, this Example 2 uses a specific example to illustrate the method of Example 1 in more detail.

[0087] like Figure 4 As shown in FIG, a schematic diagram of shared memory mapping of this embodiment is shown. When applying for a block of shared memory, this embodiment maps this block of memory to a fixed virtual address of the process. To avoid conflicts, this virtual address is selected in the area between the stack space and the file memory mapping base address (mmap_base). In order to reserve as much space as possible for the stack area, it is as close to the file memory mapping base address (mmap_base) as possible. For mapping examples, see reference Figure 4 As shown, all business processes are mapped to this address. Since the process address spaces are independent of each other, each process uses the same address without interfering with each other. The following is an example of how to determine the virtual address.

[0088] For 32-bit Linux: user space uses addresses below 0xc0000000, the stack space grows downward from here, the file memory mapping base address is 0x40000000, and it grows upward (it grows downward after kernel version 2.6.7), the code segment starting address is 0x08048000, followed by the data segment, BSS segment, heap memory, and it grows upward, so we choose to offset 512M bytes from 0x40000000 and align it with 4 bytes, page alignment is 0x60000000, and use this address to map to shared memory.

[0089] For 64-bit Linux: According to different virtual address bit widths, such as 48-bit bit width (the bit width is determined when compiling the Linux kernel), it can be seen that user space uses addresses below 0xffffffffffff, and the stack space also grows downward. The method of calculating the starting address of file memory mapping is: if it is arm or powerpc, it is TASK_SIZE / 4; if it is x86 or mips architecture, it is TASK_SIZE / 3. TASK_SIZE is determined according to the virtual address bit width. For example, when the bit width is 48, mmap_base is (1<<48) / 4=0x400000000000. This address is offset upward by 512M bytes and aligned to 8 bytes. Page alignment is 0x400020000000, and this address can be mapped to shared memory.

[0090] This embodiment applies for a block of shared memory, determines a fixed virtual address, and maps this virtual address to the shared memory, solving the problem of pointer usage in shared memory. For example, a block of 100 bytes of shared memory is applied for by two processes, A and B. A maps the shared memory using the conventional method, and the system returns a virtual address of 1000. B also maps the shared memory using the conventional method, and the system returns a virtual address of 2000. Suppose A stores the address 1016 at the starting address 1000 of the shared memory, which is equivalent to a pointer pointing to address 1016. Then, B attempts to access this pointer. Because B mapped the starting address of the shared memory to 2000, it reads the value at the starting address as 1016. At this point, for B, 1016 is not within the shared memory range. The correct address should be 2016, so this pointer cannot be used. The address spaces of processes A and B are independent of each other. The shared memory address range seen by A is 1000-1099, and the shared memory address range seen by B is 2000-2099. These two virtual address ranges are mapped to the same physical memory, so they can be shared. However, the virtual address ranges are different, resulting in pointer errors. When mapping shared memory using conventional methods, the virtual addresses mapped by each process may be different. The conventional method to solve this problem is to use offsets. For example, A stores an offset of 16 at 1000. After B reads 16, it calculates the actual address as 2000+16=2016, and then accesses the actual address 2016; or B stores an offset of 32 at 2000. After A reads 32, it calculates the actual address as 1000+32=1032, and then accesses the actual address 1032. Although this method is feasible, it is inconvenient. However, using the method of this embodiment to map fixed virtual addresses, there is no problem with the use of pointers, and there is no need to use offsets, which is more convenient to use.

[0091] like Figure 5 As shown in FIG, the symbol table structure diagram of this embodiment is shown. In this embodiment, due to the presence of many redundant and repeated fixed strings in XML, such as node names, in order to save memory, a deduplicated symbol table is created and stored in the second area of the shared memory (see FIG. Figure 2 The static symbol table storage area shown in the figure is Figure 5 , these strings are separated by end symbols (such as '\0'), and the starting address, ending address, and total number of fixed strings of this symbol table are stored in the first area (see Figure 2The shared memory context information storage area shown in the figure) can be used to implement the symbol table search function through these basic information. Specifically, the search function example is as follows: given the starting address P of the symbol table and the total number of symbols N, since all symbols are separated by the character '\0', starting from the starting address P, find the first '\0', and you can calculate the length of the first string. Then, continue from the position of the first '\0' to find the second '\0', and the length of the second string can be calculated in the same way. And so on. When the traversal reaches the Nth string, the traversal ends. The search is to compare with the strings encountered in the traversal process one by one. If they are the same, it means that the string has been found.

[0092] like Figure 6 The figure shows the overall scheme of the shared memory pool of this embodiment. The overall scheme of implementing a shared memory pool in this embodiment is referenced as follows: Figure 6 This shared memory pool is mainly used to implement the third area of shared memory (see Figure 2 The shared memory pool manager manages the shared memory area (shown in Figure 1), allocating and releasing shared memory space and generating a dynamically configurable XML spanning tree. The shared memory pool includes a shared memory pool context header, free blocks, and allocated blocks. Each block has additional information (described below) used to allocate, release, and merge adjacent free blocks. Free blocks are linked together using pointers to form a freelist, facilitating fast space allocation.

[0093] like Figure 7 Figure 1 shows a schematic diagram of a freelist. In this embodiment, the freelist is a circular, doubly linked list, with each block representing a free block. The fptr in the figure is the forward pointer, the bptr is the backward pointer, the head is the head, and the tail is the tail, all connected head to tail. The addresses of these blocks are discontinuous. If there are consecutive blocks, they are merged into a single free block. Allocated blocks are not included in this doubly linked list. The top free block represents a newly generated free block and is inserted after the head of the linked list.

[0094] The detailed structure of the shared memory pool in this embodiment is as follows Figure 8 As shown in FIG, the shared memory pool mainly includes three types of blocks: shared memory pool context header information, free blocks, and allocated blocks.

[0095] Shared memory pool context header information: mainly includes one or more of the length of the adjacent front free block (free block), the current block length, the forward pointer, and the backward pointer. When applying for memory, the search for free blocks starts from the forward pointer and the backward pointer. The shared memory pool context header information also includes statistical information, which includes one or more of the current total number of allocated bytes, the current number of allocations, and the current number of releases, and is used to view the current usage of the shared memory pool.

[0096] Free block: mainly includes one or more of the length of the adjacent free block in front, the length of the current block, the forward pointer, the backward pointer and the free area. The free blocks are connected in series using pointers to form a free list. If there are adjacent free blocks, they are merged. When a free block allocates space, it needs to be removed from the free list, and its forward pointer, backward pointer and free area are allocated to the user. After that, the free block becomes an allocated block.

[0097] Allocated blocks: mainly include one or more of the length of the adjacent front free block, the length of the current block and user data. These allocated blocks are removed from the free list after allocation, so the forward pointer and the backward pointer are no longer needed, so the forward pointer and the backward pointer are also removed. When the user releases the corresponding space, the forward pointer and the backward pointer of the corresponding allocated block are re-enabled, and the allocated block is appended to the end of the free list. If it can be merged, the merge operation is performed, and then the allocated block becomes a free block.

[0098] Specifically, refer to Figure 8 , the available digital numbers correspond to Figure 8 The grid number of a block in the memory is 1, and each block is numbered starting from 1. Adjacent numbers mean that the two memory address spaces are completely continuous.

[0099] (1) Shared memory pool context header information

[0100] 1. Indicates the length of the previous block adjacent to the current memory block. This is the context header area and is initialized to 0.

[0101] 2. The total length of the current memory block. Since all memory blocks are free at initialization, it is the length of the entire memory block. It will be updated as allocation and release actions occur.

[0102] 3. Forward pointer, such as Figure 7 The fptr pointer in.

[0103] 4. Backward pointer, such as Figure 7 The bptr pointer in.

[0104] 5. The total number of bytes currently allocated is the total amount of memory space that has been requested, in bytes, as statistical information.

[0105] 6. The current number of allocations indicates the total number of applications currently made, which is used as statistical information.

[0106] 7. The current number of releases indicates the total number of releases currently, as statistical information.

[0107] (2) Free blocks

[0108] 1. Since adjacent free blocks need to be merged, the value here is 0, indicating that there are no adjacent free blocks in front.

[0109] 2. The total length of the current free memory block.

[0110] 3. Forward pointer, such as Figure 7 The fptr pointer in.

[0111] 4. Backward pointer, such as Figure 7 The bptr pointer in .

[0112] (3) Allocated blocks

[0113] 1. If the adjacent block in front of the currently allocated block is free, the value is the length of the previous free block. If the previous adjacent block has been allocated, the value is 0, indicating that there is no adjacent free block in front.

[0114] 2. The total length of the currently allocated memory block.

[0115] like Figure 9 As shown in the figure, it is a tree structure diagram of this embodiment. The shared memory pool of this embodiment needs to parse XML, convert XML into a tree structure, and then connect the tree structure in series using pointers. The tree structure example is as follows Figure 9As shown, the nodes in the tree structure are also connected in series using pointers, and the specific association relationship is shown in the arrow lines in the figure. In this embodiment, the memory space required in the process of generating the tree structure is obtained through a shared memory pool, and the fixed strings in the tree structure use pointers to reference a deduplicated symbol table; in order to speed up access efficiency, after converting the XML into a tree structure, locks are also assigned to all nodes except the leaf nodes on the XML generation tree to achieve mutual exclusion and synchronization. Specifically: the node is identified as a dynamic or static node according to its purpose. Dynamic nodes can perform instance addition or deletion operations, while static nodes cannot perform addition or deletion operations and can only read and write node values or attributes. In this way, for static nodes, the end node of the static node is locked, and for dynamic nodes, the parent node of the dynamic node is locked. In this way, only one node needs to be locked for one operation, which simplifies the operation. It should be noted that leaf nodes are parameter sets, and there are many of them. Sharing the lock of the parent node is similar to the relationship between folders and files in an operating system. The parent node is a folder and the leaf node is a file. If a lock is assigned to each leaf node, it is a waste of resources and unnecessary. Only the parent node needs to be locked. When a leaf node needs to be accessed, the parent node must be tried to be locked. After success, the leaf node can be accessed. Figure 9 In the example, the leaf node refers to param, and the parent node refers to object (obj). Object is similar to a folder, and param is similar to a file. We only assign locks to object-type nodes, and do not assign locks to param leaf nodes. Object-type nodes are identified as dynamic or static nodes. For a dynamic node, locking the parent node of the dynamic node can be understood as: when you need to read a leaf node below or below a dynamic node, lock the dynamic node; for a static node, locking the end node of the static node can be understood as: when you need to read a leaf node below or below a static node, lock the object node at the last level of the static node. For example: if obj3 is a dynamic node, and you want to read param1 below obj5, lock obj3; if obj3 and obj5 are both static nodes, and you want to read param1 below obj5, lock obj5; the end node refers to the last level of object. Because no lock is assigned to param, you cannot directly lock param.

[0116] After the above settings are made, this embodiment also needs to implement the xml node data access interface. This interface can be implemented as a dynamic library to provide it to each business process for use. It mainly implements interfaces such as data addition, deletion, modification, and query. Other interfaces can also be expanded as needed. The interface access process is as follows: Figure 10 As shown in the figure, a typical process read and write scenario is described, where there is only one shared memory pool lock and multiple node locks depending on the number of nodes. In this way, if multiple processes read different nodes, they can be executed concurrently. The following describes them separately. Figure 10 Flowchart steps of process 1 and process 2.

[0117] The flowchart of process 1 describes the node modification process of the data access interface (that is, the process of modifying node A), and the steps are as follows.

[0118] Step 11 (Acquire the lock of node A): In this step, try to acquire the lock of node A.

[0119] Step 12 (Determine whether the lock of node A is acquired successfully): In this step, if the lock of node A is not acquired successfully, wait and try again until the acquisition is successful. If the acquisition is successful, proceed to the next step, that is, step 13.

[0120] Step 13 (determine whether the length of the written content exceeds the original length): In this step, if the length of the written content exceeds the original length, proceed to step 14, otherwise proceed to step 17.

[0121] Step 14 (Acquire Shared Memory Pool Lock): In this step, an attempt is made to acquire a lock for the shared memory pool.

[0122] Step 15 (determining whether the shared memory pool lock is successfully acquired): In this step, if the shared memory pool lock is not successfully acquired, wait and retry until the acquisition is successful. If the acquisition is successful, proceed to the next step, i.e., step 16.

[0123] Step 16 (Release the shared memory pool lock after releasing the original space and allocating new shared memory): In this step, if the length of the written content exceeds the node's original space capacity, the original space is released, and new space is requested according to the new length requirement. If successful, the shared memory pool lock is released. It should be noted that only when the length of the newly written content is longer than the previous space and exceeds the node's original space capacity will the original space be released and re-allocated. Otherwise, no memory is released or re-allocated.

[0124] Step 17 (Modify and write to node A): In this step, perform the node modification operation and write new content to node A.

[0125] Step 18 (Release the lock of node A): In this step, release the lock of node A and then end the process.

[0126] The flowchart of process 2 describes the node reading process of the data access interface (ie, the process of reading node B), and the steps are as follows.

[0127] Step 21 (Acquire Node B Lock): In this step, try to acquire the lock of node B.

[0128] Step 22 (Determine whether the lock of node B is acquired successfully): In this step, if the lock of node B is not acquired successfully, wait and try again until the acquisition is successful. If the acquisition is successful, proceed to the next step, that is, step 23.

[0129] Step 23 (Read Node B): In this step, the content of Node B is read.

[0130] Step 24 (Release Node B Lock): In this step, release the lock of node B and then end the process.

[0131] Based on the above process, this embodiment also implements a data storage flash module, namely a log processing module, which is mainly responsible for saving the modification logs generated in the fourth step, removing duplicate logs, and ensuring the use of the least logs, so that when the system is restarted, it can be restored to the latest content based on these modification logs.

[0132] In summary, this embodiment stores XML configuration data in shared memory, significantly improving interface access efficiency. Compared to traditional approaches, this embodiment effectively reduces memory usage while significantly improving data read and write efficiency. This embodiment converts XML into a tree structure, freeing read and write operations from concerns about XML syntax and formatting, allowing users to focus on the data itself. This embodiment eliminates the need to save the entire XML file for XML modification operations, only saving a partial modification log, thus reducing flash read and write operations.

[0133] This embodiment can solve the problem of fast access to XML configuration data by multiple processes of a terminal device, reduce memory usage and improve access efficiency, and solve the problem of pointer usage in shared memory and dynamic space allocation and release.

[0134] Example 3:

[0135] Based on the high-performance data model access method based on the shared memory pool provided in the above-mentioned embodiments 1 and 2, embodiment 3 of the present invention further provides a high-performance data model access device based on the shared memory pool that can be used to implement the above-mentioned method and system, such as Figure 11 , which is a schematic diagram of the device architecture of an embodiment of the present invention. The high-performance data model access device based on a shared memory pool of this embodiment includes one or more processors 21 and a memory 22. Figure 11 A processor 21 is taken as an example.

[0136] The processor 21 and the memory 22 may be connected via a bus or other means. Figure 11 The bus connection is taken as an example.

[0137] The memory 22, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules, such as the high-performance data model access method based on a shared memory pool in Examples 1 and 2. The processor 21 executes the various functional applications and data processing of the high-performance data model access device based on a shared memory pool by running the non-volatile software programs, instructions, and modules stored in the memory 22, thereby implementing the high-performance data model access method based on a shared memory pool in Examples 1 and 2.

[0138] The memory 22 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some embodiments, the memory 22 may optionally include a memory remotely located relative to the processor 21, and such remote memory may be connected to the processor 21 via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0139] The program instructions / modules are stored in the memory 22. When executed by one or more processors 21, the high-performance data model access method based on the shared memory pool in the above-mentioned embodiments 1 and 2 is executed. For example, the above-described Figure 1 、 Figure 3 、 Figure 10 The steps shown.

[0140] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a disk or an optical disk, etc.

[0141] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention. Any matters not described in detail in this specification constitute prior art known to those skilled in the art.

Claims

1. A high-performance data model access method based on a shared memory pool, characterized in that: include: Apply for shared memory and map the shared memory to the fixed virtual address of the process; Traverse the XML, create a deduplicated symbol table and place it in shared memory; Parse XML and convert it into a tree structure based on a shared memory pool to form an XML spanning tree; Provide a data access interface and define input parameters to search for the corresponding memory node in the XML spanning tree through the input parameters, so as to read and write its content; When applying for shared memory, the shared memory is divided into three areas, wherein: The first area is a shared memory context information storage area, which is used to store various basic information, including: one or more of the shared memory pool start address, XML spanning tree root node address, and symbol table address size; The second area is a static symbol table storage area, which is used to store the symbol table after deduplication; The third area is the shared memory pool manager management area, which is used to parse the original XML file into a tree structure and manage the memory allocation and release in the process of creating and operating the XML spanning tree; When the created deduplicated symbol table is placed in the shared memory, the deduplicated symbol table is stored in the second area of the shared memory, and the basic information of the symbol table is also stored in the first area of the shared memory, so as to realize the symbol table search function through the stored symbol table basic information, wherein the stored symbol table basic information includes one or more of the starting address, the ending address, and the total number of fixed character strings.

2. The high-performance data model access method based on shared memory pool according to claim 1, characterized in that: The shared memory pool is used to manage the third shared memory area. The shared memory pool includes shared memory pool context header information, free blocks, and allocated blocks, wherein: The shared memory pool context header information includes one or more of the length of the adjacent front free block, the length of the current block, a forward pointer, and a backward pointer. When applying for memory, the free block search starts from the forward pointer and the backward pointer. The shared memory pool context header information also includes statistical information, and the statistical information includes one or more of the current total number of allocated bytes, the current number of allocated bytes, and the current number of released bytes, for checking the current usage of the shared memory pool. The free block includes one or more of the length of the adjacent previous free block, the length of the current block, a forward pointer, a backward pointer, and a free area. The free blocks are connected in series using the pointers to form a free list. If there are adjacent free blocks, they are merged. When a free block is allocated space, it needs to be removed from the free list, and its forward pointer, backward pointer, and free area are allocated to the user for use. After that, the free block becomes an allocated block. The allocated blocks include one or more of the length of the adjacent front free block, the current block length, and user data. These allocated blocks are removed from the free list after allocation, and the forward pointer and the backward pointer are removed. When the user releases the corresponding space, the forward pointer and the backward pointer of the corresponding allocated block are re-enabled, and the allocated block is appended to the end of the free list. If it can be merged, the merge operation is performed, and then the allocated block becomes a free block.

3. The high-performance data model access method based on shared memory pool according to claim 1, characterized in that: When converting XML into a tree structure based on a shared memory pool, the required memory space is allocated through the shared memory pool. During the conversion process, the nodes in the tree structure are connected in series using pointers, and the fixed strings in the tree structure are referenced to a deduplicated symbol table using pointers.

4. The high-performance data model access method based on shared memory pool according to claim 3 is characterized in that: After converting the XML into a tree structure, locks are assigned to all nodes except the leaf nodes on the XML generated tree to achieve mutual exclusion and synchronization. Specifically: Nodes are identified as dynamic nodes or static nodes based on their purpose. Dynamic nodes can be added or deleted, while static nodes cannot be added or deleted. Only the value or attribute of the node can be read and written. For static nodes, the end node of the static node is locked, and for dynamic nodes, the parent node of the dynamic node is locked, so that only one node needs to be locked in one operation.

5. The high-performance data model access method based on a shared memory pool according to any one of claims 1 to 4, characterized in that: The process of modifying a node of the data access interface includes: Try to acquire the lock of node A; If the lock of node A is not successfully obtained, wait and try again until it is successfully obtained; Determine whether the length of the written content exceeds the original length. If not, directly execute the operation of modifying node A and release the lock of node A; If it exceeds the original length, try to acquire the lock of the shared memory pool; If the shared memory pool lock is not successfully obtained, wait and try again until it is successfully obtained; If the shared memory pool lock is acquired successfully, the original space is released and new space is requested according to the new length requirement. After success, the shared memory pool lock is released. Execute the node modification operation and write new content to node A; Release the lock of node A.

6. The high-performance data model access method based on a shared memory pool according to any one of claims 1 to 4, characterized in that: The reading node process of the data access interface includes: Try to acquire the lock of node B; If the lock of node B is not successfully obtained, wait and try again until it is successfully obtained; Read the content of node B; Release the lock of node B.

7. The high-performance data model access method based on a shared memory pool according to any one of claims 1 to 4, characterized in that: The method also includes: A log processing module is provided to save node logs modified through the data access interface and remove duplicate logs.

8. A high-performance data model access device based on a shared memory pool, characterized in that: It includes at least one processor and a memory, the at least one processor and the memory are connected via a data bus, the memory stores instructions that can be executed by the at least one processor, and the instructions, after being executed by the processor, are used to complete the high-performance data model access method based on the shared memory pool as described in any one of claims 1 to 7.

Images