Memory data processing method, device, equipment and readable storage medium

Abstract

In the field of storage technology, the present application discloses a memory data processing method, apparatus, device and readable storage medium, the method comprising: restarting the operating system in the boot state; after entering the operating system, mounting a persistent file system located in the reserved memory; wherein the reserved memory comprises a super block, a metadata area and a data area, the super block stores the system status of the persistent file system, the metadata area stores file information, and the data area stores data content; using a mapping tool and combining the system status and file information, mapping the same file in the persistent file system; and processing the data content using the persistent file system. The technical effect of the present application: by reserving memory and managing it through a persistent memory file system, the application can be combined with the operating system hot start when using this part of the reserved memory, that is, the memory data accessed before and after the system restart can be not lost, so that the application can quickly rebuild the memory data.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a memory data processing method, apparatus, device and readable storage medium. Background Art

[0002] Memory is a crucial component in a computer, consisting of memory chips, circuit boards, and gold fingers. It serves as the bridge for communication with the CPU. Also known as internal storage, memory temporarily stores CPU computational data and exchanges data with external storage devices like hard drives. All computer programs run in memory, and its operation determines the stability of the computer. Therefore, memory performance significantly impacts the computer.

[0003] When the system has security vulnerabilities, important bugs, or needs to upgrade the kernel to provide high performance or new hardware support, the kernel upgrade requires a system restart.

[0004] How to ensure that the memory retains the data content before the system restarts is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The purpose of this application is to provide a memory data processing method, device, equipment and readable storage medium to achieve persistent memory after restarting the operating system.

[0006] To solve the above technical problems, this application provides the following technical solutions:

[0007] A data processing method, comprising:

[0008] In the boot state, restart the operating system;

[0009] After entering the operating system, the persistent file system in the reserved memory is mounted; wherein the reserved memory includes a super block, a metadata area, and a data area, the super block stores the system state of the persistent file system, the metadata area stores file information, and the data area stores data content;

[0010] Mapping the same file in the persistent file system using a mapping tool in combination with the system state and the file information;

[0011] The data content is processed using the persistent file system.

[0012] Preferably, processing the data content using the persistent file system includes:

[0013] In response to the file creation request, creating a file in the persistent file system;

[0014] In response to the file deletion request, deleting the file in the persistent file system;

[0015] In response to the file mapping request, establishing a file mapping in the persistent file system;

[0016] In response to the file mapping release request, the file mapping is released in the persistent file system.

[0017] Preferably, in response to the file creation request, creating a file in the persistent file system includes:

[0018] Parsing the file creation request to determine the file size and file name corresponding to the file to be created;

[0019] Determining whether the persistent file system meets the creation conditions by using the system status;

[0020] If yes, determining a free metadata node from the metadata area;

[0021] Fill the file size and the file name in the idle metadata node;

[0022] Determining a free memory block that matches the file size from the data area, and updating a data index of the determined free memory block;

[0023] The metadata bitmap of the idle metadata node is set to 1, the data bitmap of the idle memory block is set to 1, and the number of created files in the super block is increased by 1.

[0024] Preferably, determining a free memory block matching the file size from the data area includes:

[0025] One or more free memory blocks are determined from the data area using a free continuous memory linked list; wherein the total space of the one or more free memory blocks matches the file size.

[0026] Preferably, determining a free memory block matching the file size from the data area includes:

[0027] Using a free continuous memory linked list, searching from the data area for a single free memory block with the smallest space and matching the file size;

[0028] After a single free memory block search fails, the free continuous memory linked list is used to search for two or more free memory blocks with the smallest total space and matching the file size from the data area.

[0029] Preferably, in response to a file deletion request, deleting a file in the persistent file system includes:

[0030] Parsing the file deletion request to determine the file name of the file to be deleted;

[0031] Using the file name, determining a corresponding metadata node in the persistent file system;

[0032] Using the metadata node to determine whether the file to be deleted is in use;

[0033] If not, traverse the data index items and set the corresponding data bitmap position to 0 in sequence, set the metadata bitmap position corresponding to the metadata node to 0, and reduce the number of created files in the super block by 1;

[0034] If so, it returns a deletion failure.

[0035] Preferably, after traversing the data index items and setting the corresponding data bitmap positions to 0 in sequence, the method further includes:

[0036] If the starting physical address of the memory block corresponding to the to-be-deleted file is adjacent to an ending physical address in the free continuous memory block linked list, or the ending physical address of the memory block is adjacent to a starting physical address in the free continuous memory block linked list, the memory block corresponding to the to-be-deleted file is merged into the corresponding item in the free continuous memory block linked list.

[0037] Preferably, using the metadata node to determine whether the file to be deleted is in use includes:

[0038] Obtaining a reference count of the metadata node;

[0039] If the reference count is 0, it is determined that the file to be deleted is in an unused state;

[0040] If the reference count is greater than 0, it is determined that the file to be deleted is in use.

[0041] Preferably, in response to the file mapping request, establishing a file mapping in the persistent file system includes:

[0042] Parsing the file mapping request to determine the target file to be mapped;

[0043] Performing mapping processing on the target file;

[0044] When a page fault is triggered, a mapping from a virtual address to a physical address is established for the target file;

[0045] After the mapping is completed, the reference count of the metadata node corresponding to the target file is increased by 1 to indicate that the target file is being used and cannot be deleted.

[0046] Preferably, in response to the file mapping release request, releasing the file mapping in the persistent file system includes:

[0047] Parsing the file mapping release request to determine the target file to be unmapped;

[0048] Release the file mapping corresponding to the target file;

[0049] The reference count of the metadata node corresponding to the target file is reduced by 1.

[0050] Preferably, it also includes:

[0051] Determine the largest free continuous memory block from the free continuous memory block list;

[0052] Determine whether the space of the largest free continuous memory block is less than a free memory threshold;

[0053] If yes, migrate the used memory adjacent to the largest free continuous memory block to expand the space of the largest free continuous memory block.

[0054] Preferably, the process of creating the persistent file system includes:

[0055] Dividing the reserved memory using kernel startup parameters;

[0056] Creating a persistent file system and allocating the reserved memory to the persistent file system;

[0057] Determining a page size used by the persistent file system when allocating memory;

[0058] Dividing the reserved memory into regions to obtain the super block region, the metadata region, and the data region;

[0059] The system status includes the number of files created, the total number of files that can be created, and the remaining space.

[0060] The metadata area is divided into a metadata bitmap and metadata nodes. The bits in the metadata bitmap indicate whether the metadata is used. A metadata node includes a file name, a file size, and a data index. The number of data indexes corresponds to the number of memory blocks. The data index includes a starting physical address and an ending physical address of a memory block.

[0061] The data area is divided into blocks according to the page size, and the data area includes a data bitmap and data blocks; a bit in the data bitmap indicates whether a data block is used and the data content in the data block;

[0062] Determining free data blocks using the data bitmap;

[0063] Merge free and continuous data blocks into free memory blocks;

[0064] A free continuous memory block linked list is established according to the size order of the free memory blocks, so as to perform free memory space management based on the free continuous memory block linked list.

[0065] A memory data processing device, comprising:

[0066] System restart module, used to restart the operating system in the boot state;

[0067] A file system mounting module, configured to mount a persistent file system in a reserved memory after entering the operating system; wherein the reserved memory includes a super block, a metadata area, and a data area; the super block stores the system state of the persistent file system, the metadata area stores file information, and the data area stores data content;

[0068] A file mapping module, configured to map the same file in the persistent file system by using a mapping tool in combination with the system state and the file information;

[0069] A memory usage module is used to process the data content using the persistent file system.

[0070] An electronic device, comprising:

[0071] memory for storing computer programs;

[0072] A processor is used to implement the steps of the above-mentioned memory data processing method when executing the computer program.

[0073] A readable storage medium stores a computer program, which implements the steps of the above-mentioned memory data processing method when executed by a processor.

[0074] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned memory data processing method.

[0075] Applying the method provided in the embodiment of the present application, the operating system is restarted in the boot state; after entering the operating system, the persistent file system located in the reserved memory is mounted; wherein the reserved memory includes a super block, a metadata area and a data area, the super block stores the system status of the persistent file system, the metadata area stores file information, and the data area stores data content; using a mapping tool and combining the system status and file information, the same files in the persistent file system are mapped; and the data content is processed using the persistent file system.

[0076] Data in memory is lost when power is lost and after the operating system is restarted. However, when the reserved memory is restarted (i.e., hot-booted) while the operating system is powered on, the data in it is not lost due to the operating system restart. Therefore, this application sets up reserved memory and creates a persistent file system based on the reserved memory. When the operating system is powered on, the operating system is restarted. This ensures that all data in the persistent file system in the reserved memory is not lost due to the operating system restart. To ensure that the data stored in the reserved memory is available, the persistent operating system in the reserved memory is remounted after the operating system is restarted. This reserved memory includes a superblock, a metadata area, and a data area. The superblock stores the system state of the persistent file system, the metadata area stores file information, and the data area stores data content. Then, using a mapping tool and combining the system state and file information, the same files in the persistent file system can be mapped, allowing the persistent file system to continue processing the data content. In other words, by reserving memory and managing it through the persistent memory file system, applications can use this reserved memory in conjunction with the operating system hot boot, ensuring that memory data accessed before and after the system restart is not lost, allowing applications to quickly rebuild memory data.

[0077] Accordingly, the embodiments of the present application also provide memory data processing devices, equipment, readable storage media and computer program products corresponding to the above-mentioned memory data processing method, which have the above-mentioned technical effects and are not repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0079] Figure 1 This is a flowchart of an implementation method of memory data processing in an embodiment of the present application;

[0080] Figure 2 This is a schematic diagram of the area division of a prediction memory in an embodiment of the present application;

[0081] Figure 3 This is a schematic diagram of the regional division of a metadata area in an embodiment of the present application;

[0082] Figure 4 This is a schematic diagram of a metadata node in an embodiment of the present application;

[0083] Figure 5 This is a schematic diagram of the regional division of a data area in an embodiment of the present application;

[0084] Figure 6 This is a schematic diagram of a free continuous memory block chain in an embodiment of the present application;

[0085] Figure 7 This is a schematic diagram of free memory management in an embodiment of the present application;

[0086] Figure 8 This is a structural diagram of a memory data processing device according to an embodiment of the present application;

[0087] Figure 9 This is a schematic structural diagram of an electronic device according to an embodiment of the present application;

[0088] Figure 10 This is a schematic diagram of the specific structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0089] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the embodiments described are only a part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making any creative efforts are within the scope of protection of the present application.

[0090] Please refer to Figure 1 , Figure 1 This is a flow chart of a method for processing memory data in an embodiment of the present application, the method comprising the following steps:

[0091] S101. In the power-on state, restart the operating system.

[0092] When the device is powered on, the memory will not lose power. In addition, the reserved memory will not be managed by the operating system's partner system. Therefore, restarting the operating system while the device is powered on will not cause the data in the reserved memory to be lost.

[0093] In a specific implementation of the present application, the operating system can be restarted through the kexec mechanism (hot restart) so that memory data accessed before and after the restart is not lost.

[0094] The kexec mechanism allows you to jump directly from the currently running kernel to a new kernel (new operating system). The kexec mechanism skips the boot phase, thus reducing reboot time.

[0095] Kexec implements the specific implementation of operating system restart: the new kernel (kernel) image and initrd (root file system) image are stored continuously in the memory, and the location of initrd is recorded in boot_params; switching to the new kernel means jumping to the memory location where the new kernel image is located, and the CPU executes its entry code. The new kernel completes the loading of the root file system content through the initrd location recorded in boot_params.

[0096] S102: After entering the operating system, mount the persistent file system in the reserved memory.

[0097] The reserved memory includes a super block, a metadata area, and a data area. The super block stores the system status of the persistent file system, the metadata area stores file information, and the data area stores data content.

[0098] After rebooting and entering the operating system, you can mount the persistent file system in the reserved memory. For how to mount a persistent file system, please refer to the specific mounting implementation solution, which will not be described in detail here.

[0099] In this application, the reserved memory is divided into a super block, a metadata area, and a data area. The super block stores the system status of the persistent file system, the metadata area stores file information, and the data area stores data content.

[0100] In a specific implementation of the present application, the process of creating a persistent file system includes:

[0101] Use kernel startup parameters to divide reserved memory;

[0102] Create a persistent file system and allocate reserved memory to the persistent file system;

[0103] Determine the page size used when allocating memory for the persistent file system;

[0104] Divide the reserved memory into areas such as super block, metadata area and data area;

[0105] The system status includes the number of files created, the total number of files that can be created, and the remaining space.

[0106] The metadata area is divided into a metadata bitmap and metadata nodes. The bits in the metadata bitmap indicate whether the metadata is used. A metadata node includes the file name, file size, and data index. The number of data indexes corresponds to the number of memory blocks. The data index includes the starting physical address and the ending physical address of the memory block.

[0107] The data area is divided into blocks according to the page size. The data area includes a data bitmap and data blocks. A bit in the data bitmap indicates whether a data block is used and the data content in the data block.

[0108] Use data bitmap to determine free data blocks;

[0109] Merge free and continuous data blocks into free memory blocks;

[0110] A free continuous memory block linked list is established according to the size order of the free memory blocks, so that free memory space management is performed based on the free continuous memory block linked list.

[0111] For ease of understanding, the above steps are combined and explained below.

[0112] Reserve some memory for the persistent file system through kernel startup parameters. For example, in a system with 512GB of memory, reserve 400GB of memory for the persistent file system. Also, specify the page size used when the file system allocates memory. Options include 4KB, 64KB, and 2MB. A larger page size increases TLB hit rates, which improves performance but may waste memory. Therefore, select an appropriate size based on your business scenario.

[0113] Since this reserved memory will not be managed by the operating system's partner system, it needs to be managed based on a persistent file system.

[0114] Please refer to Figure 2 In this embodiment, the reserved memory size is divided into three areas, including a super block area, a metadata area, and a data area.

[0115] Among them, the super block: describes the status of the file system, including the number of files created, the total number of files that can be created, the remaining space size and other key information

[0116] Metadata area: describes the detailed information of each file, including key information such as file name, file size, file content index (i.e. data index). For example, if each metadata is described with 1K bytes (such as 64 bytes for file name, 8 bytes for file size, and several 16-byte data indexes), the total memory size of the metadata area is 2M, so the metadata area is divided into 2000 1K bytes for management, of which the first two areas are used to describe whether each metadata is used, using a bitmap to represent it. Each bit represents whether a metadata is used, 1 means it is used, and 0 means it is not used. Figure 3 As shown in the figure, a 2MB metadata area can describe a total of 2000 – 2 = 1998 metadata. Both the metadata area and the data area have bitmaps. The metadata area's bitmap is the metadata bitmap, and the data area's corresponding bitmap is the data bitmap.

[0117] The information of each metadata is as follows Figure 4 As shown in the figure, the data index structure is {starting physical address, ending physical address}, each 8 bytes, a total of 16 bytes, indicating the start and end of a data block in the data area. There are a total of 119 data index items. If the previous data index item can already describe the memory size of the metadata, the subsequent data index items will be set to 0. For example, if a 1G file is created in the file system, if a 1G continuous memory block is found in the data area, only the {starting physical address, ending physical address} of the first data index item needs to be updated, and all subsequent data index items need to be set to 0.

[0118] Data area: stores specific data information. The data area can be divided into blocks according to the page size specified by the kernel startup parameters (such as 4K, 64K, 2M). Assuming the specified page size is 2M, the division is as follows: Figure 5 As shown, it is also divided into two areas. The data bitmap area is similar to the metadata bitmap, indicating whether each data block is used, 1 means used, and 0 means not used.

[0119] In addition, a temporary free continuous memory block linked list is maintained in the memory according to the data bitmap, all free continuous data blocks are merged to obtain free memory blocks, and then sorted and managed according to the size of the free memory blocks, such as Figure 6 shown.

[0120] S103: Map the same files in the persistent file system using a mapping tool in combination with the system status and file information.

[0121] Specifically, you can use mmap (a mapping tool for mapping a file or other object into memory) to map the same file in the persistent file system. For details on how to implement file mapping, refer to the specific implementation of mmap and will not be detailed here.

[0122] S104: Process the data content using the persistent file system.

[0123] In this embodiment, after the file mapping is completed, management can be performed based on the data content in the persistent file system that is retained in the memory, such as creating, deleting, accessing, etc. files in response to application requirements.

[0124] In a specific embodiment of the present application, processing data content using a persistent file system includes:

[0125] In response to the file creation request, create a file in the persistent file system;

[0126] In response to a file deletion request, deleting a file in the persistent file system;

[0127] In response to the file mapping request, establishing a file mapping in the persistent file system;

[0128] In response to the file mapping release request, the file mapping is released in the persistent file system.

[0129] When an application or virtual machine uses memory, it can initiate a request to the persistent file system. Based on different requests, the persistent file system can perform corresponding data processing to manage the data content stored in the persistent memory.

[0130] Creating a file involves creating a file in the persistent file system based on demand, while deleting a file involves deleting an already created file based on demand. File mapping involves establishing a mapping between virtual and physical addresses when a user needs to access a file, ensuring that the file is not deleted. Unmapping, on the other hand, indicates that the file is no longer in use and removes the mapping between virtual and physical addresses.

[0131] For ease of understanding, this application is specifically based on how the file persistence system manages the data content retained in the memory. The following is a detailed description for different requests.

[0132] For a file creation request, that is, in response to the file creation request, a file is created in the persistent file system, including:

[0133] Parse the file creation request and determine the file size and file name corresponding to the file to be created;

[0134] Use the system status to determine whether the persistent file system meets the creation conditions;

[0135] If yes, determine the free metadata node from the metadata area;

[0136] Fill the free metadata nodes with file size and file name;

[0137] Determine a free memory block that matches the file size from the data area, and update the data index of the determined free memory block;

[0138] Set the metadata bitmap of the free metadata node to 1, set the data bitmap of the free memory block to 1, and increase the number of created files in the super block by 1.

[0139] The process of determining a free memory block that matches the file size from the data area includes:

[0140] One or more free memory blocks are determined from the data area using a free continuous memory linked list; wherein the total space of the one or more free memory blocks matches the file size.

[0141] The process of determining a free memory block that matches the file size from the data area includes:

[0142] Use the free continuous memory linked list to find a single free memory block with the smallest space and matching the file size from the data area;

[0143] After a single free memory block search fails, the free continuous memory linked list is used to search for two or more free memory blocks with the smallest total space and matching the file size from the data area.

[0144] For ease of description, the above steps are combined and explained below.

[0145] When creating a file of a specified size in a persistent file system, first determine whether the remaining space is sufficient and whether there are still files that can be created based on the super block. If the creation conditions are met (such as the number of files that can be created is greater than 0, and the remaining space is greater than the size of the file to be created), find a free metadata node in the metadata area based on the metadata bitmap and fill the name and size in the metadata in the metadata node. Then allocate one or more continuous memory blocks from the data area based on the free continuous memory linked list and update the data index item until it is consistent with the size of the created file. When completed, the corresponding positions of the metadata bitmap and data bitmap are set to 1, and the number of created files in the super block is increased by 1.

[0146] Among them, when creating a file of a specified size in the file system, it is preferred to find a minimum memory block that can meet the requirements from the free memory block list and set the corresponding position to 1 in the data bitmap. In this way, the actual memory utilization can be improved.

[0147] For a file deletion request, that is, in response to the file deletion request, deleting the file in the persistent file system includes:

[0148] Parse the file deletion request and determine the file name of the file to be deleted;

[0149] Use the file name to determine the corresponding metadata node in the persistent file system;

[0150] Use metadata nodes to determine whether the file to be deleted is in use;

[0151] If not, traverse the data index items and set the corresponding data bitmap to 0, set the metadata bitmap corresponding to the metadata node to 0, and reduce the number of created files in the super block by 1;

[0152] If so, it returns a deletion failure.

[0153] The metadata node is used to determine whether the file to be deleted is in use, including:

[0154] Get the reference count of the metadata node;

[0155] If the reference count is 0, it is determined that the file to be deleted is in an unused state;

[0156] If the reference count is greater than 0, it is determined that the file to be deleted is in use.

[0157] Among them, after traversing the data index items and setting the corresponding data bitmap positions to 0, it also includes:

[0158] If the starting physical address of the memory block corresponding to the file to be deleted is adjacent to an ending physical address in the free continuous memory block linked list, or the ending physical address of the memory block is adjacent to a starting physical address in the free continuous memory block linked list, the memory block corresponding to the file to be deleted is merged into the corresponding item in the free continuous memory block linked list.

[0159] For ease of description, the above steps are combined and explained below.

[0160] When deleting a file in the persistent file system, the metadata node is found according to the file name to determine whether the metadata is still in use. If not, the corresponding data bitmap position is set to 0 through the traversal data index items and the free continuous memory block linked list is merged. After completion, the metadata bitmap position corresponding to the metadata node is set to 0, and finally the number of created files in the super block is reduced by 1. If the metadata is still in use, the deletion failure is returned, indicating that the file is in use.

[0161] When a file is deleted from the file system, whenever a block of memory is released based on the data index item of the metadata, the corresponding position in the data bitmap is set to 0. At the same time, if the starting physical address of the memory block is adjacent to an ending physical address in the free continuous memory block list, or the ending physical address of the memory block is adjacent to a starting physical address in the free continuous memory block list, it is merged into the corresponding item in the free continuous memory block list.

[0162] For a file mapping request, that is, in response to the file mapping request, a file mapping is established in the persistent file system, including:

[0163] Parse the file mapping request and determine the target file to be mapped;

[0164] Map the target file;

[0165] When a page fault is triggered, a mapping from virtual address to physical address is established for the target file;

[0166] After the mapping is completed, the reference count of the metadata node corresponding to the target file is increased by 1 to indicate that the target file is in use and cannot be deleted.

[0167] For ease of description, the above steps are combined and explained below.

[0168] When an application uses a persistent file system, it needs to map the files therein through mmap. In this way, when a page fault is triggered, the page fault processing logic of the persistent file system will establish a mapping from virtual addresses to physical addresses. The specific physical addresses mapped to are queried based on the data index items in the metadata node. After the mapping is completed, the corresponding metadata reference count is increased by 1, indicating that the file is in use and cannot be deleted.

[0169] For the file mapping contact request, in response to the file mapping release request, the file mapping is released in the persistent file system, including:

[0170] Parse the file mapping release request and determine the target file to be unmapped;

[0171] Release the file mapping corresponding to the target file;

[0172] Decrement the reference count of the metadata node corresponding to the target file by 1.

[0173] For ease of description, the above steps are combined and explained below.

[0174] When the application no longer uses the persistent file system, it unmaps the corresponding file through munmap (a tool for unmapping files or other objects into memory), and the reference count of the metadata is reduced by 1. When the reference count reaches 0, the file can be deleted.

[0175] It can be seen that when applications such as virtual machines and in-memory databases need to use memory, they first create a file of the corresponding size in the persistent memory file system, and then use mmap to map the file to the virtual address. When the memory data needs to be persisted when the system is restarted, it is necessary to use the kexec mechanism to restart the system. After entering the new system, the persistent file system is mounted and the same file in the persistent file system is mapped again using mmap. At this time, the memory data accessed by the application will be consistent with that before the system restart, thereby achieving the purpose of quickly resuming the application.

[0176] Applying the method provided in the embodiment of the present application, the operating system is restarted in the boot state; after entering the operating system, the persistent file system located in the reserved memory is mounted; wherein the reserved memory includes a super block, a metadata area and a data area, the super block stores the system status of the persistent file system, the metadata area stores file information, and the data area stores data content; using a mapping tool and combining the system status and file information, the same files in the persistent file system are mapped; and the data content is processed using the persistent file system.

[0177] Data in memory is lost when power is lost and after the operating system is restarted. However, when the reserved memory is restarted (i.e., hot-booted) while the operating system is powered on, the data in it is not lost due to the operating system restart. Therefore, this application sets up reserved memory and creates a persistent file system based on the reserved memory. When the operating system is powered on, the operating system is restarted. This ensures that all data in the persistent file system in the reserved memory is not lost due to the operating system restart. To ensure that the data stored in the reserved memory is available, the persistent operating system in the reserved memory is remounted after the operating system is restarted. This reserved memory includes a superblock, a metadata area, and a data area. The superblock stores the system state of the persistent file system, the metadata area stores file information, and the data area stores data content. Then, using a mapping tool and combining the system state and file information, the same files in the persistent file system can be mapped, allowing the persistent file system to continue processing the data content. In other words, by reserving memory and managing it through the persistent memory file system, applications can use this reserved memory in conjunction with the operating system hot boot, ensuring that memory data accessed before and after the system restart is not lost, allowing applications to quickly rebuild memory data.

[0178] It should be noted that, based on the above embodiments, the embodiments of the present application also provide corresponding improved solutions. In the preferred / improved embodiments, the same steps or corresponding steps as those in the above embodiments can be referenced to each other, and the corresponding beneficial effects can also be referenced to each other, and will not be described in detail in the preferred / improved embodiments of this document.

[0179] In a specific embodiment of the present application, it also includes:

[0180] Determine the largest free continuous memory block from the free continuous memory block list;

[0181] Determine whether the space of the largest free continuous memory block is less than the free memory threshold;

[0182] If so, the used memory adjacent to the largest free continuous memory block is migrated to expand the space of the largest free continuous memory block.

[0183] For ease of description, the above steps are combined and explained below.

[0184] When the largest free continuous memory block is less than MAX_FREE_MEM (i.e., the free memory threshold) (MAX_FREE_MEM is configurable and can be configured according to the reserved memory size), and the total remaining memory is greater than MAX_FREE_MEM, memory consolidation will be performed. Its function is to organize a continuous memory block through memory migration, such as Figure 7 As shown, first find the largest free continuous memory block, and then use the memory at the front or back end of the memory block (such as Figure 7 The memory corresponding to the solid line box) is migrated to a smaller free memory block (such as Figure 7 In the memory corresponding to the dotted box in the figure, the maximum free memory block size is allocated until the condition is met. In this way, when creating a file, continuous free data blocks can be allocated as much as possible.

[0185] Corresponding to the above method embodiment, an embodiment of the present application further provides a memory data processing device. The memory data processing device described below and the memory data processing method described above can refer to each other.

[0186] See also Figure 9 As shown, the device includes the following modules:

[0187] The system restart module 101 is used to restart the operating system in the boot state;

[0188] The file system mounting module 102 is used to mount the persistent file system in the reserved memory after entering the operating system. The reserved memory includes a super block, a metadata area, and a data area. The super block stores the system status of the persistent file system, the metadata area stores file information, and the data area stores data content.

[0189] The file mapping module 103 is used to map the same files in the persistent file system by using a mapping tool and combining the system status and file information;

[0190] The memory use module 104 is used to process data content using the persistent file system.

[0191] Using the device provided in the embodiment of the present application, the operating system is restarted in the boot state; after entering the operating system, the persistent file system located in the reserved memory is mounted; wherein the reserved memory includes a super block, a metadata area and a data area, the super block stores the system status of the persistent file system, the metadata area stores file information, and the data area stores data content; using a mapping tool and combining the system status and file information, the same files in the persistent file system are mapped; and the data content is processed using the persistent file system.

[0192] Data in memory is lost when power is lost and after the operating system is restarted. However, when the reserved memory is restarted (i.e., hot-booted) while the operating system is powered on, the data in it is not lost due to the operating system restart. Therefore, this application sets up reserved memory and creates a persistent file system based on the reserved memory. When the operating system is powered on, the operating system is restarted. This ensures that all data in the persistent file system in the reserved memory is not lost due to the operating system restart. To ensure that the data stored in the reserved memory is available, the persistent operating system in the reserved memory is remounted after the operating system is restarted. This reserved memory includes a superblock, a metadata area, and a data area. The superblock stores the system state of the persistent file system, the metadata area stores file information, and the data area stores data content. Then, using a mapping tool and combining the system state and file information, the same files in the persistent file system can be mapped, allowing the persistent file system to continue processing the data content. In other words, by reserving memory and managing it through the persistent memory file system, applications can use this reserved memory in conjunction with the operating system hot boot, ensuring that memory data accessed before and after the system restart is not lost, allowing applications to quickly rebuild memory data.

[0193] In a specific embodiment of the present application, the memory usage module is specifically configured to create a file in the persistent file system in response to a file creation request;

[0194] In response to a file deletion request, deleting a file in the persistent file system;

[0195] In response to the file mapping request, establishing a file mapping in the persistent file system;

[0196] In response to the file mapping release request, the file mapping is released in the persistent file system.

[0197] In a specific embodiment of the present application, the memory usage module is specifically used to parse the file creation request and determine the file size and file name corresponding to the file to be created;

[0198] Use the system status to determine whether the persistent file system meets the creation conditions;

[0199] If yes, determine the free metadata node from the metadata area;

[0200] Fill the free metadata nodes with file size and file name;

[0201] Determine a free memory block that matches the file size from the data area, and update the data index of the determined free memory block;

[0202] Set the metadata bitmap of the free metadata node to 1, set the data bitmap of the free memory block to 1, and increase the number of created files in the super block by 1.

[0203] In a specific embodiment of the present application, the memory usage module is specifically used to use the free continuous memory linked list to determine one or more free memory blocks from the data area; wherein the total space of the one or more free memory blocks matches the file size.

[0204] In a specific embodiment of the present application, the memory usage module is specifically configured to use a free continuous memory linked list to search a single free memory block with the smallest space and matching the file size from the data area;

[0205] After a single free memory block search fails, the free continuous memory linked list is used to search for two or more free memory blocks with the smallest total space and matching the file size from the data area.

[0206] In a specific embodiment of the present application, the memory usage module is specifically used to parse the file deletion request and determine the file name of the file to be deleted;

[0207] Use the file name to determine the corresponding metadata node in the persistent file system;

[0208] Use metadata nodes to determine whether the file to be deleted is in use;

[0209] If not, traverse the data index items and set the corresponding data bitmap to 0, set the metadata bitmap corresponding to the metadata node to 0, and reduce the number of created files in the super block by 1;

[0210] If so, it returns a deletion failure.

[0211] In a specific embodiment of the present application, the memory usage module is specifically used to merge the memory block corresponding to the file to be deleted into the corresponding item in the free continuous memory block linked list if the starting physical address of the memory block corresponding to the file to be deleted is adjacent to an ending physical address in the free continuous memory block linked list, or the ending physical address of the memory block is adjacent to a starting physical address in the free continuous memory block linked list.

[0212] In a specific embodiment of the present application, the memory usage module is specifically used to obtain the reference count of the metadata node;

[0213] If the reference count is 0, it is determined that the file to be deleted is in an unused state;

[0214] If the reference count is greater than 0, it is determined that the file to be deleted is in use.

[0215] In a specific embodiment of the present application, the memory usage module is specifically used to parse the file mapping request and determine the target file to be mapped;

[0216] Map the target file;

[0217] When a page fault is triggered, a mapping from virtual address to physical address is established for the target file;

[0218] After the mapping is completed, the reference count of the metadata node corresponding to the target file is increased by 1 to indicate that the target file is in use and cannot be deleted.

[0219] In a specific embodiment of the present application, the memory usage module is specifically used to parse the file mapping release request and determine the target file to be unmapped;

[0220] Release the file mapping corresponding to the target file;

[0221] Decrement the reference count of the metadata node corresponding to the target file by 1.

[0222] In a specific embodiment of the present application, it also includes:

[0223] A free memory management module is used to determine the largest free continuous memory block from the free continuous memory block linked list;

[0224] Determine whether the space of the largest free continuous memory block is less than the free memory threshold;

[0225] If so, the used memory adjacent to the largest free continuous memory block is migrated to expand the space of the largest free continuous memory block.

[0226] In a specific embodiment of the present application, the file system creation module is used to divide the reserved memory using kernel startup parameters;

[0227] Create a persistent file system and allocate reserved memory to the persistent file system;

[0228] Determine the page size used when allocating memory for the persistent file system;

[0229] Divide the reserved memory into areas such as super block, metadata area and data area;

[0230] The system status includes the number of files created, the total number of files that can be created, and the remaining space.

[0231] The metadata area is divided into a metadata bitmap and metadata nodes. The bits in the metadata bitmap indicate whether the metadata is used. A metadata node includes the file name, file size, and data index. The number of data indexes corresponds to the number of memory blocks. The data index includes the starting physical address and the ending physical address of the memory block.

[0232] The data area is divided into blocks according to the page size. The data area includes a data bitmap and data blocks. A bit in the data bitmap indicates whether a data block is used and the data content in the data block.

[0233] Use data bitmap to determine free data blocks;

[0234] Merge free and continuous data blocks into free memory blocks;

[0235] A free continuous memory block linked list is established according to the size order of the free memory blocks, so that free memory space management is performed based on the free continuous memory block linked list.

[0236] Corresponding to the above method embodiment, an embodiment of the present application further provides an electronic device. The electronic device described below and the memory data processing method described above can refer to each other.

[0237] See also Figure 10 As shown, the electronic device includes:

[0238] Memory 332, for storing computer programs;

[0239] The processor 322 is configured to implement the steps of the memory data processing method of the above method embodiment when executing a computer program.

[0240] For details, please refer to Figure 10 , Figure 10 This is a schematic diagram of the specific structure of an electronic device provided in this embodiment. The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) (for example, one or more processors) and a memory 332, and the memory 332 stores one or more computer programs 342 or data 344. Among them, the memory 332 can be a temporary storage or a permanent storage. The program stored in the memory 332 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the data processing device. Furthermore, the processor 322 can be configured to communicate with the memory 332 to execute a series of instruction operations in the memory 332 on the electronic device 301.

[0241] The electronic device 301 may further include one or more power supplies 326 , one or more wired or wireless network interfaces 350 , one or more input / output interfaces 358 , and / or one or more operating systems 341 .

[0242] The steps in the memory data processing method described above can be implemented by the structure of an electronic device.

[0243] Corresponding to the above method embodiment, an embodiment of the present application further provides a readable storage medium. The readable storage medium described below and the memory data processing method described above can refer to each other.

[0244] A readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the memory data processing method of the above method embodiment.

[0245] The readable storage medium may specifically be any readable storage medium capable of storing program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0246] Corresponding to the above method embodiment, an embodiment of the present application further provides a computer program product. The computer program product described below and the memory data processing method described above can refer to each other.

[0247] A computer program product includes a computer program / instruction. When the computer program / instruction is executed by a processor, the steps of the above-mentioned memory data processing method are implemented.

[0248] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0249] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0250] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0251] Finally, it should be noted that, in this document, relationships such as first and second, etc., are used solely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms include, comprise, or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0252] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A memory data processing method, characterized in that: include: In the boot state, restart the operating system; After entering the operating system, the persistent file system in the reserved memory is mounted; wherein the reserved memory includes a super block, a metadata area, and a data area, the super block stores the system state of the persistent file system, the metadata area stores file information, and the data area stores data content; Mapping the same file in the persistent file system using a mapping tool in combination with the system state and the file information; Processing the data content using the persistent file system; The process of creating the persistent file system includes: Dividing the reserved memory using kernel startup parameters; Creating a persistent file system and allocating the reserved memory to the persistent file system; Determining a page size used by the persistent file system when allocating memory; Dividing the reserved memory into regions to obtain the super block region, the metadata region, and the data region; The system status includes the number of files created, the total number of files that can be created, and the remaining space. The metadata area is divided into a metadata bitmap and metadata nodes. The bits in the metadata bitmap indicate whether the metadata is used. A metadata node includes a file name, a file size, and a data index. The number of data indexes corresponds to the number of memory blocks. The data index includes a starting physical address and an ending physical address of a memory block. The data area is divided into blocks according to the page size, and the data area includes a data bitmap and data blocks; a bit in the data bitmap indicates whether a data block is used, and the data block stores data content; Determining free data blocks using the data bitmap; Merge free and continuous data blocks into free memory blocks; A free continuous memory block linked list is established according to the size order of the free memory blocks, so as to perform free memory space management based on the free continuous memory block linked list.

2. The method according to claim 1, characterized in that Processing the data content using the persistent file system includes: In response to the file creation request, creating a file in the persistent file system; In response to the file deletion request, deleting the file in the persistent file system; In response to the file mapping request, establishing a file mapping in the persistent file system; In response to the file mapping release request, the file mapping is released in the persistent file system.

3. The method according to claim 2, characterized in that In response to the file creation request, creating a file in the persistent file system includes: Parsing the file creation request to determine the file size and file name corresponding to the file to be created; Determining whether the persistent file system meets the creation conditions by using the system status; If yes, determining a free metadata node from the metadata area; Fill the file size and the file name in the idle metadata node; Determining a free memory block that matches the file size from the data area, and updating a data index of the determined free memory block; The metadata bitmap of the idle metadata node is set to 1, the data bitmap of the idle memory block is set to 1, and the number of created files in the super block is increased by 1.

4. The method according to claim 3, characterized in that Determining a free memory block matching the file size from the data area includes: One or more free memory blocks are determined from the data area using a free continuous memory linked list; wherein the total space of the one or more free memory blocks matches the file size.

5. The method according to claim 3, characterized in that Determining a free memory block matching the file size from the data area includes: Using a free continuous memory linked list, searching from the data area for a single free memory block with the smallest space and matching the file size; After a single free memory block search fails, the free continuous memory linked list is used to search for two or more free memory blocks with the smallest total space and matching the file size from the data area.

6. The method according to claim 2, characterized in that In response to the file deletion request, deleting the file in the persistent file system includes: Parsing the file deletion request to determine the file name of the file to be deleted; Using the file name, determining a corresponding metadata node in the persistent file system; Using the metadata node to determine whether the file to be deleted is in use; If not, traverse the data index items and set the corresponding data bitmap position to 0, set the metadata bitmap position corresponding to the metadata node to 0, and reduce the number of created files in the super block by 1; If so, it returns a deletion failure.

7. The method according to claim 6, characterized in that After traversing the data index items and setting the corresponding data bitmap positions to 0, it also includes: If the starting physical address of the memory block corresponding to the to-be-deleted file is adjacent to an ending physical address in the free continuous memory block linked list, or the ending physical address of the memory block is adjacent to a starting physical address in the free continuous memory block linked list, the memory block corresponding to the to-be-deleted file is merged into the corresponding item in the free continuous memory block linked list.

8. The method according to claim 6, characterized in that Using the metadata node to determine whether the file to be deleted is in use includes: Obtaining a reference count of the metadata node; If the reference count is 0, it is determined that the file to be deleted is in an unused state; If the reference count is greater than 0, it is determined that the file to be deleted is in use.

9. The method according to claim 2, characterized in that In response to the file mapping request, establishing a file mapping in the persistent file system includes: Parsing the file mapping request to determine the target file to be mapped; Performing mapping processing on the target file; When a page fault is triggered, a mapping from a virtual address to a physical address is established for the target file; After the mapping is completed, the reference count of the metadata node corresponding to the target file is increased by 1 to indicate that the target file is being used and cannot be deleted.

10. The method according to claim 2, characterized in that In response to the file mapping release request, the file mapping is released in the persistent file system, including: Parsing the file mapping release request to determine the target file to be unmapped; Release the file mapping corresponding to the target file; The reference count of the metadata node corresponding to the target file is reduced by 1.

11. The method according to claim 1, characterized in that Also includes: Determine the largest free continuous memory block from the free continuous memory block list; Determine whether the space of the largest free continuous memory block is less than a free memory threshold; If yes, migrate the used memory adjacent to the largest free continuous memory block to expand the space of the largest free continuous memory block.

12. A memory data processing device, characterized in that: include: System restart module, used to restart the operating system in the boot state; A file system mounting module, configured to mount a persistent file system in a reserved memory after entering the operating system; wherein the reserved memory includes a super block, a metadata area, and a data area; the super block stores the system state of the persistent file system, the metadata area stores file information, and the data area stores data content; A file mapping module, configured to map the same file in the persistent file system by using a mapping tool in combination with the system state and the file information; A memory usage module, configured to process the data content using the persistent file system; A file system creation module is used to divide the reserved memory using kernel startup parameters; create a persistent file system and allocate the reserved memory to the persistent file system; determine the page size used by the persistent file system when allocating memory; divide the reserved memory into regions to obtain the super block, the metadata area and the data area; wherein the system status includes the number of created files, the total number of files that can be created and the remaining space size; the metadata area is divided into a metadata bitmap and metadata nodes, the bits in the metadata bitmap indicate whether the metadata is used, a metadata node includes a file name, a file size and a data index, the number of data indexes corresponds to the number of memory blocks, and the data index includes the starting physical address and the ending physical address of the memory block; the data area is divided into blocks according to the page size, and the data area includes a data bitmap and data blocks; a bit in the data bitmap indicates whether a data block is used, and the data block stores data content; using the data bitmap to determine free data blocks; merging free and continuous data blocks into free memory blocks; establishing a free continuous memory block linked list in the order of size of the free memory blocks, so as to manage free memory space based on the free continuous memory block linked list.

13. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the memory data processing method according to any one of claims 1 to 11 when executing the computer program.

14. A readable storage medium, characterized in that The readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the memory data processing method according to any one of claims 1 to 11.

Images