Data access method, device, equipment and computer readable storage medium

By dynamically adjusting the polling thread and shared memory processing methods, the performance bottleneck problem of virtual disk distributed block storage mechanism on solid-state drives is solved, and more efficient data processing performance is achieved.

CN114528123BActive Publication Date: 2024-12-06BEIJING TOPSEC NETWORK SECURITY TECH +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210129169.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-11
Publication Date
2024-12-06
Estimated Expiration
2042-02-11

AI Technical Summary

Technical Problem

The existing virtual disk distributed block storage (TCMU) mechanism cannot effectively support the high read and write performance of solid-state drives, especially when the UIO mechanism provides IO event notification, and encounters performance bottlenecks when transferring shared memory data.

Method used

By obtaining the access frequency of the data processing module to the shared memory, when the frequency exceeds the threshold, the polling thread replaces the character device message notification to enter performance mode; when the frequency increases further, multiple shared memory are created to process IO requests in parallel, using Multiple NUMA nodes divide shared memory to enter ultra-high performance mode.

Benefits of technology

Improve data processing performance, avoid performance bottlenecks in the UIO mechanism, and achieve ultra-high performance when transferring shared memory data, adapting to the needs of high-speed storage devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114528123B_ABST
    Figure CN114528123B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a data access method, device, equipment and computer-readable storage medium. The embodiment of the present disclosure obtains the frequency of data access to the first shared memory by the data processing module; if the frequency is greater than the first threshold, the first data polling thread is started, so that when the TCMU module encounters a bottleneck in the IO event notification using the UIO mechanism, the performance mode is dynamically entered, and the method of replacing the character device message notification with the polling thread is used to improve the performance of data processing. If the frequency is greater than the second threshold, a second shared memory is created, as well as a second data polling thread and a second forwarding thread corresponding to the second shared memory, so that when the shared memory data transmission becomes a performance bottleneck, the ultra-high performance mode is dynamically entered, multiple shared memories are divided using multiple NUMA nodes, and IO requests are processed in parallel, further improving the performance of data processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data access method, device, equipment and computer-readable storage medium. Background Art

[0002] With the continuous development of computer technology, solid-state drives have become more popular. For example, both individual users and enterprise-level data centers have generally used solid-state drives, and the performance of solid-state drives is also improving rapidly. For example, the random read and write times per second of solid-state drives are very high.

[0003] However, the current virtual disk distributed block storage (TCMU) mechanism or shared memory may not be able to support such high read and write performance of SSDs. Summary of the invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a data access method, apparatus, device and computer-readable storage medium, so that when the TCMU module encounters a bottleneck when using the UIO mechanism for IO event notification, it dynamically enters the performance mode and uses a polling thread to replace the character device message notification method, thereby improving the data processing performance; when shared memory data transmission becomes a performance bottleneck, it dynamically enters the ultra-high performance mode, uses multiple NUMA nodes to divide multiple shared memories, and processes IO requests in parallel, thereby further improving the data processing performance.

[0005] In a first aspect, an embodiment of the present disclosure provides a data access method, including:

[0006] Acquire the frequency of data access by the data processing module to the first shared memory;

[0007] If the frequency is greater than a first threshold, starting a first data polling thread, where the first data polling thread is used to access data in the first shared memory in real time;

[0008] If the frequency is greater than a second threshold, creating a second shared memory, and a second data polling thread and a second forwarding thread corresponding to the second shared memory;

[0009] wherein the first threshold is less than the second threshold;

[0010] The first forwarding thread and the second forwarding thread corresponding to the first shared memory are used to forward data to the first shared memory and the second shared memory in parallel;

[0011] The first data polling thread and the second data polling thread are used to access data in the first shared memory and the second shared memory in parallel.

[0012] In a second aspect, an embodiment of the present disclosure provides a data processing device, including:

[0013] A working status detection module, used for obtaining the frequency of data access by the data processing module to the first shared memory;

[0014] A data processing module, configured to start a first data polling thread if the frequency is greater than a first threshold, the first data polling thread being used to access the data in the first shared memory in real time; and to create a second data polling thread corresponding to the second shared memory if the frequency is greater than a second threshold;

[0015] A dynamic multi-shared memory detection module, configured to create a second shared memory if the frequency is greater than a second threshold;

[0016] The virtual disk distributed block storage module is used to create a second forwarding thread for a second shared memory session if the frequency is greater than a second threshold.

[0017] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0018] Memory;

[0019] Processor; and

[0020] Computer programs;

[0021] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect.

[0022] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method described in the first aspect.

[0023] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the method as described above is implemented.

[0024] The data access method, device, equipment and computer-readable storage medium provided by the embodiments of the present disclosure obtain the frequency of data access to the first shared memory by the data processing module; if the frequency is greater than the first threshold, the first data polling thread is started, so that when the TCMU module encounters a bottleneck in IO event notification using the UIO mechanism, the performance mode is dynamically entered, and the method of replacing the character device message notification with the polling thread is used to improve the performance of data processing. If the frequency is greater than the second threshold, a second shared memory is created, as well as a second data polling thread and a second forwarding thread corresponding to the second shared memory, so that when the shared memory data transmission becomes a performance bottleneck, the ultra-high performance mode is dynamically entered, multiple shared memories are divided using multiple NUMA nodes, and IO requests are processed in parallel, further improving the performance of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0027] Figure 1 A schematic diagram of the implementation principle of the equalization mode provided in the embodiment of the present disclosure;

[0028] Figure 2 A flow chart of a data access method provided by an embodiment of the present disclosure;

[0029] Figure 3 A schematic diagram of a performance mode implementation provided by an embodiment of the present disclosure;

[0030] Figure 4 A binding relationship diagram provided for an embodiment of the present disclosure;

[0031] Figure 5 A schematic diagram of the implementation principle of the ultra-high performance mode provided in the embodiment of the present disclosure;

[0032] Figure 6 Another data access method flow chart provided by the embodiment of the present disclosure;

[0033] Figure 7 Another data access method implementation principle diagram provided for the disclosed embodiment;

[0034] Figure 8 A structural diagram of a data access device provided by an embodiment of the present disclosure;

[0035] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0037] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0038] Generally speaking, solid-state drives are widely used by both individual users and enterprise-level data centers. With the development and maturity of solid-state drive technology, its performance has also improved rapidly.

[0039] TCMU is a mechanism provided by the operating system. Its main function is to generate a virtual disk (virtual block device), receive input and output (IO) operations on the virtual block disk, and complete data sharing from kernel to user state through user state and kernel state shared memory. It implements IO event read and write notifications through the IO mechanism (UIO) running in user space and kernel space.

[0040] The existing TCMU mechanism implementation principle diagram (i.e., the equalization mode implementation principle diagram provided by the embodiment of the present disclosure) is as follows: Figure 1 As shown, 10 represents a terminal device, and the memory space of the terminal device is divided into two parts, namely user space and kernel space. The user space includes memory space that can be operated and accessed by ordinary applications; the kernel space includes memory space that can be accessed by the operating system kernel, and the above kernel space is independent of ordinary applications.

[0041] 11 is an application program existing in the terminal device, 12 is a virtual disk (block device) generated by TCMU, and the application program treats this virtual disk the same as an ordinary physical disk. 13, 14, and 15 are kernel space structures. The terminal user performs an IO request operation on the application program 11. The IO request operation passes through the common block layer 13 of the operating system kernel and the small computer system interface (Small Computer System InteSCSI) high-level driver 14, and then is transferred to the SCSI low-level driver 15. Among them, the SCSI low-level driver is implemented by the kernel target subsystem (Linux-IO / TargetCore Mod LIO / TCM) framework 16 and the TCMU module 17.

[0042] When the IO request reaches TCMU17, TCMU forwards the IO request data to the shared memory 18, generates a character device 110 through the UIO mechanism 19, and at the same time TCMU17 sends a readable and writable message event to the character device 110. The implementation method of sending the readable and writable message event can be: writing a custom character or string in the character device, and the custom character or string is used to prompt the data processing module 111 that there is IO request data in the shared memory 18 that needs to be processed.

[0043] Specifically, the data processing module 111 monitors the character device 110 in real time. When a character or string is written to the character device, the data processing device 110 obtains the IO request data from the shared memory 111 and processes it. After the IO request data is processed by the data processing module, the data can be stored in the local storage device 112, the storage array 113, or the distributed storage backend 114 according to the specific implementation.

[0044] The existing TCMU technical solution for implementing IO read / write event notification through the UIO mechanism has the following two problems:

[0045] First, based on the commonly used enterprise-level central processing unit (CPU), 400,000 notifications can be achieved per second. However, in the scenario of using high-speed storage devices, such as solid-state drives (SSDs) based on the high-speed serial computer expansion bus standard (Peripheral Component Interconnect express PCIe), the 4K read and write performance of each card can exceed 1 million times, so the TCMU mechanism cannot fully utilize the storage performance of PCIe SSD, that is, using the UIO mechanism to implement IO read and write message notifications has become a performance bottleneck.

[0046] Secondly, when there is a single high-speed storage device in the storage backend, the read and write speed of the shared memory is greater than the performance of the single high-speed storage device. However, when there are multiple PCIe SSDs in the storage backend, the sum of the performance of the multiple PCIe SSDs is greater than the read and write speed of the shared memory, so the shared memory becomes a new performance bottleneck.

[0047] In view of the above two problems, the embodiments of the present disclosure provide a data access method, which will be introduced below in conjunction with specific embodiments.

[0048] Figure 2A flow chart of a data access method provided in an embodiment of the present disclosure. The data access method can be applied to high-speed storage devices that require ultra-high-performance virtual machines or block storage services. The data access method can be executed by a data access device, which can be implemented in hardware or software and can be integrated in any electronic device with computing capabilities, such as a terminal, specifically including a handheld computer, a tablet computer, a wearable device with a display screen, a desktop computer, a laptop computer, an all-in-one machine, a smart home device, etc.

[0049] Figure 3 , Figure 4 , Figure 5 The following is a specific implementation principle diagram and binding relationship diagram of this embodiment. Figure 3 , Figure 4 , Figure 5 ,right Figure 2 The data access method shown is introduced. The specific steps of this method are as follows:

[0050] S201. Obtain a frequency at which a data processing module accesses data to a first shared memory.

[0051] Specifically, the data processing module is used to process IO request data. Based on the number of IO request data processed per unit time, the frequency of data processing by the data processing module can be counted, and the frequency of data processing can be provided to the working status detection module for calling through an interface.

[0052] The working status detection module calls the data processing frequency interface provided by the data processing module to read the data processing frequency of the data processing module.

[0053] When the data processing frequency of the data processing module does not reach the first threshold, that is, does not reach the bottleneck of the TCMU implementing the IO event notification through the UIO mechanism, the data processing module and the TCMU module are in a balanced mode.

[0054] In the balanced mode, the data processing module and the TCMU module use the existing TCMU mechanism to process IO request data, that is, the IO read / write message event notification is performed through the UIO mechanism. Figure 1 The implementation principle shown is the same and will not be repeated in this embodiment.

[0055] The first threshold may be custom configured according to a specific memory size, and the embodiment of the present disclosure does not impose any limitation on this.

[0056] S202: If the frequency is greater than a first threshold, start a first data polling thread, where the first data polling thread is used to access data in the first shared memory in real time;

[0057] Specifically, as the number of IO requests increases, after the working status detection module reads that the frequency of the data processing module reaches a first threshold, the data processing module and the TCMU module are notified to enter the performance mode.

[0058] After the data processing module receives the notification message of entering the performance mode, it will start the first polling thread, which will poll the shared memory for access. Specifically, the first polling thread polls the shared memory to check whether there is IO request data. If there is, it will read it out and hand over the IO request data to the data processing module for processing.

[0059] Optionally, if the frequency is greater than a first threshold, the data processing module is notified to start a first data polling thread, including: if the frequency is greater than the first threshold, the first data polling thread is started and sending readable and writable message events of the first shared memory is stopped.

[0060] Specifically, after receiving the notification message of entering the performance mode, the TCMU module stops writing custom characters or strings to the character device, and the data processing module no longer monitors whether there is data written to the character device.

[0061] Combine the following Figure 3 The implementation principle of this mode is introduced in detail. Figure 3 As shown, the terminal user performs an IO request operation on the application 11, and the IO request passes through the common block layer 13 of the operating system kernel and the SCSI high-level driver 14, and then is transferred to the SCSI low-level driver 15. Among them, the SCSI low-level driver 15 is implemented by the LIO / TCM framework 16 and the TCMU module 17.

[0062] After the IO request reaches the TCMU module 17, the TCMU stores the IO request data in the shared memory 18. The first polling thread checks in real time whether there is IO request data in the shared memory. If there is, the IO request data is read out and handed over to the data processing module for processing. After the processing is completed, the data can be stored in the local storage device 112, the storage array 113, or the distributed storage backend 114 according to the specific implementation.

[0063] S203: If the frequency is greater than a second threshold, create a second shared memory, and a second data polling thread and a second forwarding thread corresponding to the second shared memory.

[0064] Specifically, as the IO requests further increase, the working status detection module detects that the frequency of data processing by the data processing module reaches the second threshold, that is, the shared memory encounters a performance bottleneck, and notifies the data processing module and the TCMU module to enter the ultra-high performance mode. The second threshold is the maximum bandwidth that can be read and written by the first shared memory. It is understandable that the first threshold is less than the second threshold.

[0065] After TCMU receives the notification message of entering the ultra-high performance mode, it adds a second forwarding thread and checks the current number of shared memories and the number of non-uniform memory access (NUMA) nodes. If the number of shared memories does not exceed the number of NUMA nodes, it notifies the working status detection module to increase the shared memory. After receiving the notification message, the working status detection module notifies the dynamic multi-shared memory management module to add a second shared memory.

[0066] Among them, NUMA refers to non-uniform memory access, which is a computer memory design for multi-processors. It is a system composed of several independent nodes connected by a high-speed dedicated network. Each node can be a single central processing unit CPU or a symmetric multi-processing (SMP) system. Under NUMA, the processor accesses its own local memory faster than non-local memory (the memory location to another processor or memory shared between processors). A simple description is that a physical CPU (generally including multiple logical CPUs or multiple cores) constitutes a node. This node includes not only the CPU, but also a group of memory slots. It can be understood that a physical CPU and a piece of memory constitute a node.

[0067] After receiving the message for adding the second shared memory, the dynamic multi-shared memory management module adds the second shared memory.

[0068] After the data processing module receives the signal to enter the ultra-high performance mode, a second data polling thread is added.

[0069] Among them, the first shared memory, the first forwarding thread, and the first polling thread are bound to the first NUMA node, and the second shared memory, the second forwarding thread, and the second polling thread are bound to the second NUMA node. The binding relationship is as follows: Figure 4 shown.

[0070] It is understandable that the first forwarding thread and the second forwarding thread corresponding to the first shared memory are used to forward data to the first shared memory and the second shared memory in parallel; the first data polling thread and the second data polling thread are used to access the data in the first shared memory and the second shared memory in parallel.

[0071] Specific, combined Figure 5 The implementation principle of this mode is introduced in detail. Figure 5 As shown, the terminal user performs an IO request operation on the application 11, and the IO request operation passes through the common block layer 13 of the operating system kernel and the SCSI high-level driver 14, and then is transferred to the SCSI low-level driver 15. Among them, the SCSI low-level driver is implemented by the LIO / TCM framework 16 and the TCMU module 17.

[0072] After the IO request arrives at the TCMU module 17, based on the dynamic multi-shared memory management module 51 described above, a second shared memory base 52 is newly added, and the first forwarding thread forwards the IO request data to the first shared memory, and the second forwarding thread forwards the IO request data to the second shared memory, wherein the first forwarding thread and the second forwarding thread run in parallel.

[0073] The first polling thread checks in real time whether there is IO request data in the first shared memory. If yes, the IO request data will be read out and handed over to the data processing module. At the same time, the second polling thread checks in real time whether there is IO request data request in the second shared memory. If yes, the IO request data request will be read out and handed over to the data processing module for processing, wherein the first polling thread and the second polling thread run in parallel.

[0074] After the data processing module completes the processing, the data may be stored in a local storage device 112, a storage array 113, or a distributed storage backend 114 according to the specific implementation.

[0075] It is understandable that in the embodiment of the present disclosure, the number of shared memories is related to the number of NUMA nodes. When the number of shared memories does not exceed the number of NUMA nodes, multiple shared memories can be added according to the frequency of IO request data processed by the data processing module. This application does not limit the number of shared memories.

[0076] It is understandable that the three modes of the embodiment of the present disclosure can be automatically switched according to the frequency of IO request data processed by the data processing module. According to the frequency of IO request data processed by the data processing module, it may switch from the balanced mode to the performance mode and then to the ultra-high performance mode, or it may directly switch from the balanced mode to the ultra-high performance mode.

[0077] The disclosed embodiment obtains the frequency of data access to the first shared memory by the data processing module; if the frequency is greater than the first threshold, the first data polling thread is started, so that when the TCMU module encounters a bottleneck in the IO event notification using the UIO mechanism, the performance mode is dynamically entered, and the method of replacing the character device message notification with the polling thread is used to improve the performance of data processing. If the frequency is greater than the second threshold, a second shared memory is created, as well as a second data polling thread and a second forwarding thread corresponding to the second shared memory, so that when the shared memory data transmission becomes a performance bottleneck, the ultra-high performance mode is dynamically entered, multiple shared memories are divided using multiple NUMA nodes, and IO requests are processed in parallel, further improving the performance of data processing.

[0078] In addition, the embodiments of the present disclosure reduce the additional performance overhead caused by cross-NUMA memory access through a one-to-one correspondence between NUMA nodes and shared memory, and dynamically switch between three working modes, that is, it can maintain low IO request data without using polling to waste CPU resources, and can also provide ultra-high performance to adapt to more application scenarios.

[0079] Figure 6 Another data access method according to an embodiment of the present disclosure is as follows: Figure 7 FIG. 1 is a schematic diagram of another data access method according to an embodiment of the present disclosure. Based on the above embodiment, the data access method further includes the following steps: Figure 7 right Figure 6 Another data access method is shown below:

[0080] S601. Create a mapping table, the mapping table including a first corresponding relationship and a second corresponding relationship; the first corresponding relationship includes an identifier of the first shared memory, an identifier of the first forwarding thread, and an identifier of the first data polling thread; the second corresponding relationship includes an identifier of the second shared memory, an identifier of the second forwarding thread, and an identifier of the second data polling thread.

[0081] Specifically, after entering the performance mode, the working status detection module creates a new mapping table, the data processing module starts the first polling thread, and the TCMU module forwards the IO request data to the first shared memory through the first forwarding thread. It can be understood that there is a corresponding relationship between the first shared memory, the first forwarding thread, and the first polling thread. The above relationship is called the first corresponding relationship, and the above corresponding relationship is saved in the mapping table 71. Among them, the mapping table is saved separately in the shared memory space.

[0082] After entering the ultra-high performance mode, the dynamic multi-shared memory management module adds a second shared memory, the TCMU module adds a second forwarding thread, and the data processing module creates a second polling thread. There is a corresponding relationship between the above-mentioned second shared memory, the second forwarding thread, and the second polling thread. The above-mentioned corresponding relationship is called the second corresponding relationship, and the above-mentioned corresponding relationship is stored in the above-mentioned mapping table.

[0083] Optionally, the first corresponding relationship also includes status information of the first shared memory; and the second corresponding relationship also includes status information of the second shared memory.

[0084] Specifically, the status information is used to indicate the existence status of the shared memory. Exemplarily, the shared memory status may include the following: writable, to be reduced, reducible, and reduction completed.

[0085] The writable state includes that the TCMU module can forward IO request data to the shared memory through the forwarding thread corresponding to the shared memory. This state is the initial state of the shared memory.

[0086] The state to be reduced includes that the TCMU module stops forwarding IO request data to the shared memory.

[0087] The reducible state includes that the IO request data in the shared memory has been accessed by the corresponding polling thread, and the shared memory can be reduced.

[0088] The reduction completion status includes that the above shared memory has been deleted.

[0089] The purpose of the state information in the disclosed embodiment is to reflect the real-time state of the shared memory, and the specific classification is not limited in this embodiment.

[0090] S602: Determine a target shared memory from the first shared memory and the second shared memory according to the state information of the first shared memory and the state information of the second shared memory, where the target shared memory is the shared memory to be reduced.

[0091] Optionally, the target shared memory may select the last newly added shared memory as the target shared memory according to the order in which the shared memories are added;

[0092] Specifically, when the amount of IO request data decreases, the ultra-high performance module will automatically switch to the performance mode or the balance mode according to the frequency of the data processing module processing the IO request data. At this time, the working status detection module will use the last newly added shared memory in the mapping table as the target shared memory, and change the status information of the target shared memory in the mapping table to be reduced.

[0093] Optionally, according to the size of the shared memory occupied by the IO request data, the shared memory with the least occupancy is selected as the target shared memory.

[0094] Specifically, when the amount of IO request data decreases, the ultra-high performance module will automatically switch to the performance mode or the balance mode according to the frequency of the data processing module processing the IO request data. The working status detection module obtains the size of each shared memory occupied, selects the shared memory with the least occupancy as the target shared memory, and changes the status information of the target shared memory in the mapping table to be reduced.

[0095] It is understandable that there may be other ways to determine the target shared memory, and the present application does not limit the method for determining the target shared memory.

[0096] S603: Stop forwarding new data to the target shared memory.

[0097] After determining the target shared memory, the TCMU module stops the target forwarding thread and sends IO request data to the target shared memory.

[0098] S604: After all the data in the target shared memory is processed, the target forwarding thread and the target data polling thread corresponding to the target shared memory are stopped, and the target shared memory is deleted.

[0099] Specifically, the working status detection module obtains the data processing status of the target polling thread corresponding to the target shared memory through the data processing module. When the target polling thread cannot read the IO request data, it indicates that the IO request data processing of the target shared memory is completed.

[0100] Optionally, the target data polling thread is stopped after the target forwarding thread corresponding to the target shared memory is stopped.

[0101] Specifically, when the IO request data processing of the target shared memory is completed, the working status detection module notifies the TCMU module to stop the operation of the target forwarding thread, and then notifies the data processing module to stop the operation of the target data polling thread, and modifies the state information of the target shared memory in the mapping table to be reducible. Finally, the dynamic multi-shared memory management module is notified to delete the target shared memory. After the dynamic multi-shared memory management module deletes the target shared memory, the state of the record corresponding to the target shared memory in the mapping table is marked as reduction completed.

[0102] The embodiment of the present disclosure creates a mapping table, wherein the mapping table includes a first corresponding relationship and a second corresponding relationship; the first corresponding relationship includes an identifier of the first shared memory, an identifier of the first forwarding thread, and an identifier of the first data polling thread; the second corresponding relationship includes an identifier of the second shared memory, an identifier of the second forwarding thread, and an identifier of the second data polling thread; according to the status information of the first shared memory and the status information of the second shared memory, a target shared memory is determined from the first shared memory and the second shared memory; forwarding new data to the target shared memory is stopped; when all data in the target shared memory is processed, the target forwarding thread and the target data polling thread corresponding to the target shared memory are stopped, and the target shared memory is deleted, so that the target shared memory can be dynamically deleted according to the corresponding relationship in the mapping table, while ensuring that IO request data will not be lost and the security of IO request data is ensured, the resource occupation of the shared memory is reduced; at the same time, the target forwarding thread and the target polling thread are dynamically deleted, thereby reducing the waste of CPU resources.

[0103] Figure 8 The data access device provided in the embodiment of the present disclosure is a schematic diagram of the structure of the data access device. The data access device may be the terminal device described in the above embodiment, or the user interface processing device may be a terminal device component or assembly as described above. The data access device provided in the embodiment of the present disclosure may execute the processing flow provided in the data access method embodiment, such as Figure 7 As shown, the data access device 80 includes:

[0104] The working status detection module 81 is used to obtain the frequency of the data processing module accessing the first shared memory.

[0105] The data processing module 82 is used to start a first data polling thread if the frequency is greater than a first threshold, and the first data polling thread is used to access the data in the first shared memory in real time; if the frequency is greater than a second threshold, create a second data polling thread corresponding to the second shared memory.

[0106] The dynamic multiple shared memory detection module 83 is configured to create a second shared memory if the frequency is greater than a second threshold.

[0107] The virtual disk distributed block storage module 84 is configured to create a second forwarding thread corresponding to the second shared memory if the frequency is greater than a second threshold.

[0108] Optionally, if the frequency is greater than a first threshold, when the data processing module starts the first data polling thread, the virtual disk distributed block storage module 84 is used to: stop sending the readable and writable message event of the first shared memory.

[0109] Optionally, the working status detection module 81 is also used to: create a mapping table, the mapping table including a first corresponding relationship and a second corresponding relationship; the first corresponding relationship includes the identifier of the first shared memory, the identifier of the first forwarding thread and the identifier of the first data polling thread; the second corresponding relationship includes the identifier of the second shared memory, the identifier of the second forwarding thread and the identifier of the second data polling thread.

[0110] Optionally, the working status detection module 81 is used to create a mapping table, and when the mapping table includes a first corresponding relationship and a second corresponding relationship, specifically: the first corresponding relationship also includes the status information of the first shared memory; the second corresponding relationship also includes the status information of the second shared memory.

[0111] Optionally, the working status detection module 81 is further used to: determine a target shared memory from the first shared memory and the second shared memory according to the status information of the first shared memory and the status information of the second shared memory, wherein the target shared memory is the shared memory to be reduced.

[0112] Optionally, after the working status detection module 81 is used to determine the target shared memory from the first shared memory and the second shared memory, the virtual disk distributed block storage module 84 is further used to: stop forwarding new data to the target shared memory.

[0113] Optionally, after all the data in the target shared memory is processed, the virtual disk distributed block storage module 84 is also used to: stop the target forwarding thread corresponding to the target shared memory; the data processing module 82 is also used to: stop the target data polling thread corresponding to the target shared memory; the dynamic multi-shared memory detection module 83 is also used to: delete the target shared memory.

[0114] Optionally, the data processing module 82 is used to: when stopping the target data polling thread corresponding to the target shared memory, specifically to: after the virtual disk distributed block storage module 84 stops the target forwarding thread corresponding to the target shared memory, stop the target data polling thread.

[0115] Figure 8 The data access device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment, and its implementation principle and technical effect are similar and will not be repeated here.

[0116] Fig. 9 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 8As shown, the device 90 includes: a memory 91, a processor 92, a computer program and a communication interface 93; wherein the computer program is stored in the memory 91 and is configured so that the processor 92 executes the data access method as described above.

[0117] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the data access method described in the above embodiment.

[0118] In addition, an embodiment of the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the data access method as described above is implemented.

[0119] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0120] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data access method, characterized in that: The method comprises: Acquire the frequency at which the data processing module processes data on the first shared memory; If the frequency is greater than a first threshold, starting a first data polling thread, where the first data polling thread is used to access data in the first shared memory in real time; If the frequency is greater than a second threshold, creating a second shared memory, and a second data polling thread and a second forwarding thread corresponding to the second shared memory; wherein the first threshold is less than the second threshold; The first forwarding thread and the second forwarding thread corresponding to the first shared memory are used to forward data to the first shared memory and the second shared memory in parallel; The first data polling thread and the second data polling thread are used to access data in the first shared memory and the second shared memory in parallel.

2. The method according to claim 1, characterized in that: If the frequency is greater than a first threshold, the data processing module is notified to start a first data polling thread, including: If the frequency is greater than a first threshold, a first data polling thread is started, and sending of readable and writable message events of the first shared memory is stopped.

3. The method according to claim 1, characterized in that The method further comprises: Creating a mapping table, wherein the mapping table includes a first corresponding relationship and a second corresponding relationship; The first corresponding relationship includes an identifier of the first shared memory, an identifier of the first forwarding thread, and an identifier of the first data polling thread; The second corresponding relationship includes an identifier of the second shared memory, an identifier of the second forwarding thread, and an identifier of the second data polling thread.

4. The method according to claim 3, characterized in that The first corresponding relationship also includes state information of the first shared memory; The second corresponding relationship also includes status information of the second shared memory.

5. The method according to claim 4, characterized in that The method further comprises: According to the state information of the first shared memory and the state information of the second shared memory, a target shared memory is determined from the first shared memory and the second shared memory, and the target shared memory is the shared memory to be reduced.

6. The method according to claim 5, characterized in that After determining the target shared memory from the first shared memory and the second shared memory, the method further includes: Stop forwarding new data to the target shared memory; When all the data in the target shared memory are processed, the target forwarding thread and the target data polling thread corresponding to the target shared memory are stopped, and the target shared memory is deleted.

7. The method according to claim 6, characterized in that Stopping the target forwarding thread and the target data polling thread corresponding to the target shared memory includes: After stopping the target forwarding thread corresponding to the target shared memory, the target data polling thread is stopped.

8. A data access device, characterized in that: include: A working status detection module, used for obtaining the frequency of data access by the data processing module to the first shared memory; A data processing module, configured to start a first data polling thread if the frequency is greater than a first threshold, the first data polling thread being used to access the data in the first shared memory in real time; and to create a second data polling thread corresponding to the second shared memory if the frequency is greater than a second threshold; A dynamic multi-shared memory detection module, configured to create a second shared memory if the frequency is greater than a second threshold; The virtual disk distributed block storage module is used to create a second forwarding thread corresponding to the second shared memory if the frequency is greater than a second threshold.

9. An electronic device, characterized in that: include Memory; processor; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.