Database double-cache optimization method and device, equipment and medium

Abstract

The invention relates to the technical field of data processing, can be applied to the field of medical health, and discloses a database double-cache optimization method, device, equipment and medium, and the method comprises the following steps: receiving configuration parameters which are input by a user and are used for triggering a direct I / O mode, and configuring a database according to the configuration parameters, the configuration parameters comprise a plurality of different file types; in response to the fact that the configuration of the database takes effect, performing integral multiple alignment of the size of a predefined block on a heap memory, a stack memory and a shared memory of a memory buffer area of the database; if it is detected that the file type of the received to-be-processed data belongs to one file type in the configuration parameters, a direct I / O mode is triggered, and the to-be-processed data is directly processed based on the aligned memory buffer area and a storage device of an operating system, so that a database engine can directly perform data interaction with the storage device; and the operation system cache is bypassed, so that the data processing efficiency of the medical system is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology and can be applied to the field of medical health, and in particular to a database double cache optimization method, device, equipment and medium. Background Art

[0002] In the field of medical information technology, core businesses such as electronic health records (EHRs), prescription systems, and medical image storage rely heavily on database systems to efficiently manage massive amounts of medical data. Medical data is typically multimodal, encompassing structured diagnostic records, semi-structured medical history text, unstructured images (such as CT / MRI), and test reports. Its storage and access place stringent demands on database performance, consistency, and resource utilization efficiency. Existing medical database systems utilize a dual-caching mechanism between the database engine and the operating system—where the database engine maintains its own cache (such as the buffer cache) in memory, while the operating system maintains a file system cache in kernel space—leading to multiple technical bottlenecks.

[0003] Cached data between the database engine and the operating system may be inconsistent, especially in high-concurrency scenarios. Frequent data writes and reads increase cache synchronization overhead, impacting system performance. For example, in scenarios with high-frequency electronic medical record updates (such as real-time data writes for emergency patients), synchronization delays between the two caches can cause terminals in different departments to read inconsistent medical records, threatening the accuracy and reliability of medical data and increasing data consistency risks.

[0004] Data transferred from disk to the database engine must pass through the operating system cache, resulting in long data transfer paths and high latency. This is especially true when processing large amounts of data. Multiple data copies and context switches significantly reduce system throughput. For example, transferring a single CT image (typically hundreds of MB) from a storage device to a database engine requires multiple copies within the operating system cache. When images are accessed concurrently by a PACS system, context switches and memory copies significantly increase response times, impacting physician diagnostic efficiency and leading to inefficient large-scale data transfer.

[0005] The database engine and operating system each maintain their own cache, resulting in duplicate memory resource usage. This resource competition, especially in environments with limited memory resources, further exacerbates performance bottlenecks. For example, lightweight database servers in primary healthcare institutions often face memory capacity constraints, while the database engine and operating system each maintain independent cache systems, resulting in duplicate storage of medical data in the database buffer cache and the file system cache. For example, a patient's test report data must reside in both the database engine's buffer and the operating system's file cache. The duplicate usage of memory resources further compresses the available memory space for medical applications such as image processing and data analysis, exacerbating system performance bottlenecks in memory-constrained environments and causing data retrieval and business processing to stall.

[0006] In high-concurrency scenarios, the double-buffer mechanism can cause data transmission between the database engine and the operating system to become a performance bottleneck. This is especially true in scenarios where frequent data reads and writes are required, significantly impacting system response time and throughput. For example, in a cross-institutional data sharing scenario within a regional medical platform, when thousands of terminals simultaneously access resident health records, the double-buffered data transmission link becomes a throughput bottleneck, causing significant system response latency and creating a performance bottleneck for high-concurrency access.

[0007] Although some medical databases have attempted to optimize their caching strategies, they have not fundamentally addressed the inherent flaws of the double-caching mechanism. Especially given the core requirements of medical data for "high concurrency, high reliability, and low latency," the performance and resource conflicts of existing technologies have become increasingly prominent. Summary of the Invention

[0008] The present invention provides a database double-buffer optimization method, device, computer equipment and medium to solve the technical problems of performance bottleneck and resource waste caused by double-buffer.

[0009] In a first aspect, a database double-buffer optimization method is provided, which is applied to a database and an operating system, and the method includes:

[0010] receiving configuration parameters input by a user for triggering a direct I / O mode, and configuring the database according to the configuration parameters, wherein the configuration parameters include a plurality of different file types;

[0011] In response to the configuration of the database taking effect, aligning the heap memory, the stack memory, and the shared memory of the memory buffer of the database to an integer multiple of a predefined block size;

[0012] If it is detected that the file type of the received data to be processed belongs to one of the file types in the configuration parameters, the direct I / O mode is triggered, and the data to be processed is directly processed between the aligned memory buffer and the storage device of the operating system.

[0013] In a second aspect, a database double-cache optimization device is provided, comprising a unit for executing the above method.

[0014] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0015] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0016] In the solution implemented by the above-mentioned database double-cache optimization method, device, computer equipment and medium, the database is configured according to the configuration parameters received from the user for triggering the direct I / O mode, wherein the configuration parameters include multiple different file types; in response to the database configuration taking effect, the heap memory, stack memory and shared memory of the database's memory buffer are aligned to integer multiples of a predefined block size; if it is detected that the file type of the received data to be processed belongs to one of the file types in the configuration parameters, the direct I / O mode is triggered, and the data to be processed is directly processed between the aligned memory buffer and the storage device of the operating system, so that the database engine can directly interact with the storage device for data, bypassing the operating system cache, reducing the intermediate links in the data transmission path, reducing data transmission delay, and significantly reducing the occupation of memory resources, thereby improving the resource utilization of the system, while reducing data redundancy and ensuring data consistency, significantly improving the data processing efficiency of the medical system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 1 is a flow chart of a method for optimizing database double cache according to an embodiment of the present invention;

[0019] Figure 2 yes Figure 1 Schematic diagram of the flow of sub-steps of step S110;

[0020] Figure 3 yes Figure 1 Schematic diagram of the flow of sub-steps of step S120;

[0021] Figure 4 yes Figure 1 Another flowchart of the sub-steps of step S120;

[0022] Figure 5 yes Figure 1 Another flowchart of the sub-steps of step S120;

[0023] Figure 6 yes Figure 1 Schematic diagram of the flow of sub-steps of step S130;

[0024] Figure 7 yes Figure 1 Another flowchart of the sub-steps of step S130;

[0025] Figure 8 It is a schematic block diagram of a database double-cache optimization device according to an embodiment of the present invention;

[0026] Figure 9 is a structural diagram of a computer device according to an embodiment of the present invention;

[0027] Figure 10 FIG. 2 is another structural diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0029] The database double-buffer optimization method provided in the embodiment of the present invention can be applied to a client or a server. The client can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server can be implemented with an independent server or a server cluster consisting of multiple servers. Both the client and the server include a database and an operating system, and the database double-buffer optimization method of the embodiment of the present invention can be applied to the database and the operating system. Compared with the existing double-buffer mechanism of the database and the operating system, the present invention can bypass the operating system cache through the direct I / O mode (Direct I / O) and directly establish an efficient data transmission channel between the database engine and the storage device, thereby solving the above-mentioned problems caused by the double-buffer mechanism. The present invention is described in detail below through specific embodiments.

[0030] See also Figure 1 As shown, Figure 1A flowchart of a database double-cache optimization method provided by an embodiment of the present invention includes the following steps: S110-S130.

[0031] S110, receiving configuration parameters input by a user for triggering a direct I / O mode, and configuring the database according to the configuration parameters, wherein the configuration parameters include a plurality of different file types;

[0032] In this embodiment, the direct I / O mode (Direct I / O) refers to a technical mode in which the database engine bypasses the file system cache layer of the operating system and directly interacts with the disk through the storage device driver to perform data interaction. The configuration parameter is a database global operation parameter (GUC parameter), and its data type is in the form of a string, which is used to specify the data file type that enables the direct I / O mode. The database refers to a structured data management system that supports medical data management, including but not limited to medical data storage systems such as patient electronic health records, imaging examination reports and prescription records. The file types include master data files (core table data that stores patient medical records), write-ahead log files (WAL, recording transaction operation logs) and log initialization files (WAL initialization allocation files), and their classification is based on the differentiated requirements of different data for real-time and consistency in medical data processing scenarios. Since the medical system needs to handle massive examination report generation and concurrent prescription update operations at the same time during peak hours, the data synchronization delay between the operating system and the database cache under the traditional double buffering mechanism may cause critical value report reading lags or prescription version conflicts. Therefore, it is necessary to selectively enable the direct I / O mode through dynamic configuration parameters to optimize the critical data path. In specific implementation, the database administrator sets the configuration parameter by modifying the database configuration file (postgresql.conf) or executing the SQL command (ALTER SYSTEM SET io_direct = 'data, wal'). When the parameter value is an empty string, the direct I / O mode is disabled by default and the traditional caching mechanism is adopted. When the parameter value is comma-separated data, wal, or wal_init, the database startup phase parses the parameter and verifies whether the current operating system supports the direct I / O function for the corresponding file type. If the verification fails, an illegal parameter exception is thrown and the startup process is terminated. For example, when processing DICOM files of CT images in a tertiary hospital's PACS system, the administrator sets the io_direct parameter to "data,wal," enabling direct I / O for the database's master data file (which stores image path indexes) and write-ahead log files (which record image access transactions). When radiologists retrieve historical patient images, the database bypasses the operating system cache and reads the index data directly from the storage array. Atomic commits in the write-ahead log ensure the write consistency of concurrent diagnostic reports across multiple terminals. However, if io_direct is incorrectly configured as "data,wal,invalid" and the system doesn't support the invalid type, an "ERROR: unsupported IO direct option 'invalid'" error message is generated, and the original cache mode is maintained. This step enables configurable direct I / O triggering for critical data file types in medical scenarios, laying the foundation for subsequent policy execution for memory alignment and data transfer optimization.

[0033] In one embodiment, if Figure 2 As shown, the step S110 includes: S111-S116.

[0034] S111. Receive a configuration parameter of a string type input by a user, wherein the configuration parameter is an empty string or contains at least one file type identifier separated by commas;

[0035] S112. When the configuration parameter is an empty string, disable the direct I / O mode and retain the operating system cache function;

[0036] S113. When the configuration parameter includes a file type identifier, parse the file type identifier to generate a set of file types to be verified;

[0037] S114, traversing each file type identifier in the set of file types to be verified, and detecting whether the direct I / O function corresponding to the file type identifier is supported by the current operating system;

[0038] S115. If an unsupported file type identifier is detected, abort database configuration and generate a configuration failure response including an error code;

[0039] S116: If all file type identifiers in the set of file types to be verified are supported, write the configuration parameters into the database configuration table and restart the database service process to make the configuration effective.

[0040] In this embodiment, the configuration parameter is received in the form of a string, and its input format is an empty string or a comma-separated combination of file type identifiers, where an empty string indicates that the direct I / O mode is disabled, and an identifier combination such as "data,wal" indicates that direct I / O is enabled for the primary data file and the write-ahead log file; when the database administrator enters SET When io_direct = 'data, wal' (S111), the database configuration engine first determines whether the parameter value is an empty string. If so, the direct I / O function is disabled and the data transfer path cached by the operating system is maintained (S112). Otherwise, the parameter is parsed into a set of file type identifiers (e.g., ["data", "wal"]) using comma separators, duplicates are removed, and the file is converted into a case-insensitive standardized format to generate a set of file types to be verified (S113). The database configuration engine then traverses each identifier in the set and attempts to open a temporary file with the O_DIRECT flag by calling the fcntl function. If the EINVAL error code is returned, it is determined that the current operating system kernel or file system does not support the direct I / O function corresponding to the identifier. For example, when configuring the wal_init identifier in an older version of the Linux kernel, a compatibility exception is triggered due to the lack of support for write-ahead log initialization (S114). If any identifier fails to be verified (e.g., an invalid identifier "invalid" is mistakenly configured), the database configuration process is immediately terminated and an error response ERROR: Unsupported IO direct option is returned to the administrator.

[0041] 'invalid' (SQLSTATE: 22023), while retaining the original configuration to prevent interruption of medical transaction log writing (S115); if all identifiers pass the verification, the standardized parameter values are written to the io_direct entry of the database global configuration table pg_settings, and a SIGTERM signal is sent to the database service process to trigger a restart, so that after the radiology PACS system reloads the configuration, the reading operation of the CT image index file (data type) bypasses the operating system cache and directly connects to the storage array, while ensuring that the writing path of the diagnostic report transaction log (wal type) is decoupled from the kernel cache (S116). For example, when the emergency department database is configured with io_direct = 'data,wal_init', if the current file system does not support direct I / O initialized by the write-ahead log corresponding to the wal_init identifier, the system outputs ERROR: Unsupported IO directoption 'wal_init' and refuses to restart, thus preventing the loss of patient emergency records due to configuration errors. If the verification passes, the restarted database enables direct I / O for DICOM image master data, allowing the GPU-accelerated image analysis module to bypass the cache and directly access the raw data blocks in the storage device, significantly shortening the image loading time for critical cases.

[0042] S120: In response to the configuration of the database taking effect, aligning the heap memory, stack memory, and shared memory of the memory buffer of the database to an integer multiple of a predefined block size;

[0043] In this embodiment, the memory buffer includes three storage areas: heap memory, stack memory, and shared memory. Heap memory is a dynamically allocated unstructured data space (such as a cache of real-time patient monitoring stream data), stack memory is a structured data space automatically allocated when a function is called (such as thread context information for medical transactions), and shared memory is a data interaction area shared by multiple processes (such as a hospital-wide electronic medical record index called across departments). Block size is the smallest logical unit of data reading and writing in a computer storage system. The block size can be defined as 4096 bytes, corresponding to the physical sector size of the storage device and the general alignment benchmark of the operating system's memory page. Its integer multiple alignment requirement is a hardware-enforced constraint for data interaction in direct I / O mode. Since large-scale data such as DICOM files of CT images in medical scenarios need to be strictly aligned for storage to match the DMA access rules of the GPU acceleration processor, if the memory buffer is not aligned according to the block size, the direct I / O operation will trigger an EINVAL system error due to address misalignment, resulting in image loading failure or delayed diagnostic report generation. Therefore, the memory space needs to be uniformly aligned after the database configuration takes effect. Specifically, heap memory uses memory allocation functions (such as ralloc_aligned) to enforce allocation of memory blocks with starting addresses that are multiples of 4096, while inserting offset metadata to track the original address for backtracking during release. Stack memory ensures that transaction log blocks are always stored starting from aligned addresses by embedding compiler alignment attribute macros (such as __attribute__((aligned(4096)))) in the log buffer structure declaration. Shared memory calls address alignment functions (such as TYPEALIGN) during initialization to adjust the buffer pool starting address to the nearest 4096 multiple boundary and allocates additional redundant space to compensate for offset loss. For example, in the image preprocessing module of the radiology department's PACS system, when direct I / O is enabled, the database aligns the heap memory blocks of the CT image index cache at 4096 bytes, allowing the GPU to read data directly from the aligned address via DMA, avoiding transmission interruptions due to memory fragmentation. At the same time, when multiple terminals in the emergency department concurrently write emergency records, the forced alignment of shared memory eliminates cache line contention in inter-process communication, ensuring that real-time vital signs such as blood oxygen and heart rate are free of bit offset errors when accessed across departments. This step eliminates the risk of memory access anomalies in direct I / O mode, optimizes the transmission path of medical data into a continuous sequence of aligned blocks, and provides underlying memory layout guarantees for subsequent efficient interactions across the operating system cache, reducing the reading latency of large-scale image files and completely avoiding the problem of emergency record writing failures due to memory misalignment.

[0044] In one embodiment, if Figure 3 As shown, the step S120 includes: S121-S123.

[0045] S121, calling a memory allocation function to allocate target heap memory space, where an input parameter of the memory allocation function includes an alignment parameter, and the alignment parameter is a power of 2;

[0046] S122. Writing a redirection metadata block at the head of the target heap memory space, where the redirection metadata block is used to record the offset of the original heap memory allocation address relative to the aligned address;

[0047] S123: When releasing the target heap memory space, parse the redirection metadata block to obtain the original heap memory allocation address, and perform a memory release operation based on the original heap memory allocation address.

[0048] Specifically, the memory allocation function ralloc_aligned() or MemoryContextAllocAligned() is called to allocate the target heap memory space, and the input parameter alignto of the function specifies the alignment reference value (i.e., the alignment parameter), whose value range is an integer power of 2 (such as 4096, 8192) and the maximum value does not exceed 2^26, so as to meet the alignment constraints of GPU video memory and storage device sectors in medical image processing; during the allocation process, the actual memory requirement is calculated as the requested size + alignto + sizeof(MemoryChunk), and the returned pointer is adjusted to the nearest integer multiple boundary of alignto through the address offset operation (for example, the requested address 0x1001 returns 0x2000 after alignment), and the redirection metadata block (MemoryChunk) is written to the header of the adjusted memory block. The metadata block stores the offset between the original allocation address and the alignment address by rewriting the block_offset field (such as 0x1001 → offset 0x0FFF), and its structure is defined as typedef struct{size_t offset; size_t magic;}MemoryChunk, where the magic field is written with a fixed magic number 0x12345678 for integrity verification during release; when releasing the aligned heap memory, call the rfree() function and pass in the alignment address pointer, use the pointer to rollback sizeof(MemoryChunk) to locate the metadata block, parse the offset field to calculate the original allocation address (such as alignment address 0x2000 - offset 0x0FFF = original address 0x1001), if an exception in the magic field is detected or the address is out of bounds, the assertion error FATAL: invalid alignedpointer is triggered, and finally the standard memory release interface is called to release the memory block corresponding to the original address. For example, when allocating a 256KB DICOM image buffer in a real-time ultrasound image processing system, ralloc_aligned(256*1024,4096) is called with alignto=4096, returning the aligned address 0x7FABCD2000. Its header metadata block records the original address 0x7FABCD1000 and offset 0x1000. When image analysis is completed and the memory is released, the metadata is read back 16 bytes from 0x7FABCD2000 (assuming the MemoryChunk size is 16B), the magic number is verified, and the original address 0x7FABCD1000 is finally released. If the radiology workstation mistakenly releases the aligned address 0x7FABCD2000 directly, the magic number field verification failure triggers the error log ERROR: corrupted memory chunk detected, preventing the risk of residual patient image data due to heap memory leaks.This step ensures that the physical addresses of the medical data buffer in direct I / O mode are strictly aligned, while being compatible with the existing database memory management mechanism. This enables the high-speed data acquisition modules of medical equipment such as electrocardiographs to directly access the aligned memory blocks through DMA without software transfer.

[0049] In one embodiment, if Figure 4 As shown, the step S120 includes: S124-S126.

[0050] S124, defining an alignment reference value of the stack memory as 4096 bytes, where the alignment reference value is an integral multiple of the predefined block size;

[0051] S125. Inserting a compiler alignment attribute macro into a predefined log block union structure declaration so that the starting address of each instantiated object of the log block union structure in the memory is automatically aligned according to the stack memory alignment reference value;

[0052] S126 . When creating a transaction log buffer block, map the memory address of the instantiated object of the log block union structure to a stack memory area that satisfies alignment to an integer multiple of the block size.

[0053] In this embodiment, in the stack memory alignment operation, the stack memory alignment reference value is defined as RASE_IO_ALIGN_SIZE 4096, which is an integer multiple of the direct I / O mode block size and satisfies the power of 2 constraint; the compiler alignment macro RASE_ATTRIBUTE_ALIGNED (RASE_IO_ALIGN_SIZE) is inserted into the declaration of the predefined log block union structure RaseAlignedXLogBlock, and the union structure is used to store the atomic transaction record of the write-ahead log (WAL). Its declaration code is typedef union RaseAlignedXLogBlock{char data[RASE_IO_ALIGN_SIZE]; XLogRecord record;} RASE_ATTRIBUTE_ALIGNED(RASE_IO_ALIGNED N_SIZE); so that the memory address of each instantiated object is automatically aligned to a 4096-byte boundary; when the database transaction subsystem creates a log buffer block, it calls the stack memory allocation function palloc() and adds the alignment constraint MemoryContextAllocAligned(MCXT_SKIP_ALIGN, RASE_IO_ALIGN_SIZE) to force the RaseAlignedXLogBlock instance to be mapped to a stack memory area with a starting address that is a multiple of 4096 (such as 0x7FFE2800). For example, when the outpatient registration system processes concurrent registration requests from multiple windows, it calls the MemoryContextAllocAligned() function to allocate aligned addresses (such as 0x5000, 0x6000) in the stack memory to ensure that each registration transaction log block occupies an independent 4KB area. For example, window A writes the "Cardiology" registration record of patient 1001 to address 0x5000, and window B writes the "Pediatrics" record of patient 1002 to 0x6000. There is no memory overlap and lock contention between threads, which improves the concurrent registration processing capability. If the system loses power when writing the registration record of patient 1003 to address 0x7008, the recovery process detects that the log block at address 0x7000 is complete based on the 4KB boundary (the record "Ophthalmology" has been completely written), and some data after address 0x7008 is discarded, ensuring that the information of all registered departments is complete and conflict-free, shortening the system recovery time and avoiding the clinical risk of "incomplete registration department information". Through this step, the stack memory alignment mechanism reduces log conflicts in high-concurrency medical transactions and shortens recovery time after system crashes, significantly improving medication safety and system reliability.

[0054] In one embodiment, if Figure 5 As shown, the step S120 includes: S127-S129.

[0055] S127. When initializing the shared memory pool during the database startup phase, calling an address alignment function to perform a forced alignment operation on the shared memory start address of the buffer block, wherein the address alignment function adjusts the start address to a value that is closest to and not less than an integer multiple of the block size of the original address;

[0056] S128. Allocate additional memory space for each buffer block, where the length of the additional memory space is equal to the block size value, and is used to compensate for the address offset caused by forced alignment;

[0057] S129. Recalculate the physical page mapping table of the buffer block based on the aligned shared memory start address to ensure that the database engine directly hits the aligned memory area when accessing the buffer block.

[0058] In this embodiment, the "shared memory pool" is a memory area shared by multiple processes or threads in the database system, used to store frequently accessed data to improve access efficiency; the "address alignment function" is a tool function (such as TYPEALIGN) that aligns memory addresses according to a specific block size, and its function is to adjust the memory address to an integer multiple of the block size; the "buffer block" is the basic unit for caching data in the shared memory pool; the "physical page mapping table" is used to record the actual address mapping relationship of the buffer block in the physical memory, ensuring that the database engine can correctly locate the data. Specifically, when initializing the shared memory pool during database startup, an address alignment function (such as TYPEALIGN(RASE_IO_ALIGN_SIZE,...)) is first called to force alignment of the original shared memory start address of the buffer block. This function adjusts the start address to a value greater than or equal to the original address and the nearest integer multiple of the block size (such as 4096 bytes as defined by RASE_IO_ALIGN_SIZE) to meet the strict memory address alignment requirements of the direct I / O mode. Because forced alignment causes the actual allocated memory start address to shift backward, additional memory space equal to the block size must be allocated for each buffer block to compensate for the address offset and prevent data storage from being lost or misplaced due to address offset. Finally, the physical page mapping table of the buffer block is recalculated based on the aligned shared memory start address. By updating the mapping relationship, the database engine ensures that when accessing the buffer block, it can directly hit the memory area aligned according to the block size, avoiding I / O errors or performance loss caused by address misalignment. Taking the medical scenario as an example, the database of a regional medical platform needs to process the electronic medical records and imaging data of tens of millions of patients. When initializing the shared memory pool, a shared memory alignment operation is performed on the buffer block storing the CT image data block: the original starting address is 0x10000123, and the address alignment function is adjusted to the value of 0x10001000, which is the closest integer multiple of 4096 bytes of the address and not less than the address. At the same time, 4096 bytes of additional memory space are allocated for each 4096-byte image data block to compensate for the offset. The physical page mapping table is then recalculated so that the engine can directly locate the aligned image data block when accessing the address 0x10001000. When multiple medical institutions concurrently review chest CT images of the same patient, the aligned shared memory area supports the database engine to read and cache image data directly from the storage device in 4KB blocks, avoiding cross-block read delays caused by memory address misalignment. This increases the concurrent processing capability of image review from 500 times per second to 1,800 times, eliminates image data fragmentation errors caused by address offsets, and ensures the integrity and access efficiency of medical image data.This step achieves precise alignment and efficient management of the shared memory pool, significantly improving the hit rate and access speed of the data cache in high-concurrency scenarios such as medical applications. It also ensures the legitimacy of memory addresses and the accuracy of data access in direct I / O mode, providing technical support for the stable and efficient processing of large-scale medical data.

[0059] S130: If it is detected that the file type of the received data to be processed belongs to one of the file types in the configuration parameters, the direct I / O mode is triggered, and the data to be processed is directly processed between the aligned memory buffer and the storage device of the operating system.

[0060] In this embodiment, the data to be processed refers to data received by the database system that requires read and write operations, including but not limited to electronic medical records, medical imaging files, and test reports; storage devices are hardware devices used to persistently store data, such as mechanical hard drives and solid-state drives (SSDs). When the user specifies that direct I / O mode is enabled for a specific file type through configuration parameters, the database needs to bypass the operating system cache for eligible data files and directly interact with the storage device using an aligned memory buffer to address the redundancy and efficiency issues of the traditional double-buffer mechanism. Specifically, the database system first analyzes the file type of the data to be processed (such as a main data file or a WAL file) to determine whether it belongs to the set of file types that enable direct I / O specified in the configuration parameter io_direct; if a match is successful, direct I / O mode is triggered, calling the underlying driver interface to bypass the operating system file system cache and interact with the storage device through an aligned heap memory, stack memory, or shared memory buffer. When reading data, data is directly loaded from the storage device into the aligned buffer according to the integer multiple of the block size (such as 4096 bytes); when writing data, the data is first stored in the aligned buffer and then directly written to the storage device according to the block alignment requirements. Taking the medical scenario as an example, when the electronic medical record system of a tertiary hospital processes patient image archiving business, the data to be processed is MRI image files in DICOM format (stored in the main data file), and its file type has been set to enable direct I / O mode through the configuration parameter io_direct. When the radiologist reviews the patient's historical MRI images, the database detects a file type match and triggers the direct I / O mode. The image data block is read directly from the high-speed storage device through the aligned shared memory buffer (the starting address is an integer multiple of 4096 bytes), avoiding the repeated copying and synchronization overhead of data between the operating system cache and the database cache. Compared with the traditional mode, the image loading time is improved, and when processing a large number of concurrent review requests at the same time, the system throughput is improved, avoiding the problem of image data misalignment or loading failure due to cache inconsistency. This step enables efficient processing of data of user-specified file types. The direct I / O mode and memory alignment technology shorten the data transmission path, reduce cache synchronization overhead and resource waste, and significantly improve data access speed and system stability in high-concurrency access scenarios for large-scale imaging data and electronic medical records in the medical field, providing key technical guarantees for the real-time and reliability of medical services.

[0061] In one embodiment, if Figure 6 As shown, the step S130 includes: S131-S134.

[0062] S131. Verify whether the starting address of the memory buffer corresponding to the data to be processed, the file offset in the storage device, and the data length meet the constraint of an integer multiple of a predefined block size;

[0063] S132: When the starting address of the memory buffer does not satisfy the integer multiple constraint, fill blank data at the target position of the memory buffer to the next alignment boundary to generate an aligned data block;

[0064] S133. When the file offset does not satisfy the integer multiple constraint, truncating the read and write range of the data to be processed according to the predefined block size boundary;

[0065] S134: Transmit the processed aligned data block or the truncated data block between the memory buffer and the storage device through a direct I / O interface, and bypass the file system cache of the operating system.

[0066] In this embodiment, the starting address refers to the address of the starting position in the memory buffer used to store the data to be processed; the file offset in the storage device is the number of bytes offset from the beginning of the file in the storage device file to the starting position of the data to be processed; the data length is the number of bytes occupied by the data to be processed; the predefined block size is a fixed number of bytes set in advance, usually an integer power of 2, such as 4096 bytes, which is used to standardize the basic unit of data transmission between the memory and the storage device; the aligned data block is the data block formed by aligning the data in the memory buffer according to the predefined block size; the truncated data block is the data block formed by truncating the read and write range of the data to be processed according to the predefined block size boundary; the direct I / O interface is an interface that allows programs to bypass the operating system file system cache and directly interact with the storage device for data. Because the direct I / O mode has strict alignment requirements for the starting address, file offset, and data length of the data, failure to meet these requirements can result in data transmission errors or performance degradation. Specifically, the database system checks the starting address of the memory buffer, the file offset in the storage device, and the data length to determine whether they are integer multiples of the predefined block size. If the starting address of the memory buffer does not meet the integer multiple constraint, blank data is padded at appropriate locations in the memory buffer until the next alignment boundary is reached, thereby generating an aligned data block. If the file offset does not meet the integer multiple constraint, the read / write range of the data to be processed is truncated according to the predefined block size boundary. Finally, the processed aligned data block or truncated data block is transferred between the memory buffer and the storage device using a direct I / O interface, bypassing the operating system's file system cache. For example, a hospital's database system needs to process a batch of CT imaging data for a group of patients. This data is stored in a file on the storage device, and the memory buffer is ready to receive the data. Before the data is transferred, the system detects that the starting address of the memory buffer is 0x10001, the file offset is 2048 bytes, and the data length is 3000 bytes, while the predefined block size is 4096 bytes. Neither the starting address nor the file offset meets the integer multiple constraint. Therefore, the system padded the starting position of the memory buffer with 2047 bytes of blank data, bringing the starting address to 0x10008 (which meets the 4096-byte alignment), generating an aligned data block. The data length is then truncated to 2048 bytes (an integral multiple of 4096). Finally, the truncated data blocks are transferred directly from the storage device to the aligned memory buffer via the direct I / O interface, bypassing the operating system's file system cache, shortening data transfer time, and avoiding data transmission errors.This step ensures the accuracy and efficiency of data transmission in direct I / O mode, avoiding errors and performance loss caused by data misalignment. Especially in medical data processing scenarios, it can quickly and accurately process large amounts of critical data, improving the overall performance and stability of the system and providing strong guarantees for the smooth development of medical services.

[0067] In one embodiment, if Figure 7 As shown, the step S130 includes: S135-S137.

[0068] S134, calling an atomic write operation interface to write transaction log data in the data to be processed into the storage device, where the file types include a primary data file type, a write-ahead log file type, and a log initialization file type;

[0069] S135: Real-time monitoring during the transaction log data writing process, and if an abnormal write interruption is detected, rolling back the incompletely written log data segment to the nearest valid alignment boundary based on the boundary of an integer multiple of the predefined block size;

[0070] S136. Record the boundary position and data segment identifier of the rollback operation in the transaction recovery log, so that the database engine can reconstruct the transaction state according to the record of the rollback operation.

[0071] In this embodiment, the atomic write operation interface refers to an instruction provided by the storage device to complete the writing of a data block in a single I / O operation (such as Linux's O_DIRECT combined with the write() system call); the nearest valid alignment boundary refers to the boundary position closest to the incompletely written data segment within a predefined integer multiple of the block size, where the data has been completely and correctly written; the "data segment identifier" is information used to uniquely identify a segment of transaction log data, such as the data segment number and starting address, to facilitate the database engine's accurate identification and management of different data segments. Transaction log data records detailed information about database transaction operations and is used to ensure the atomicity and durability of transactions; the primary data file type is used to store core database data, such as basic patient information and medical records in medical scenarios; the write-ahead log file type records changes to database transactions and is written to the log before the transaction is committed to ensure data recoverability; and the log initialization file type contains log information required for database startup and initialization. In database operations, accurate transaction log recording is crucial for maintaining data consistency and recovering transaction status. If the write process is abnormally interrupted, data inconsistency may occur, affecting the normal operation of the database. Therefore, a reliable write and recovery mechanism is required. Specifically, when transaction log data needs to be processed, an atomic write operation interface is called to write the transaction log data in the pending data to the storage device. This interface ensures the atomicity of data writing. Then, in step S135, real-time monitoring is performed during the transaction log data writing process. Once a write interruption is detected, the incompletely written log data segment is rolled back to the nearest valid alignment boundary based on a predefined integer multiple of the block size to ensure data integrity. Finally, the boundary position and data segment identifier of the rollback operation are recorded in the transaction recovery log. This allows the database engine to reconstruct the transaction state based on these records and ensure transaction consistency. Taking a medical scenario as an example, a hospital's database system processes patient medical record update transactions, involving operations on the master data file, pre-write log file, and log initialization file. During the patient medical record update process, the system calls the atomic write operation interface to write the transaction log data to the storage device. However, during the write process, a sudden power failure causes the write interruption. After the system detects the anomaly, it rolls back the incompletely written log data segment to the nearest valid alignment boundary based on a predefined integer multiple of the block size (e.g., 4096 bytes). If log data writing is interrupted halfway through the third block, the system will roll back the incomplete portion to the end of the second block (i.e., the nearest valid alignment boundary). Simultaneously, the rollback boundary position (the end of the second block) and the data segment identifier (e.g., the transaction log data segment number) are recorded in the transaction recovery log. Upon system restart, the database engine reconstructs the transaction state based on the transaction recovery log, ensuring the consistency of the patient record update transaction and preventing data loss or errors.This step enables reliable writing and exception recovery of database transaction log data, ensuring that the database can quickly recover to a consistent state when an abnormal situation occurs, ensuring the integrity of medical data and the correctness of transactions, and providing a solid guarantee for the stable operation of medical services.

[0072] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0073] The embodiment of the present invention further provides a database double cache optimization device, which corresponds to the database double cache optimization method in the above embodiment. Figure 8 As shown, the database double cache optimization device includes: a configuration unit 201, an alignment unit 202 and a processing unit 203. The functional units are described in detail as follows:

[0074] A configuration unit 201 is configured to receive configuration parameters input by a user for triggering a direct I / O mode, and configure the database according to the configuration parameters, wherein the configuration parameters include a plurality of different file types;

[0075] an alignment unit 202 for aligning heap memory, stack memory, and shared memory of a memory buffer of the database to an integer multiple of a predefined block size in response to the configuration of the database taking effect;

[0076] The processing unit 203 is used to trigger the direct I / O mode if it is detected that the file type of the received data to be processed belongs to one of the file types in the configuration parameters, and directly process the data to be processed between the aligned memory buffer and the storage device of the operating system.

[0077] In one embodiment, the configuration unit 201 is specifically configured to:

[0078] Receive a configuration parameter of a string type input by a user, wherein the configuration parameter is an empty string or contains at least one file type identifier separated by commas;

[0079] When the configuration parameter is an empty string, direct I / O mode is disabled and the operating system cache function is retained;

[0080] When the configuration parameter includes a file type identifier, parsing the file type identifier to generate a set of file types to be verified;

[0081] Traversing each file type identifier in the set of file types to be verified, and detecting whether the direct I / O function corresponding to the file type identifier is supported by the current operating system;

[0082] If an unsupported file type identifier is detected, aborting the database configuration and generating a configuration failure response including an error code;

[0083] If all file type identifiers in the set of file types to be verified are supported, the configuration parameters are written into the database configuration table and the database service process is restarted to make the configuration effective.

[0084] In one embodiment, the alignment unit 202 is specifically configured to:

[0085] Calling a memory allocation function to allocate target heap memory space, wherein an input parameter of the memory allocation function includes an alignment parameter, and the alignment parameter is a power of 2;

[0086] Writing a redirection metadata block at the head of the target heap memory space, wherein the redirection metadata block is used to record the offset of the original heap memory allocation address relative to the aligned address;

[0087] When releasing the target heap memory space, the redirection metadata block is parsed to obtain the original heap memory allocation address, and a memory release operation is performed based on the original heap memory allocation address.

[0088] Defining an alignment reference value of the stack memory to be 4096 bytes, wherein the alignment reference value is an integer multiple of the predefined block size;

[0089] Inserting a compiler alignment attribute macro into a predefined log block union structure declaration so that a starting address of each instantiated object of the log block union structure in memory is automatically aligned according to the stack memory alignment reference value;

[0090] When creating a transaction log buffer block, the memory address of the instantiated object of the log block union structure is mapped to a stack memory area that satisfies the alignment of an integer multiple of the block size.

[0091] When the shared memory pool is initialized during the database startup phase, an address alignment function is called to perform a forced alignment operation on the shared memory start address of the buffer block. The address alignment function adjusts the start address to a value that is closest to and not less than an integer multiple of the block size of the original address.

[0092] Allocate additional memory space for each buffer block, where the length of the additional memory space is equal to the block size value, and is used to compensate for the address offset caused by forced alignment;

[0093] The physical page mapping table of the buffer block is recalculated based on the aligned shared memory start address to ensure that the database engine directly hits the aligned memory area when accessing the buffer block.

[0094] In one embodiment, the processing unit 203 is specifically configured to:

[0095] Verify whether the starting address of the memory buffer corresponding to the data to be processed, the file offset in the storage device, and the data length meet the constraint of an integer multiple of a predefined block size;

[0096] When the starting address of the memory buffer does not satisfy the integer multiple constraint, filling blank data at the target position of the memory buffer to the next alignment boundary to generate an aligned data block;

[0097] When the file offset does not satisfy the integer multiple constraint, truncating the read and write range of the data to be processed according to the predefined block size boundary;

[0098] The processed aligned data blocks or truncated data blocks are transferred between the memory buffer and the storage device via a direct I / O interface, bypassing a file system cache of the operating system.

[0099] Calling an atomic write operation interface to write transaction log data in the data to be processed into the storage device, wherein the file types include a primary data file type, a write-ahead log file type, and a log initialization file type;

[0100] Real-time monitoring is performed during the transaction log data writing process, and if an abnormal write interruption is detected, the incompletely written log data segment is rolled back to the nearest valid alignment boundary based on the boundary of an integer multiple of the predefined block size;

[0101] The boundary position and data segment identifier of the rollback operation are recorded in the transaction recovery log, so that the database engine can reconstruct the transaction state according to the record of the rollback operation.

[0102] The present invention provides a database dual-cache optimization device. Through DirectIO technology, the database engine can directly interact with the storage device for data, thereby significantly reducing the intermediate links in the data transmission path and reducing data transmission delay. Experimental data shows that after adopting DirectIO technology, the system throughput is improved and the delay is reduced. By bypassing the operating system cache, the database engine can significantly reduce the occupation of memory resources, thereby improving the resource utilization of the system. This optimization effect will be more significant in an environment with limited memory resources. Through DirectIO technology, the database engine can achieve direct control of the storage device, thereby avoiding inconsistency problems with the operating system cache. This method can significantly improve the stability and reliability of the system.

[0103] The specific limitations of the database dual-buffer optimization device can be found in the limitations of the database dual-buffer optimization method described above and will not be further elaborated here. Each module in the aforementioned database dual-buffer optimization device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the aforementioned modules can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in the computer device in software form, so that the processor can call and execute the corresponding operations of each of the aforementioned modules.

[0104] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 9 As shown. The computer device includes a processor, memory, network interface and database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a database double cache optimization method.

[0105] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 10 As shown. The computer device includes a processor, memory, network interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the client side of a database double cache optimization method.

[0106] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned database double cache optimization method are implemented.

[0107] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned database double-buffer optimization method are implemented.

[0108] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can be found in the relevant descriptions of the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0109] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0110] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0111] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A database double cache optimization method, characterized in that: Applied to databases and operating systems, the method includes: receiving configuration parameters input by a user for triggering a direct I / O mode, and configuring the database according to the configuration parameters, wherein the configuration parameters include a plurality of different file types; In response to the configuration of the database taking effect, aligning the heap memory, the stack memory, and the shared memory of the memory buffer of the database to an integer multiple of a predefined block size; If it is detected that the file type of the received data to be processed belongs to one of the file types in the configuration parameters, the direct I / O mode is triggered, and the data to be processed is directly processed between the aligned memory buffer and the storage device of the operating system.

2. The method according to claim 1, wherein The step of aligning the heap memory of the memory buffer of the database to an integer multiple of a predefined block size includes: Calling a memory allocation function to allocate target heap memory space, wherein an input parameter of the memory allocation function includes an alignment parameter, and the alignment parameter is a power of 2; Writing a redirection metadata block at the head of the target heap memory space, wherein the redirection metadata block is used to record the offset of the original heap memory allocation address relative to the aligned address; When releasing the target heap memory space, the redirection metadata block is parsed to obtain the original heap memory allocation address, and a memory release operation is performed based on the original heap memory allocation address.

3. The method according to claim 1, wherein The step of aligning the stack memory of the memory buffer of the database to an integer multiple of a predefined block size comprises: Defining an alignment reference value of the stack memory to be 4096 bytes, wherein the alignment reference value is an integer multiple of the predefined block size; Inserting a compiler alignment attribute macro into a predefined log block union structure declaration so that a starting address of each instantiated object of the log block union structure in memory is automatically aligned according to the stack memory alignment reference value; When creating a transaction log buffer block, the memory address of the instantiated object of the log block union structure is mapped to a stack memory area that satisfies the alignment of an integer multiple of the block size.

4. The method according to claim 1, wherein The step of aligning the shared memory of the memory buffer of the database to an integer multiple of a predefined block size comprises: When the shared memory pool is initialized during the database startup phase, an address alignment function is called to perform a forced alignment operation on the shared memory start address of the buffer block. The address alignment function adjusts the start address to a value that is closest to and not less than an integer multiple of the block size of the original address. Allocate additional memory space for each buffer block, where the length of the additional memory space is equal to the block size value, and is used to compensate for the address offset caused by forced alignment; The physical page mapping table of the buffer block is recalculated based on the aligned shared memory start address to ensure that the database engine directly hits the aligned memory area when accessing the buffer block.

5. The method according to claim 1, wherein The step of receiving configuration parameters input by a user for triggering the direct I / O mode, and configuring the database according to the configuration parameters, wherein the configuration parameters include a plurality of different file types, comprises: Receive a configuration parameter of a string type input by a user, wherein the configuration parameter is an empty string or contains at least one file type identifier separated by commas; When the configuration parameter is an empty string, direct I / O mode is disabled and the operating system cache function is retained; When the configuration parameter includes a file type identifier, parsing the file type identifier to generate a set of file types to be verified; Traversing each file type identifier in the set of file types to be verified, and detecting whether the direct I / O function corresponding to the file type identifier is supported by the current operating system; If an unsupported file type identifier is detected, aborting the database configuration and generating a configuration failure response including an error code; If all file type identifiers in the set of file types to be verified are supported, the configuration parameters are written into the database configuration table and the database service process is restarted to make the configuration effective.

6. The method according to claim 1, wherein The step of directly processing the to-be-processed data between the aligned memory buffer and the storage device of the operating system comprises: Verify whether the starting address of the memory buffer corresponding to the data to be processed, the file offset in the storage device, and the data length meet the constraint of an integer multiple of a predefined block size; When the starting address of the memory buffer does not satisfy the integer multiple constraint, filling blank data at the target position of the memory buffer to the next alignment boundary to generate an aligned data block; When the file offset does not satisfy the integer multiple constraint, truncating the read and write range of the data to be processed according to the predefined block size boundary; The processed aligned data blocks or truncated data blocks are transferred between the memory buffer and the storage device via a direct I / O interface, bypassing a file system cache of the operating system.

7. The method according to claim 6, wherein When the file type of the data to be processed is a write-ahead log file type, the step of transmitting the data block through the direct I / O interface includes: Calling an atomic write operation interface to write transaction log data in the data to be processed into the storage device, wherein the file types include a primary data file type, a write-ahead log file type, and a log initialization file type; Real-time monitoring is performed during the transaction log data writing process, and if an abnormal write interruption is detected, the incompletely written log data segment is rolled back to the nearest valid alignment boundary based on the boundary of an integer multiple of the predefined block size; The boundary position and data segment identifier of the rollback operation are recorded in the transaction recovery log, so that the database engine can reconstruct the transaction state according to the record of the rollback operation.

8. A database double cache optimization device, characterized in that: The method comprises a unit for executing the method according to any one of claims 1 to 7.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images