Disk data writing method, device, equipment, system and chip

Abstract

The invention provides a disk data writing method, device, equipment, system and chip, and the method comprises the steps: constructing a bitmap which is used for mapping a storage area of a disk and corresponds to a storage rear end; receiving a data write-in request of the virtual machine disk, and sending the data write-in request to the plurality of storage rear ends at the same time; performing data synchronization operation by utilizing the asynchronous data synchronization thread to obtain a data synchronization result so as to write data of a virtual machine disk to different storage rear ends at the same time; and detecting whether the data synchronization result is successful. According to the method, the disks of the virtual machine can be written to different storage rear ends at the same time, the fault tolerance of a system is improved, the data consistency of multiple storage rear ends is ensured, and the reliability degree of data synchronization is guaranteed.

Description

Technical Field

[0001] This application relates to the technical fields of computer software and disk data management, and particularly to a disk data writing method, apparatus, device, system, and chip. Background Art

[0002] With the rapid development of cloud computing technology, the virtual machine dual-active technology has become an important means to ensure the high availability of the system. The virtual machine dual-active requires that the disk data of the virtual machine can be written into different storage clusters simultaneously to ensure data redundancy and system reliability. However, in practical applications, this requirement faces many technical challenges, mainly concentrated in the fault tolerance ability and fault recovery ability in the fault scenario. Fault tolerance means that when a certain storage cluster cannot be written, the system can avoid blocking normal input / output (IO) operations, thus avoiding performance degradation; the fault recovery ability refers to how the system can quickly return to the normal state after a fault occurs and is eliminated.

[0003] Traditional disk writing methods usually rely on standard writing functions and have significant deficiencies in multi-storage-backend scenarios. First, when a certain storage backend fails, the writing operation may be blocked, resulting in a decline in overall performance or even system interruption. Second, after a fault occurs, traditional methods lack an effective automatic recovery mechanism and cannot quickly synchronize data to the faulty backend, affecting data consistency and system availability. In addition, existing technologies often have difficulty balancing performance and reliability when dealing with large-scale concurrent writes, especially in the case of heterogeneous storage backends or unstable networks. Therefore, how to map the data at the pipe network terminal has become an urgent problem to be solved.

[0004] Patent CN104571956A discloses a distributed data writing method based on multiple storage nodes, including: applied to a data writing system, the data writing system includes a request device, a splitting device, and a data storage device, the data storage device includes multiple storage nodes, and the method includes: the splitting device obtains a data writing request sent by the request device; splits the data writing request into multiple sub-requests, the multiple sub-requests are used to request writing multiple sub-data, the multiple sub-requests correspond to the multiple sub-data one by one, and the multiple sub-data constitute the data requested to be written by the request device; sends the multiple sub-requests to the multiple storage nodes respectively, the multiple sub-requests correspond to the multiple storage nodes one by one; and sends the sub-data requested to be written by each sub-request to the storage node corresponding to the sub-request. This method only involves writing distributed data to multiple storage nodes and does not have synchronous writing of data.

[0005] Based on this, this application provides a disk data writing method, apparatus, device, system, and chip to improve the prior art. Summary of the Invention

[0006] The purpose of the present application is to provide a disk data writing method, device, equipment, system and chip. This method can write the disk of a virtual machine to different storage backends simultaneously, improving the fault tolerance of the system, ensuring data consistency among multiple storage backends, and guaranteeing the reliability of data synchronization.

[0007] The purpose of the present application is achieved by the following technical solutions:

[0008] In a first aspect, the present application provides a disk data writing method for writing the data of a virtual machine disk to different storage backends simultaneously. The method includes:

[0009] Construct a bitmap, which is used to map the storage area of the disk and correspond to the storage backend;

[0010] Receive a data writing request for the virtual machine disk and send the data writing request to multiple storage backends simultaneously;

[0011] Use an asynchronous data synchronization thread to perform a data synchronization operation to obtain a data synchronization result, so as to write the data of the virtual machine disk to different storage backends simultaneously;

[0012] Detect whether the data synchronization result is successful.

[0013] The beneficial effects of this technical solution are as follows: By constructing a bitmap to map the disk storage area, receiving a virtual machine disk writing request and distributing it to multiple storage backends, using an asynchronous data synchronization thread to perform data synchronization, and detecting whether the synchronization result is successful, the real-time writing and fault management of the virtual machine disk data to multiple storage backends are realized. First, the bitmap data structure can dynamically record the writing status of each storage area by mapping the disk storage area and corresponding to the storage backend, providing an efficient basis for fault detection and data synchronization, and significantly reducing the risk of system interruption caused by storage backend failures. Second, the introduction of the asynchronous data synchronization thread realizes the automatic recovery of data in the faulty backend, avoiding the IO blocking problem caused by faults in traditional methods, improving the fault tolerance of the system, and ensuring data consistency among multiple storage backends at the same time. In addition, the step of detecting the synchronization result guarantees the reliability of data synchronization by judging the change of the version number, reducing the data overwrite problem caused by concurrent writing.

[0014] In some optional embodiments, the receiving a data writing request for the virtual machine disk and sending the writing request to multiple storage backends simultaneously includes:

[0015] Use an IO Hub to transmit the data writing request of the virtual machine disk to multiple IO Handlers;

[0016] Use an IO Handler to transfer the write request of the virtual machine disk to multiple storage backends, where each IO Handler corresponds to one storage backend.

[0017] The beneficial effects of this technical solution are as follows: By using an IO Hub to transfer the write request of the virtual machine disk to multiple IO Handlers, and then having the IO Handlers transfer the requests to the corresponding storage backends, efficient distribution and concurrent processing of write requests are achieved. First, as a central distribution module, the IO Hub can quickly distribute write requests to multiple IO Handlers, avoiding the bottleneck problem of single-point transmission and significantly improving the efficiency of request distribution. Second, each IO Handler corresponds to one storage backend, ensuring accurate transmission and independent management of requests, reducing interference between storage backends, and enhancing the stability of the system. In addition, this hierarchical design supports dynamic expansion. When the number of storage backends increases, the system can flexibly adapt to meet the needs of large-scale cloud computing scenarios.

[0018] In some optional embodiments, receiving the data write request of the virtual machine disk and simultaneously sending the data write request to multiple storage backends includes:

[0019] Judging one by one whether the data write request fails to execute;

[0020] If the data write to a certain storage backend fails, update the version number of the relevant block mapping in the bitmap corresponding to this storage backend to increment it by 1;

[0021] If there is no data write failure, send the data write request to multiple storage backends simultaneously.

[0022] The beneficial effects of this technical solution are as follows: By judging one by one whether the data write request fails, updating the version number of the block mapping in the bitmap of the failed storage backend, and not performing additional operations when there is no failure, real-time monitoring and dynamic management of the storage backend status are achieved. First, the operation of judging write failures one by one can timely identify faulty storage backends, avoiding waste of system resources caused by invalid writes and improving operation efficiency. Second, when the data write to a certain storage backend fails, update the version number of the relevant block mapping in the bitmap (increment by 1). Through the version number increment mechanism, fault information is accurately recorded, providing a key basis for subsequent fault handling and data synchronization, and enhancing the traceability of the system. In addition, the design of not performing additional operations when there is no write failure reduces unnecessary system overhead and optimizes resource utilization efficiency.

[0023] In some optional embodiments, the judging one by one whether the data write request fails to execute includes:

[0024] Determine whether there is a block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend;

[0025] If there is a block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend, return a failure message for execution;

[0026] If there is no block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend, perform the data writing operation.

[0027] The beneficial effects of this technical solution are as follows: By judging whether there is a block mapping with a version number greater than 0 in the storage backend bitmap before the writing operation, returning a failure message for execution or performing the writing operation, the premature identification and efficient handling of faulty backends are achieved. First, the operation of judging the bitmap version number can identify faulty backends in advance. If there is a block mapping with a version number greater than 0, the system returns a failure message for execution and skips the writing operation of this storage backend, avoiding the waste of system resources caused by invalid writing and significantly improving the operation efficiency. Second, if there is no block mapping with a version number greater than 0 in the bitmap, the writing operation is normally executed, ensuring the smoothness and stability of data writing. This pre-check mechanism effectively reduces the risk of IO blockage caused by faulty backends and improves the fault tolerance of the system. At the same time, the dynamic management of the bitmap version number provides an accurate basis for subsequent data synchronization and enhances the operability of the system.

[0028] In some optional embodiments, the data synchronization result is obtained by using an asynchronous data synchronization thread to perform a data synchronization operation to write the data of the virtual machine disk to different storage backends at the same time, including:

[0029] Use an asynchronous data synchronization thread to monitor the bitmaps corresponding to all storage backends and detect whether there is a block mapping with a version number greater than 0;

[0030] If it is detected that there is a block mapping with a version number greater than 0 in the bitmap corresponding to a certain storage backend, calculate the range of the corresponding physical address according to the encoding of the block mapping;

[0031] Read the data corresponding to the range of the physical address from the normal storage backend and write it to the corresponding physical address of the faulty storage backend to obtain the data synchronization result.

[0032] The beneficial effects of this technical solution are as follows: Through the operations of the asynchronous data synchronization thread listening to the bitmap, detecting block mappings with a version number greater than 0, calculating the physical address range, and reading data from the normal storage backend and writing it to the faulty backend, the automatic recovery of data in the faulty backend and the data consistency of multiple storage backends are achieved. First, the asynchronous data synchronization thread continuously listens to the bitmap status, detects block mappings with a version number greater than 0 in real time, quickly identifies the faulty backends that need to be synchronized, reduces the need for manual intervention, and improves the automation level of the system. Second, the physical address range is calculated according to the block mapping encoding, and data is read from the normal storage backend and overwritten and written to the faulty backend to ensure that the data in the faulty backend is consistent with the normal backend, avoiding system interruptions caused by data inconsistency in traditional methods. In addition, the design of asynchronous operations reduces interference with the main thread, ensures the real-time nature of the write operation, and improves the fault tolerance of the system.

[0033] In some alternative embodiments, detecting whether the data synchronization result is successful includes:

[0034] Detecting whether the version number of the block mapping continues to increase during data synchronization;

[0035] If it does not increase, reset the version number of the corresponding block mapping to 0 and send a synchronization success indication;

[0036] If it continues to increase, send a synchronization failure indication.

[0037] The beneficial effects of this technical solution are as follows: Through the operations of detecting whether the version number of the block mapping continues to increase during data synchronization, resetting the version number or sending a synchronization failure indication according to the result, the accurate judgment and reliable processing of the data synchronization result are achieved. First, the step of detecting whether the version number continues to increase can monitor the data consistency during the synchronization process in real time. If the version number does not increase, the block mapping version number is reset to 0 and a synchronization success indication is sent to ensure that the data recovery in the faulty backend is completed; if the version number continues to increase, a synchronization failure indication is sent and the next synchronization is awaited, avoiding data overwrite problems caused by concurrent writes. Second, this detection mechanism provides clear feedback for system operation and maintenance through clear success and failure indications, facilitating subsequent fault troubleshooting and optimization, and enhancing the maintainability of the system. In addition, this method effectively guarantees the integrity of data synchronization and reduces the risk of synchronization failure.

[0038] In a second aspect, the present application provides a disk data writing device, the device includes a processor, and the processor is configured to implement the following steps:

[0039] A bitmap construction module, configured to construct a bitmap, where the bitmap is used to map the storage area of the disk and correspond to the storage backend;

[0040] A request sending module, configured to receive a data writing request for a virtual machine disk and send the data writing request to multiple storage backends simultaneously;

[0041] A data synchronization module, configured to perform data synchronization operations using an asynchronous data synchronization thread to obtain a data synchronization result, so as to write the data of the virtual machine disk to different storage backends simultaneously;

[0042] A result checking module, configured to detect whether the data synchronization result is successful.

[0043] In some alternative embodiments, the processor is further configured to receive a writing request for a virtual machine disk in the following manner and send the writing request to multiple storage backends simultaneously:

[0044] Use an IO Hub to transfer the writing request of the virtual machine disk to multiple IO Handlers;

[0045] Use an IO Handler to transfer the writing request of the virtual machine disk to multiple storage backends, where each IO Handler corresponds to a storage backend.

[0046] In some alternative embodiments, the processor is configured to receive a data writing request for a virtual machine disk in the following manner and send the data writing request to multiple storage backends simultaneously:

[0047] Judging one by one whether the data writing request fails to execute;

[0048] If the data writing to a certain storage backend fails, update the version number of the relevant block mapping in the bitmap corresponding to the storage backend to increment it by 1;

[0049] If there is no data writing failure, send the data writing request to multiple storage backends simultaneously.

[0050] In some alternative embodiments, the processor is configured to judge one by one whether the data writing request fails to execute in the following manner:

[0051] Judge whether there is a block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend;

[0052] If there is a block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend, return a failure execution message;

[0053] If there is no block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend, perform the data writing operation.

[0054] In some alternative embodiments, the processor is configured to perform a data synchronization operation using an asynchronous data synchronization thread in the following manner to obtain a data synchronization result, so as to write the data of the virtual machine disk to different storage backends simultaneously:

[0055] Use the asynchronous data synchronization thread to monitor the bitmaps corresponding to all storage backends, and detect whether there is a block mapping with a version number greater than 0;

[0056] If it is detected that there is a block mapping with a version number greater than 0 in the bitmap corresponding to a certain storage backend, calculate the range of the corresponding physical address according to the encoding of the block mapping;

[0057] Read the data corresponding to the range of the physical address from the normal storage backend and write it to the corresponding physical address of the faulty storage backend to obtain a data synchronization result.

[0058] In some alternative embodiments, the processor is configured to detect whether the data synchronization result is successful in the following manner:

[0059] Detect whether the version number of the block mapping continues to increase during data synchronization;

[0060] If it does not increase, reset the version number of the corresponding block mapping to 0 and send a synchronization success indication;

[0061] If it continues to increase, send a synchronization failure indication.

[0062] In a third aspect, the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of any of the above methods are implemented.

[0063] In a fourth aspect, the present application provides a disk data writing system, which includes:

[0064] The above-mentioned electronic device.

[0065] In a fifth aspect, the present application provides a chip, which stores a computer program, and when the computer program is executed by a processor, the steps of any of the above methods are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The present application will be further described below in conjunction with the drawings and embodiments.

[0067] Figure 1 A flowchart showing a disk data writing method provided by an embodiment of the present application is shown.

[0068] Figure 2 A flowchart showing a data writing request sending method provided by an embodiment of the present application is shown.

[0069] Figure 3 The flowchart shows a data synchronization method provided by an embodiment of the present application.

[0070] Figure 4 The architecture diagram shows an architecture of a disk data writing structure provided by an embodiment of the present application.

[0071] Figure 5 The block diagram shows a structure of a disk data writing device provided by an embodiment of the present application.

[0072] Figure 6 The structural framework diagram shows a structural framework of an electronic device provided by an embodiment of the present application.

[0073] Figure 7 The structural schematic diagram shows a structural schematic of a disk data writing system provided by an embodiment of the present application.

[0074] Figure 8 The structural schematic diagram shows a structural schematic of a program product provided by an embodiment of the present application. Detailed implementation manners

[0075] Next, in combination with the accompanying drawings and specific implementation manners, the embodiments of the present application will be further described. It should be noted that, on the premise of no conflict, any combination of the following described embodiments or technical features can form a new implementation manner.

[0076] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item)" or similar expressions thereof refer to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b, or c can represent: a, b, c, a and b, a and c, b and c, a and b and c, where a, b, and c can be single or multiple. It should be noted that "at least one (item)" can also be interpreted as "one (item) or more items (items)".

[0077] It should also be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "exemplary" or "for example" aims to present relevant concepts in a specific manner.

[0078] Method Embodiment

[0079] See Figure 1 , Figure 1 shows a schematic flowchart of a disk data writing method provided by an embodiment of the present application.

[0080] An embodiment of the present application provides a disk data writing method for writing the data of a virtual machine disk to different storage backends simultaneously. The method includes:

[0081] Step 101: Construct a bitmap, which is used to map the storage area of the disk and corresponds to the storage backend;

[0082] Step 102: Receive a data writing request for the virtual machine disk and send the data writing request to multiple storage backends simultaneously;

[0083] Step 103: Use an asynchronous data synchronization thread to perform a data synchronization operation to obtain a data synchronization result, so as to write the data of the virtual machine disk to different storage backends simultaneously;

[0084] Step 104: Detect whether the data synchronization result is successful.

[0085] Thus, by constructing a bitmap to map the disk storage area, receiving a virtual machine disk write request and distributing it to multiple storage backends, using an asynchronous data synchronization thread to perform data synchronization, and detecting whether the synchronization result is successful, the real-time writing and fault management of the virtual machine disk data to multiple storage backends are realized.

[0086] First, the bitmap data structure can dynamically record the write status of each storage area by mapping the disk storage area and corresponding to the storage backend, providing an efficient basis for fault detection and data synchronization, and significantly reducing the risk of system interruption caused by storage backend failures.

[0087] Secondly, the introduction of the asynchronous data synchronization thread realizes the automatic recovery of data in the failed backend, avoids the IO blocking problem caused by failures in the traditional method, improves the fault tolerance of the system, and at the same time ensures the data consistency of multiple storage backends. In addition, the step of detecting the synchronization result ensures the reliability of data synchronization by judging the change of the version number, and reduces the data overwrite problem caused by concurrent writes.

[0088] In some optional embodiments, the receiving a write request for the virtual machine disk and sending the write request to multiple storage backends simultaneously includes:

[0089] Using an IO Hub to transmit the write request of the virtual machine disk to multiple IO Handlers;

[0090] Use the IO Handler to transfer the write requests of the virtual machine disk to multiple storage backends, where each IO Handler corresponds to one storage backend.

[0091] Thus, by using the IO Hub to transfer the write requests of the virtual machine disk to multiple IO Handlers, and the IO Handlers to transfer the requests to the corresponding storage backends, the efficient distribution and concurrent processing of the write requests are achieved. First, as a central distribution module, the IO Hub can quickly distribute the write requests to multiple IO Handlers, avoiding the bottleneck problem of single-point transmission and significantly improving the efficiency of request distribution.

[0092] Second, each IO Handler corresponds one-to-one with a storage backend, ensuring the accurate transmission and independent management of requests, reducing the interference between storage backends, and enhancing the stability of the system. In addition, this hierarchical design supports dynamic expansion. When the number of storage backends increases, the system can flexibly adapt to meet the requirements of large-scale cloud computing scenarios.

[0093] The embodiment of this application does not limit the number of IO Hubs, such as 1, 2, 5, 8, 10, 20, 50, etc.

[0094] The embodiment of this application does not limit the number of IO Handlers, such as 1, 2, 5, 8, 10, 20, 50, etc.

[0095] In the embodiment of this application, the IO Hub (Input / Output Center) is a system or component for centrally managing and coordinating various input / output operations. It is usually used for data transmission, device connection, and information exchange. The main functions of the IO Hub include: data integration: integrating and processing data from different sources; device management: connecting and managing multiple input / output devices, such as sensors, storage devices, etc.; protocol conversion: converting between different communication protocols to ensure compatibility between devices.

[0096] In the embodiment of this application, the IO Handler (Input / Output Handler) is code or a module that processes specific input / output operations. It is responsible for receiving input data, processing this data, and generating output. The main functions of the IO Handler include: data processing: performing corresponding processing according to the type and format of the input data; event response: responding to specific input events, such as user input or device signals; error handling: handling errors or exceptions that may occur during the input / output process.

[0097] In the embodiments of the present application, a Bitmap is a digital graphic format used to represent an image. It is composed of pixels, and each pixel represents a point in the image.

[0098] In one embodiment, a data structure of a bitmap is added. The bitmap is a mapping of the entire physical disk. Each physical address segment (such as 4K, 8K) is a block mapping in the bitmap. The sum of all block mappings is the entire disk area. Each block mapping also contains a version number, and the initial value of the version number is 0. For example, if the physical address of disk A is 0 - 32K and the bitmap is created based on 4K, the bitmap contains 8 block mappings. The pseudo-initial data structure is: 0:0, 1:0, 2:0, 3:0, 4:0, 5:0, 6:0, 7:0. The number on the left side of the colon is the block mapping code, and the number on the right side of the colon is the version number.

[0099] See Figure 2 , Figure 2 which shows a schematic flowchart of a method for sending a data write request provided by the embodiments of the present application.

[0100] In some alternative embodiments, receiving a data write request for a virtual machine disk and simultaneously sending the data write request to multiple storage backends includes:

[0101] Step S201: Judging one by one whether the data write request fails to execute;

[0102] Step S202: If the data write fails for a certain storage backend, update the version number of the relevant block mapping in the bitmap corresponding to this storage backend to increment it by 1;

[0103] Step S203: If there is no data write failure, send the data write request to multiple storage backends simultaneously.

[0104] Thus, through the steps of judging one by one whether the data write request fails, updating the version number of the block mapping in the bitmap of the failed storage backend, and not performing additional operations when there is no failure, the real-time monitoring and dynamic management of the storage backend status are achieved. First, the operation of judging one by one whether the write fails can promptly identify the faulty storage backend, avoiding the waste of system resources caused by invalid writes and improving the operation efficiency.

[0105] Secondly, when the data write fails for a certain storage backend, update the version number of the relevant block mapping in the bitmap (increment it by 1). Through the version number increment mechanism, the fault information is accurately recorded, providing a key basis for subsequent fault handling and data synchronization, and enhancing the traceability of the system. In addition, the design of not performing additional operations when there is no write failure reduces unnecessary system overhead and optimizes the resource utilization efficiency.

[0106] In some alternative embodiments, the system starts traversing the list of storage backends and performs data writing operations on each backend. The success and failure statuses of each writing operation are recorded. For all storage backends with failed writes, the system updates the version numbers of the relevant block mappings in the corresponding bitmaps. For example, if the write to backend A fails, the system increments the version number of that block by 1 to mark the block as updated but not successfully written. If the write operations for all storage backends are successful, the system sends the data write request to all storage backends simultaneously.

[0107] Each storage backend (a storage backend refers to a cluster that implements block storage) corresponds to a bitmap. When a write to the storage backend fails, the version number in the corresponding area of the bitmap is incremented by 1 (0 indicates no failure, >0 indicates a failure). When writing, it is determined in advance whether there is a value >0 in the bitmap to decide whether to write to the storage backend. If there is a value >0, the write is not performed and a failure is directly returned.

[0108] In some alternative embodiments, the step of successively determining whether the data write request fails includes:

[0109] Determining whether there are block mappings in the bitmap corresponding to the storage backend with a version number greater than 0;

[0110] If there are block mappings in the bitmap corresponding to the storage backend with a version number greater than 0, return a failure message for the execution;

[0111] If there are no block mappings in the bitmap corresponding to the storage backend with a version number greater than 0, perform the data writing operation.

[0112] Thus, by the steps of determining whether there are block mappings in the storage backend bitmap with a version number greater than 0 before the write operation, returning a failure message for the execution, or performing the write operation, the premature identification and efficient handling of faulty backends are achieved.

[0113] First, the operation of determining the bitmap version number can prematurely identify faulty backends. If there are block mappings with a version number greater than 0, the system returns a failure message for the execution and skips the write operation for that storage backend, avoiding the waste of system resources caused by invalid writes and significantly improving the operating efficiency.

[0114] Second, if there are no block mappings in the bitmap with a version number greater than 0, the write operation is normally performed, ensuring the smoothness and stability of data writing. This pre-check mechanism effectively reduces the risk of IO blockage caused by faulty backends and improves the system's fault tolerance. At the same time, the dynamic management of the bitmap version number provides an accurate basis for subsequent data synchronization, enhancing the operability of the system.

[0115] In some alternative embodiments, after receiving a disk data write request for a virtual machine, the system first checks the bitmaps corresponding to each storage backend one by one to determine whether there are block mappings with a version number greater than 0. If it is found that there are block mappings with a version number greater than 0 in the bitmap of a certain storage backend, an information indicating a failed execution will be returned, indicating that the backend cannot perform a write operation. On the contrary, if there are no block mappings with a version number greater than 0 in the bitmap of a certain storage backend, the system will perform a data write operation and successfully write the data to the storage backend, thereby ensuring the validity and consistency of the data.

[0116] See Figure 3 , Figure 3 which shows a schematic flowchart of a data synchronization method provided by an embodiment of the present application.

[0117] In some alternative implementation manners, the data synchronization operation is performed by using an asynchronous data synchronization thread to obtain a data synchronization result, so as to write the data of the virtual machine disk to different storage backends simultaneously, including:

[0118] Step S301: Use an asynchronous data synchronization thread to monitor the bitmaps corresponding to all storage backends and detect whether there are block mappings with a version number greater than 0;

[0119] Step S302: If it is detected that there are block mappings with a version number greater than 0 in the bitmap corresponding to a certain storage backend, calculate the range of the corresponding physical address according to the encoding of the block mapping;

[0120] Step S303: Read the data corresponding to the range of the physical address from a normal storage backend and write it to the corresponding physical address of the faulty storage backend to obtain a data synchronization result.

[0121] Thus, through the operations of the asynchronous data synchronization thread to monitor the bitmap, detect block mappings with a version number greater than 0, calculate the physical address range, and read data from a normal storage backend and write it to the faulty backend, the automatic recovery of the data of the faulty backend and the data consistency of multiple storage backends are realized. First, the asynchronous data synchronization thread continuously monitors the bitmap status, detects block mappings with a version number greater than 0 in real time, quickly identifies the faulty backend that needs to be synchronized, reduces the need for manual intervention, and improves the automation level of the system.

[0122] Secondly, calculate the physical address range according to the block mapping encoding, and read data from a normal storage backend and overwrite it to the faulty backend, ensuring that the data of the faulty backend is consistent with that of the normal backend, and avoiding system interruption problems caused by data inconsistency in traditional methods. In addition, the design of asynchronous operation reduces the interference to the main thread, ensures the real-time performance of the write operation, and improves the fault tolerance of the system.

[0123] In some alternative embodiments, there is an asynchronous data synchronization thread that monitors whether bitmapA has a value greater than 0. When there is a value greater than 0, the asynchronous data synchronization thread calculates the physical address based on the block mapping in bitmapA, requests data from the normal storage backend with the physical address to overwrite the physical address corresponding to storage backend A, and then resets the block mapping back to 0 according to the version number to indicate successful synchronization. However, if the version number of the block mapping is still increasing at this time (new data writing occurs), this synchronization is regarded as a failure, and the next synchronization is awaited. The asynchronous data synchronization thread will keep trying to synchronize data until there is no bitmap with a value greater than 0.

[0124] In some alternative embodiments, detecting whether the data synchronization result is successful includes:

[0125] Detecting whether the version number of the block mapping continues to increase during data synchronization;

[0126] If it does not increase, reset the version number of the corresponding block mapping to 0 and send a synchronization success indication;

[0127] If it continues to increase, send a synchronization failure indication.

[0128] Thus, through the operations of detecting whether the version number of the block mapping continues to increase during data synchronization, resetting the version number according to the result, or sending a synchronization failure indication, an accurate judgment and reliable processing of the data synchronization result are achieved.

[0129] First, the step of detecting whether the version number continues to increase can monitor the data consistency during the synchronization process in real time. If the version number does not increase, reset the block mapping version number to 0 and send a synchronization success indication to ensure the completion of data recovery for the faulty backend; if the version number continues to increase, send a synchronization failure indication and await the next synchronization, avoiding data overwrite problems caused by concurrent writing.

[0130] Second, this detection mechanism provides clear feedback for system operation and maintenance through explicit success and failure indications, facilitating subsequent fault troubleshooting and optimization, and enhancing the maintainability of the system. In addition, this method effectively guarantees the integrity of data synchronization and reduces the risk of synchronization failure.

[0131] See Figure 4 , Figure 4 which shows an architecture diagram of a disk data writing structure provided by an embodiment of the present application.

[0132] In a specific embodiment, a virtual machine dual-active system is deployed on a server configured with a 16-core CPU and 64 GB of memory, connecting to 3 storage backends (StorageBackendA, StorageBackendB, StorageBackendC), each with a capacity of 10 TB, for implementing redundant data storage. When the system starts, a bitmap data structure is first constructed. The disk is divided into 4K blocks for mapping, and each storage backend corresponds to a bitmap with an initial version number of 0. For example, the bitmap MapDict of StorageBackendA is {"0":0,"1":0,...,"255":0}, representing 256 block mappings. A virtual machine initiates a 1MB write request with a starting address of 0x0000. After receiving the request, the IO Hub module distributes it to 3 IO Handlers simultaneously, and each IO Handler is responsible for forwarding the request to the corresponding storage backend. During the write process, the write to StorageBackendB fails due to a network interruption. After the IO Handler detects the failure, it updates the version number of the corresponding block mapping "0" (address 0x0000 - 0x1000) in the bitmap of StorageBackendB from "0":0 to "0":1. When subsequent write requests target the same address, the system checks "0":1, directly skips the write operation to StorageBackendB and increments the version number to "0":2. The asynchronous data synchronization thread (SyncThread) starts, monitors all bitmaps, discovers "0":2 in StorageBackendB, calculates the physical address range 0x0000 - 0x1000, reads data from StorageBackendA and overwrites it to StorageBackendB. During synchronization, the version number does not continue to increase, and the system resets "0":2 to "0":0, indicating successful synchronization. The entire process ensures data consistency, the system runs stably, and the fault recovery is efficient.

[0133] BitMap Pseudocode:

[0134] Class BitMap:

[0135] StorageBackend = None # Storage backend identifier

[0136] MapSize = 4K # Block mapping size

[0137] MapDict = {

[0138] "0":0, # The key value is the encoding of the 0th block mapping, and the value is the data version of the 0th block.

[0139] "1":0, # The key is the encoding of the 1st block mapping, and the value is the data version of the 1st block.

[0140] "2":0, # The key is the encoding of the 2nd block mapping, and the value is the data version of the 2nd block. ...

[0142] }

[0143] Device Embodiment

[0144] See Figure 5 , Figure 5 which shows the structural block diagram of a disk data writing device provided by the embodiment of the present application.

[0145] The embodiment of the present application also provides a disk data writing device, the specific implementation manner of which is the same as the implementation manner and the achieved technical effects recorded in the above method embodiment, and some contents will not be elaborated here.

[0146] The present application provides a disk data writing device, the device includes a processor, and the processor is configured to implement the following steps:

[0147] Bitmap construction module 101, configured to construct a bitmap, which is used to map the storage area of the disk and correspond to the storage backend;

[0148] Request sending module 102, configured to receive a data writing request of the virtual machine disk and send the data writing request to multiple storage backends simultaneously;

[0149] Data synchronization module 103, configured to perform data synchronization operations using an asynchronous data synchronization thread to obtain a data synchronization result, so as to write the data of the virtual machine disk to different storage backends simultaneously;

[0150] Result checking module 104, configured to detect whether the data synchronization result is successful.

[0151] In some optional implementation manners, the processor is further configured to receive a writing request of the virtual machine disk in the following manner and send the writing request to multiple storage backends simultaneously:

[0152] Use the IO Hub to transfer the writing request of the virtual machine disk to multiple IO Handlers;

[0153] Use the IO Handler to transfer the writing request of the virtual machine disk to multiple storage backends, where each IOHandler corresponds to a storage backend.

[0154] In some alternative embodiments, the processor is configured to receive a data write request for a virtual machine disk in the following manner and send the data write request to multiple storage backends simultaneously:

[0155] Judging one by one whether the data write request fails to execute;

[0156] If the data write to a certain storage backend fails, update the version number of the relevant block mapping in the bitmap corresponding to this storage backend to increment it by 1;

[0157] If there is no data write failure, send the data write request to multiple storage backends simultaneously.

[0158] In some alternative embodiments, the processor is configured to judge one by one whether the data write request fails to execute in the following manner:

[0159] Judge whether there is a block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend;

[0160] If there is a block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend, return a failure execution message;

[0161] If there is no block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend, perform the data write operation.

[0162] In some alternative embodiments, the processor is configured to perform a data synchronization operation using an asynchronous data synchronization thread to obtain a data synchronization result, so as to write the data of the virtual machine disk to different storage backends simultaneously:

[0163] Use the asynchronous data synchronization thread to monitor the bitmaps corresponding to all storage backends and detect whether there is a block mapping with a version number greater than 0;

[0164] If it is detected that there is a block mapping with a version number greater than 0 in the bitmap corresponding to a certain storage backend, calculate the range of the corresponding physical address according to the encoding of the block mapping;

[0165] Read the data corresponding to the range of this physical address from a normal storage backend and write it to the corresponding physical address of the faulty storage backend to obtain a data synchronization result.

[0166] In some alternative embodiments, the processor is configured to detect whether the data synchronization result is successful in the following manner:

[0167] Detect whether the version number of the block mapping continues to increase during data synchronization;

[0168] If it does not increase, reset the version number of the corresponding block mapping to 0 and send a synchronization success indication;

[0169] If it continues to increase, a synchronization failure indication is sent.

[0170] Device Embodiment

[0171] An embodiment of the present application further provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of any one of the above methods are implemented. Its specific implementation manners are the same as those recorded in the above method embodiments and the achieved technical effects are the same, and some contents will not be elaborated again.

[0172] See Figure 6 , Figure 6 which shows a structural framework diagram of an electronic device provided by an embodiment of the present application.

[0173] The electronic device includes at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0174] The memory 210 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 211 and / or a cache memory 212, and may further include a read-only memory (ROM) 213.

[0175] Among them, the memory 210 also stores a computer program, and the computer program can be executed by the processor 220, so that the processor 220 implements the steps of any one of the above methods.

[0176] The memory 210 may further include a utility 214 having at least one program module 215. Such program modules 215 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0177] Correspondingly, the processor 220 may execute the above computer program and may also execute the utility 214.

[0178] The processor 220 may adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0179] The bus 230 can be one or more representing several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus of any bus structure using multiple bus structures.

[0180] The electronic device can also communicate with one or more external devices 240 such as a keyboard, a pointing device, a Bluetooth device, etc., and can also communicate with one or more devices capable of interacting with the electronic device, and / or communicate with any device (such as a router, a modem, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication can be carried out through the input / output interface 250. Moreover, the electronic device can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 260. The network adapter 260 can communicate with other modules of the electronic device through the bus 230. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0181] System embodiments

[0182] See Figure 7 , Figure 7 which shows a schematic structural diagram of a disk data writing system provided by an embodiment of the present application.

[0183] An embodiment of the present application also provides a disk data writing system, and the system includes:

[0184] The above-mentioned electronic device.

[0185] Medium embodiments

[0186] An embodiment of the present application also provides a chip, and the chip stores a computer program, and when the computer program is executed by a processor, it implements the steps of any one of the above methods. Its specific implementation manners are the same as those recorded in the above method embodiments and the achieved technical effects are the same, and some contents will not be elaborated again.

[0187] See Figure 8 , Figure 8 which shows a schematic structural diagram of a program product provided by an embodiment of the present application.

[0188] The program product is used to implement any of the above methods. The program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In the embodiments of the present application, the readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0189] The chip may include a data signal that is included in the baseband or propagated as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium that can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above. The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as C language, Python language, or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0190] This application is described from the perspectives of purpose of use, efficacy, progress, and novelty, and has met the functional improvement and use requirements emphasized by the Patent Law. The above description and accompanying drawings of this application are only preferred embodiments of this application and do not limit this application. Therefore, all those that are similar or identical to the structure, device, features, etc. of this application, that is, all equivalent substitutions or modifications made according to the scope of the patent application of this application, shall fall within the scope of protection of the patent application of this application.

Claims

1. A method for writing disk data, characterized in that Writing the data of a virtual machine disk to different storage backends simultaneously, the method includes: Constructing a bitmap, which is used to map the storage areas of the disk and corresponds to the storage backends; Receiving a data write request for the virtual machine disk and sending the data write request to multiple storage backends simultaneously; Performing a data synchronization operation using an asynchronous data synchronization thread to obtain a data synchronization result, so as to write the data of the virtual machine disk to different storage backends simultaneously; Detecting whether the data synchronization result is successful.

2. The disk data writing method according to claim 1, wherein The receiving a data write request for the virtual machine disk and sending the write request to multiple storage backends simultaneously includes: Using an IO Hub to transmit the write request of the virtual machine disk to multiple IO Handlers; Using an IO Handler to transmit the write request of the virtual machine disk to multiple storage backends, where each IO Handler corresponds to one storage backend.

3. The disk data writing method according to claim 1, wherein The receiving a data write request for the virtual machine disk and sending the data write request to multiple storage backends simultaneously includes: Judging one by one whether the data write request fails to execute; If the data write fails for a certain storage backend, update the version number of the relevant block mapping in the bitmap corresponding to this storage backend to increment it by 1; If there is no data write failure, send the data write request to multiple storage backends simultaneously.

4. The disk data writing method according to claim 3, wherein The judging one by one whether the data write request fails to execute includes: Judging whether there is a block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend; If there is a block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend, return a failure execution message; If there is no block mapping with a version number greater than 0 in the bitmap corresponding to the storage backend, perform the data write operation.

5. The disk data writing method according to claim 1, characterized in that, The performing a data synchronization operation using an asynchronous data synchronization thread to obtain a data synchronization result, so as to write the data of the virtual machine disk to different storage backends simultaneously includes: Using an asynchronous data synchronization thread to monitor the bitmaps corresponding to all storage backends and detecting whether there is a block mapping with a version number greater than 0; If it is detected that there is a block mapping with a version number greater than 0 in the bitmap corresponding to a certain storage backend, calculate the range of the corresponding physical addresses according to the encoding of the block mapping; Read the data corresponding to the range of the physical addresses from a normal storage backend and write it to the corresponding physical addresses of the faulty storage backend to obtain a data synchronization result.

6. The disk data writing method according to claim 1, wherein The detecting whether the data synchronization result is successful includes: Detecting whether the version number of the block mapping continues to increase during data synchronization; If it does not increase, reset the version number of the corresponding block mapping to 0 and send a synchronization success indication; If it continues to increase, send a synchronization failure indication.

7. A disk data writing device, characterized in that, The disk data writing device includes: A bitmap construction module, which is used to construct a bitmap, and the bitmap is used to map the storage areas of the disk and corresponds to the storage backends; A request sending module, which is used to receive a data write request for the virtual machine disk and send the data write request to multiple storage backends simultaneously; A data synchronization module, which is used to perform a data synchronization operation using an asynchronous data synchronization thread to obtain a data synchronization result, so as to write the data of the virtual machine disk to different storage backends simultaneously; A result checking module, which is used to detect whether the data synchronization result is successful.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory stores a computer program, and the processor is configured to execute the steps of implementing the method according to any one of claims 1-6.

9. A disk data writing system, characterized in that, The disk data writing system includes: The electronic device according to claim 8.

10. A chip, characterized in that, The chip stores a computer program, and when the computer program is executed by a processor, the steps of implementing the method according to any one of claims 1-6 are realized.

Images