Disk data import and export method of virtual machine and computing device

Abstract

Embodiments of the present application relate to a disk data import and export method of a virtual machine and a computing device. The method comprises: when a server is detected to have a hardware accelerator available, splitting disk data to be processed into a plurality of data segments according to the processing capacity of the hardware accelerator; performing compression or decompression processing on the plurality of data segments by the hardware accelerator, and performing merging processing on each data segment after the compression or decompression processing to generate complete processed data; and when the server is detected to not have a hardware accelerator available, performing pure software compression or decompression processing on the disk data to be processed by a CPU, and generating complete processed data. In this way, the disk format conversion and data processing speed can be significantly improved, the computing burden of the host server can be greatly reduced, and the disk fast migration, backup and sharing operation in various virtualization environments can be effectively supported.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method for importing and exporting disk data in a virtual machine and a computing device. Background Technology

[0002] In virtualization platforms, importing and exporting virtual machine disk images is a fundamental function for enabling virtual machine migration, backup, and cross-platform sharing. Typically, virtual machines use native formats like raw to maximize performance during runtime, but during import / export, to save storage space and network bandwidth, they often need to be converted to compressed formats such as qcow2 or vhd.

[0003] In related technologies, streaming compression algorithms are typically used to compress disk data during the format conversion process. Although these technologies can reduce file size, their serial processing mechanism—meaning that the compression of the next data segment must wait for the previous segment to complete—results in low efficiency for the entire conversion process. Summary of the Invention

[0004] This application provides a method for importing and exporting disk data in a virtual machine and a computing device, which can effectively support rapid disk migration, backup and sharing operations in various virtualization environments.

[0005] According to a first aspect of the embodiments of this application, a method for importing and exporting disk data of a virtual machine is provided, the method comprising:

[0006] When a usable hardware accelerator is detected on the server, the disk data to be processed is divided into multiple data fragments according to the processing capacity of the hardware accelerator. The hardware accelerator compresses or decompresses multiple data fragments, and then merges the compressed or decompressed data fragments to generate complete processed data.

[0007] When no available hardware accelerator is detected on the server, the CPU performs pure software compression or decompression on the disk data to be processed, and generates complete processed data.

[0008] This embodiment detects available hardware accelerators on the server, segments the data based on their processing capabilities, and processes multiple data segments in parallel using the hardware accelerators. This efficiently offloads compression or decompression tasks during virtual machine disk import / export to dedicated hardware. When no available hardware accelerators are detected on the server, the CPU performs pure software compression or decompression on the disk data to be processed. This method not only significantly improves disk format conversion and data processing speed but also greatly reduces the computational burden on the host server, effectively supporting rapid disk migration, backup, and sharing operations in various virtualization environments.

[0009] In one possible implementation, the disk data to be processed is divided into multiple data fragments based on the processing power of the hardware accelerator, including: Get the size of the data processing window for the hardware accelerator; Based on the data processing window size, the disk data to be processed is divided into multiple data fragments; the size of each data fragment is an integer multiple of the data processing window size.

[0010] This embodiment ensures optimal matching between data fragments and hardware processing units by directly aligning the data partitions to integer multiples of the data processing window size of the hardware accelerator. This not only significantly improves the utilization efficiency of hardware resources and single-processing performance, but also provides a prerequisite for achieving high-concurrency, low-latency pipelined compression / decompression processing, thus laying the foundation for accelerated processing.

[0011] In one possible implementation, when exporting the disk data to be processed, the disk data is divided into multiple data fragments according to the processing capability of the hardware accelerator, and the process also includes: Get the buffer size of the hardware accelerator; The size of each data slice is adjusted according to the buffer size of the hardware accelerator; wherein the adjusted size of each data slice does not exceed the buffer size.

[0012] This embodiment effectively adjusts the data fragments by introducing buffer size constraints in the export compression scenario, ensuring that each data fragment strictly matches the input cache capacity of the hardware accelerator. This not only effectively prevents boundary anomalies during data transmission and processing, but also further optimizes the utilization efficiency of hardware resources and the reliability of task scheduling. It is one of the key guarantees for achieving high-speed and stable disk export compression.

[0013] In one possible implementation, when importing disk data to be processed, the disk data is divided into multiple data fragments based on the processing capability of the hardware accelerator, and further includes: Identify the fragment delimiters between compressed data blocks in the disk data to be processed; The disk data to be processed is divided into multiple independent data fragments based on the fragment delimiter.

[0014] This embodiment achieves lossless identification and restoration of the original concurrent compression structure by directly parsing and using the pre-set fragment delimiters in the compressed file during the decompression scenario. This method not only provides accurate and efficient segmentation and avoids additional computational overhead, but also provides a suitable data unit for submitting each independent compressed block directly and in parallel to the hardware accelerator for decompression processing.

[0015] In one possible implementation, multiple data fragments are compressed or decompressed using a hardware accelerator, including: Queries the amount of available computing resources for the hardware accelerator; Determine the concurrency level for concurrent processing based on the amount of available computing resources; Based on the concurrency level, multiple data fragments are submitted concurrently to the hardware accelerator for compression or decompression.

[0016] This embodiment achieves precise matching and dynamic balance between hardware parallelism and software task scheduling by using a dynamic concurrency control mechanism to sense the available computing resources of the hardware accelerator in real time and determine and adjust the task concurrency accordingly. This not only significantly improves the utilization efficiency of hardware resources and the throughput of task processing, but also effectively avoids processing delays or resource idleness caused by overload or underload, thereby ensuring the efficiency and stability of virtual machine disk import and export processing at the system level.

[0017] In one possible implementation, when multiple data shards are submitted concurrently to the hardware accelerator for processing, a unique sequence identifier is attached to each data shard.

[0018] By attaching a unique sequence identifier to each concurrently submitted data fragment, this embodiment provides an accurate and reliable basis for reassembling potentially out-of-order processing results returned by the hardware accelerator. This mechanism effectively solves the problem of result order uncertainty in high-concurrency processing, ensuring that subsequent merge processing can quickly and error-free restore all fragment results into a complete and continuous data stream. Thus, while fully leveraging the advantages of hardware parallel performance, it strictly guarantees the integrity and correctness of the final output data.

[0019] In one possible implementation, the various data fragments after compression or decompression are merged, including: Listen for the processing results returned by the hardware accelerator, including compression or decompression results; Based on the sequence identifier carried by the processing results, all processing results are merged in order to reconstruct complete compressed or decompressed data.

[0020] This embodiment achieves accurate and efficient reassembly of the result data stream under high-concurrency processing mode by actively listening for hardware responses and strictly sorting and merging according to preset order identifiers. This method fully preserves the processing speed advantages brought by hardware parallelism while completely solving the data misalignment problem caused by out-of-order responses, ensuring the integrity and logical correctness of the final output data.

[0021] In one possible implementation, the hardware accelerator is hardware for data compression and decompression processing, and the hardware accelerator includes at least one of the following: a coprocessor integrated into the central processing unit or a separate hardware accelerator card.

[0022] By explicitly supporting various hardware implementations such as integrated coprocessors and independent hardware accelerator cards, this embodiment enables the method to flexibly adapt to hardware platforms of different architectures and manufacturers, greatly improving the versatility and deployability of the technical solution. It ensures that existing or dedicated hardware acceleration resources can be effectively utilized in various virtualization environments, thereby achieving stable and efficient virtual machine disk processing acceleration in a wide range of practical application scenarios.

[0023] In one possible implementation, the CPU performs pure software compression or decompression processing on the disk data to be processed, which may specifically include: When no available hardware accelerator is detected on the server, the disk data to be processed is converted to the target format using a disk format conversion tool. During the conversion process, software compression algorithms are used to compress the disk data to be processed, or to decompress the already compressed disk data to be processed.

[0024] This embodiment introduces a complete software backup processing path, enabling the disk import / export method to possess strong adaptability and high reliability independent of specific hardware. This design ensures that the system can automatically degrade and complete core processing tasks in various deployment environments (including traditional servers without hardware accelerators), thereby greatly expanding the applicability and deployment flexibility of the method and providing users with continuous and reliable service assurance.

[0025] According to a second aspect of the embodiments of this application, a device for importing and exporting disk data of a virtual machine is provided, the device comprising: The data sharding module is used to divide the disk data to be processed into multiple data shards according to the processing capacity of the hardware accelerator when an available hardware accelerator is detected on the server. The first data processing module is used to compress or decompress multiple data fragments using a hardware accelerator, and then merge the compressed or decompressed data fragments to generate complete processed data.

[0026] The second data processing module is used to perform pure software compression or decompression of the disk data to be processed by the CPU when it is detected that no available hardware accelerator is available on the server, and to generate complete processed data.

[0027] According to a third aspect of the embodiments of this application, a computing device is provided. The computing device includes a memory and a processor, the memory storing a computer program, and the processor executing the program to implement the method as described above.

[0028] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the methods described in the embodiments of this application.

[0029] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods described above in the embodiments of this application. Attached Figure Description

[0030] More details, features, and advantages of embodiments of the present application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 A schematic diagram of the architecture of a virtual machine disk import / export acceleration system provided in an exemplary embodiment of this application; Figure 2 A flowchart of a virtual machine disk data import / export method provided as an exemplary embodiment of this application; Figure 3 A schematic diagram of a disk file compression process provided for an exemplary embodiment of this application; Figure 4 A schematic diagram illustrating the decompression and restoration process of a compressed disk file provided in an exemplary embodiment of this application; Figure 5 A flowchart illustrating the interaction between software threads, accelerator drivers, and hardware accelerators during data processing, provided as an exemplary embodiment of this application; Figure 6 A schematic block diagram of the functional modules of a disk data import / export apparatus provided in an exemplary embodiment of this application; Figure 7 A structural block diagram of a computing device provided for an exemplary embodiment of this application. Detailed Implementation

[0031] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that embodiments of this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the embodiments of this application. It should be understood that the accompanying drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0032] It should be understood that the various steps described in the method implementation of this application may be performed in different orders and / or in parallel. Furthermore, the method implementation may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0033] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in the embodiments of this application are only used to distinguish different devices, modules, or units, and are not used to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0034] It should be noted that the terms "one" and "more" mentioned in the embodiments of this application are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0035] The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0036] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation of this application's embodiments. Other methods that comply with relevant laws and regulations can also be applied to the implementation of this application's embodiments.

[0037] Figure 1This is a schematic diagram of the architecture of the virtual machine disk import / export acceleration system provided in this application embodiment. Referring to Figure 1, the system is deployed in server 10, which acts as a virtualization host, running one or more virtual machines 14, and is responsible for managing the import and export operations of virtual machine disk data for the virtual machines 14. To achieve accelerated processing, server 10 also includes a disk import / export module 11, an acceleration processing module 12, and a hardware accelerator 13. Wherein: The disk import / export module 11 is used to receive import or export requests for virtual machine disks. Specifically, the disk import / export module 11 can detect whether there is an available hardware accelerator 13 in the server 10; if it exists, the virtual machine disk data to be processed is handed over to the acceleration processing module 12 for accelerated processing; if it does not exist, the server 10 is controlled to use a conventional software processing flow to perform disk format conversion.

[0038] The acceleration processing module 12, as a software module, interacts with the hardware accelerator 13 and is responsible for executing the acceleration processing logic. Its functions include: dynamically dividing disk data into multiple data fragments based on the processing capability parameters of the hardware accelerator 13 (such as data processing window size, buffer size, etc.); submitting the fragments to the hardware accelerator 13 concurrently; and receiving and merging the processing results returned by the hardware accelerator 13. The acceleration processing module 12 is configurable and can adapt to different types of hardware accelerators.

[0039] Hardware accelerator 13 is a dedicated hardware processing unit located in server 10, used to offload computational tasks of data compression or decompression. It can take the form of a coprocessor integrated into the central processing unit or a standalone hardware accelerator card. Accelerator 14 has multi-queue processing capabilities, enabling it to concurrently process multiple data fragments submitted by acceleration processing module 12.

[0040] pass Figure 1 It is known that after determining the acceleration path, the disk import / export module 11 will call the acceleration processing module 12. The acceleration processing module 12 relies on the computing power of the hardware accelerator 13 to execute the complete acceleration process of data sharding, concurrent submission and result merging, and finally returns the processed disk data to the disk import / export module 11 to complete the import or export operation.

[0041] This embodiment offloads the virtual machine disk data compression and decompression computation tasks to a dedicated hardware accelerator and adopts a collaborative mechanism based on hardware capability dynamic sharding, multi-queue concurrent processing, and result merging. This significantly improves the speed of virtual machine disk import and export processing while greatly reducing the computing load on the host server. The system architecture makes full use of hardware acceleration resources, has good adaptability and scalability, and can be effectively applied to disk migration, backup, and cross-platform sharing operations in various virtualization scenarios such as hyper-converged infrastructure and private clouds.

[0042] Based on the above embodiments, this application also provides a method for importing and exporting disk data in a virtual machine. This method can be applied to the aforementioned server, such as... Figure 2 As shown, the method may include the following steps: In step S210, when an available hardware accelerator is detected on the server, the disk data to be processed is divided into multiple data fragments according to the processing capability of the hardware accelerator.

[0043] In this embodiment, when importing or exporting disk data to a virtual machine running on the server, the presence of an available hardware accelerator on the server can be detected. A hardware accelerator is a dedicated hardware processing unit used to offload data compression or decompression computation tasks, such as a coprocessor integrated into the CPU (central processing unit) or a separate hardware accelerator card. This detection can be achieved by querying system drivers or dedicated interfaces to determine whether available hardware acceleration resources are currently available.

[0044] Specifically, processing capability parameters include at least the data processing window size of the hardware accelerator; in export (compression) scenarios, the buffer size can be further constrained. During segmentation, based on the aforementioned parameters, the data is divided into multiple appropriately sized fragments, for example, ensuring that the size of each fragment is an integer multiple of the data processing window size, and does not exceed the buffer limit during compression.

[0045] In step S220, multiple data fragments are compressed or decompressed using a hardware accelerator, and the compressed or decompressed data fragments are merged to generate complete processed data.

[0046] In this embodiment, multiple data fragments are compressed (corresponding to export) or decompressed (corresponding to import) using a hardware accelerator, and the processed data fragments are then merged to generate complete processed data. This step employs a concurrent processing mechanism: the available computing resources of the hardware accelerator are queried in real time to dynamically determine the concurrency level, and multiple data fragments are submitted to the accelerator concurrently; simultaneously, a unique sequence identifier is attached to each fragment upon submission. During processing, the results returned by the accelerator are continuously monitored, and all results are merged sequentially according to the sequence identifiers, thereby reconstructing a continuous and correct complete data stream.

[0047] In step S230, when it is detected that no available hardware accelerator is available on the server, the CPU performs pure software compression or decompression on the disk data to be processed, and generates complete processed data.

[0048] When no available hardware accelerator is available in the server, the system can switch to a pure software processing mode, in which the host CPU independently performs disk data compression or decompression and format conversion operations, and generates complete processed data.

[0049] Throughout the entire software processing, compression or decompression computation tasks can be handled by the host's general-purpose CPU cores without relying on any dedicated hardware accelerators. The system can appropriately utilize the CPU's multi-core capabilities through multi-threading or multi-processing.

[0050] This purely software-based processing can serve as a backup solution, ensuring that virtual machine disk import and export operations can still be completed normally in server environments without hardware accelerators, thus ensuring the versatility and deployment flexibility of the solution.

[0051] This embodiment detects available hardware accelerators on the server, segments the data based on their processing capabilities, and processes multiple data segments in parallel using the hardware accelerators. This efficiently offloads compression or decompression tasks during virtual machine disk import / export to dedicated hardware. When no available hardware accelerators are detected on the server, the CPU performs pure software compression or decompression on the disk data to be processed. This method not only significantly improves disk format conversion and data processing speed but also greatly reduces the computational burden on the host server, effectively supporting rapid disk migration, backup, and sharing operations in various virtualization environments.

[0052] Figure 3 This is a schematic diagram illustrating the disk file compression process provided in an embodiment of this application. Figure 3As shown, before compression, the original virtual machine disk file (i.e., the disk data to be processed) is pre-sliced ​​into multiple contiguous data fragments, such as fragment 1, fragment 2, fragment 3, etc. These data fragments are then sent to the data compression processing stage, which can call hardware accelerators or software compression units to perform independent and parallel compression operations on each fragment. After compression, each original data fragment is converted into an independent compressed result fragment, such as result fragment 1, result fragment 2, result fragment 3, etc. These result fragments are further encapsulated and organized according to standard or custom formats (e.g., gzip compressed file 1, gzip compressed file 2, gzip compressed file 3, etc.), ultimately forming a complete compressed file. This process intuitively demonstrates the fragmented concurrent compression mechanism used in the embodiment, which significantly improves the overall compression efficiency by splitting large files into multiple independently processable data units and utilizing hardware or software resources for parallel compression, while avoiding the serial waiting overhead caused by data dependencies in traditional streaming compression.

[0053] Figure 4 This is a schematic diagram illustrating the decompression and restoration process of a compressed disk file provided in an embodiment of this application. Figure 4 As shown, in a decompression scenario, a compressed disk file (such as gzip compressed file 1) serves as the input file to be decompressed. First, based on pre-defined separators (such as fragment boundary markers) in the compressed file, multiple independent compressed data blocks are identified and extracted, i.e., data fragments (e.g., fragment 1, fragment 2, fragment 3, etc.). This process corresponds to the "slicing" stage in the diagram. Subsequently, these data fragments are submitted in parallel to an accelerator (such as a hardware accelerator or software decompression module) for independent and concurrent decompression operations. After decompression, each compressed data fragment is restored to its corresponding original data block, i.e., the result fragment (e.g., result fragment 1, result fragment 2, result fragment 3, etc.). Finally, all result fragments are merged and combined according to their internal sequence identifiers to reconstruct the complete, restored original disk file. This process clearly demonstrates the segmentation identification, concurrent decompression, and ordered merging mechanism adopted by the embodiment on the decompression side. By fully utilizing the hardware parallel capabilities and the independence of segmented processing, it significantly improves decompression efficiency and supports the rapid restoration and loading of large-scale disk images.

[0054] Based on the above embodiments, in another embodiment provided in this application, in order to specifically describe how to divide the disk data to be processed into multiple data fragments according to the processing capability of the hardware accelerator, the above step S210 may specifically include the following steps: Step S211: Obtain the data processing window size of the hardware accelerator.

[0055] In this embodiment, the data processing window size of the hardware accelerator represents the ideal size of the data unit that the hardware accelerator can efficiently process in a single processing operation, and is usually defined by the accelerator hardware design or driver interface.

[0056] Step S212: Divide the disk data to be processed into multiple data fragments according to the data processing window size; wherein the size of each data fragment is an integer multiple of the data processing window size.

[0057] In this embodiment, the size of each data shard is configured to be an integer multiple of the data processing window size. This sharding strategy, which aligns to integer multiples based on the native hardware processing units, ensures that each shard can be loaded and processed by the hardware accelerator in the most efficient and non-redundant manner. This avoids internal padding or multiple processing overhead caused by misaligned data boundaries, thereby maximizing hardware throughput and laying the foundation for subsequent concurrent submissions.

[0058] This embodiment ensures optimal matching between data fragments and hardware processing units by directly aligning the data partitions to integer multiples of the data processing window size of the hardware accelerator. This not only significantly improves the utilization efficiency of hardware resources and single-processing performance, but also provides a prerequisite for achieving high-concurrency, low-latency pipelined compression / decompression processing, thus laying the foundation for accelerated processing.

[0059] In one specific embodiment of this application, when the method is applied to the export operation of virtual machine disk data (i.e., compression scenario), the data segmentation process based on the hardware accelerator processing capability in step S210 can be further refined into the following steps: Step S213: Obtain the buffer size of the hardware accelerator.

[0060] This parameter represents the upper limit of the internal buffer area used by the hardware accelerator to receive data to be processed, and is usually determined by the hardware specifications or driver configuration.

[0061] Step S214: Adjust the size of each data slice according to the buffer size of the hardware accelerator. The adjusted size of each data slice shall not exceed the buffer size.

[0062] Specifically, based on the initial partitioning of data according to the data processing window size (or combined with window size constraints), the size of each adjusted data partition is ensured to not exceed the buffer size. This adjustment mechanism guarantees that each independent data partition can be completely and simultaneously sent to the input buffer of the hardware accelerator, avoiding buffer overflows, secondary partition splitting, or transmission anomalies caused by excessively large partitions, thereby ensuring the stability and reliability of the hardware processing flow.

[0063] This embodiment effectively adjusts the data fragments by introducing buffer size constraints in the export compression scenario, ensuring that each data fragment strictly matches the input cache capacity of the hardware accelerator. This not only effectively prevents boundary anomalies during data transmission and processing, but also further optimizes the utilization efficiency of hardware resources and the reliability of task scheduling. It is one of the key guarantees for achieving high-speed and stable disk export compression.

[0064] In another embodiment of this application, when the method is applied to the import operation of virtual machine disk data (i.e., decompression scenario), the data segmentation process in step S210 based on the hardware accelerator processing capability, using a recognition and segmentation strategy based on pre-set tags within the file, can be further refined into the following steps: Step S215: Identify the fragment separators between compressed data blocks in the disk data to be processed.

[0065] This delimiter is a specific identifier or metadata written by the system into the compressed file during the previous export compression process to clearly indicate the boundaries of each individual compressed data block.

[0066] Step S216: Divide the disk data to be processed into multiple independent data fragments according to the fragment delimiter.

[0067] Based on the identified fragment delimiters, the disk data (compressed file) to be processed can be accurately divided into multiple independent data fragments, each corresponding one-to-one with the original compressed fragments. This process does not rely on fixed-size calculations but directly parses the structured information stored within the file, thereby achieving accurate restoration of the original fragment structure within the compressed data.

[0068] This embodiment achieves lossless identification and restoration of the original concurrent compression structure by directly parsing and using the pre-set fragment delimiters in the compressed file during the decompression scenario. This method not only provides accurate and efficient segmentation and avoids additional computational overhead, but also provides a suitable data unit for submitting each independent compressed block directly and in parallel to the hardware accelerator for decompression processing.

[0069] In another embodiment provided in this application, in order to fully utilize the parallel processing potential of the hardware accelerator and achieve optimal task scheduling, the process of compressing or decompressing multiple data fragments using the hardware accelerator in step S220 can be further refined into the following dynamic concurrency control steps: Step S221: Query the number of available computing resources for the hardware accelerator.

[0070] Specifically, the number of currently available processing engines, computing queues, or thread resources can be obtained in real time by calling the hardware driver interface or querying the accelerator status register.

[0071] Step S222: Determine the concurrency level of concurrent processing based on the amount of available computing resources.

[0072] The implementation can determine the actual concurrency of the current processing task based on the amount of available computing resources, i.e., the number of data fragments that can be submitted to the hardware accelerator for processing at one time. This determination process usually follows the resource matching principle to ensure that the number of submitted tasks does not exceed the upper limit of the actual parallel processing capacity of the hardware, and can also be dynamically fine-tuned in conjunction with load strategies.

[0073] Step S223: According to the concurrency level, multiple data fragments are submitted concurrently to the hardware accelerator for compression or decompression processing.

[0074] Based on the determined concurrency level, the implementation submits a corresponding number of data shards to the hardware accelerator in parallel, where they simultaneously perform compression or decompression processing. Multiple queues, threads, or request mechanisms can be used during submission to ensure that hardware resources are fully and evenly utilized.

[0075] This embodiment achieves precise matching and dynamic balance between hardware parallelism and software task scheduling by using a dynamic concurrency control mechanism to sense the available computing resources of the hardware accelerator in real time and determine and adjust the task concurrency accordingly. This not only significantly improves the utilization efficiency of hardware resources and the throughput of task processing, but also effectively avoids processing delays or resource idleness caused by overload or underload, thereby ensuring the efficiency and stability of virtual machine disk import and export processing at the system level.

[0076] In one specific embodiment of this application, when the aforementioned multiple data fragments are concurrently submitted to the hardware accelerator for processing, the system attaches a unique sequence identifier to each data fragment. This identifier is used to clarify the original sequential position of each fragment in subsequent processes, and can typically be a monotonically increasing sequence number, timestamp, or other metadata with order-distinguishing properties. In a specific implementation, this sequence identifier can be generated before fragment submission and submitted to the hardware accelerator along with the fragment data as part of the fragment metadata (e.g., written into the fragment header information); alternatively, it can be passed through specific parameters of an interface call, provided the hardware accelerator supports it.

[0077] By attaching a unique sequence identifier to each concurrently submitted data fragment, this embodiment provides an accurate and reliable basis for reassembling potentially out-of-order processing results returned by the hardware accelerator. This mechanism effectively solves the problem of result order uncertainty in high-concurrency processing, ensuring that subsequent merge processing can quickly and error-free restore all fragment results into a complete and continuous data stream. Thus, while fully leveraging the advantages of hardware parallel performance, it strictly guarantees the integrity and correctness of the final output data.

[0078] In another embodiment of this application, the process of merging the compressed or decompressed data fragments in step S220 can be further refined into the following steps based on sequence identifier listening and sorting merging: Step S224: Listen for the processing result returned by the hardware accelerator; wherein the processing result includes compression result or decompression result.

[0079] In this embodiment, the system monitors the processing results returned by the hardware accelerator. These results include compression (in the export scenario) or decompression (in the import scenario) results for each data slice. This monitoring process is typically implemented by polling the hardware completion status register, handling hardware interrupts, or calling asynchronous callback interfaces to ensure that the data is captured and read promptly after the hardware completes processing of any slice. Each returned processing result carries a unique sequence identifier attached to the corresponding data slice at the time of submission.

[0080] Step S225: Based on the sequence identifier carried by the processing results, merge all processing results in order to reconstruct the complete compressed or decompressed data.

[0081] Based on the sequence identifiers carried by each processing result, all returned processing results are sorted and merged according to the original data order, thereby reconstructing complete and continuous compressed or decompressed data. During the merging process, the results can be written to the output buffer or target file sequentially according to the identifier values. If a result is not returned temporarily, a caching mechanism can be used to wait, ensuring strict consistency of the overall order.

[0082] This embodiment achieves accurate and efficient reassembly of the result data stream under high-concurrency processing by actively listening for hardware responses and strictly sorting and merging data according to a preset order identifier. This method fully preserves the processing speed advantages of hardware parallelism while completely resolving the data misalignment problem caused by out-of-order responses, ensuring the integrity and logical correctness of the final output data. It is a key technical guarantee for bridging high-speed concurrent processing and reliable data output.

[0083] In another embodiment provided in this application, the aforementioned hardware accelerator can be a hardware processing unit specifically designed for data compression and decompression. It can be implemented in various physical forms, including but not limited to at least one of the following: a coprocessor integrated within the CPU, such as existing in a SoC (system on a chip) as a dedicated acceleration instruction set or fixed-function unit; or a standalone hardware accelerator card in the form of an expansion card (e.g., a dedicated compression / decompression accelerator card connected to the system via a PCIe (peripheral component interconnect express) interface). Regardless of its specific form, the hardware accelerator possesses a dedicated computing core that directly executes compression and decompression algorithms. It can receive data fragments submitted by the host and process them independently, thereby offloading the computing tasks from the host CPU and improving processing efficiency while saving host resources.

[0084] By explicitly supporting various hardware implementations such as integrated coprocessors and independent hardware accelerator cards, this embodiment enables the acceleration method to flexibly adapt to hardware platforms of different architectures and manufacturers, greatly improving the versatility and deployability of the technical solution. It ensures that existing or dedicated hardware acceleration resources can be effectively utilized in various virtualization environments, thereby achieving stable and efficient virtual machine disk processing acceleration in a wide range of practical application scenarios.

[0085] In another embodiment provided in this application, in order to ensure that the method can still be executed completely and reliably in an environment lacking dedicated hardware acceleration resources, the above step S230 may specifically include the following steps: Step S231: Convert the disk data to be processed into the target format using a disk format conversion tool.

[0086] In this embodiment, when it is detected that no available hardware accelerator is available on the server, the system automatically switches to the backup processing path and directly converts the disk data to be processed into the required target format (such as qcow2, vhd, etc.) by calling a standard disk format conversion tool (such as qemu-img).

[0087] Step S232: During the conversion process, a software compression algorithm is used to compress the disk data to be processed, or the compressed disk data to be processed is decompressed.

[0088] During the aforementioned format conversion process, software compression algorithms (such as gzip, zlib, etc.) can be used simultaneously to compress the disk data to be processed (corresponding to the export scenario), or to decompress the already compressed disk data to be processed (corresponding to the import scenario). This software processing flow can be executed by the server's main CPU to complete the integrated conversion of data format and compression status.

[0089] This embodiment introduces a complete software backup processing path, enabling the disk import / export method to possess strong adaptability and high reliability independent of specific hardware. This design ensures that the system can automatically degrade and complete core processing tasks in various deployment environments (including traditional servers without hardware accelerators), thereby greatly expanding the applicability and deployment flexibility of the method and providing users with continuous and reliable service assurance.

[0090] It should be noted that in the embodiments provided in this application, the compression or decompression of multiple data fragments using a hardware accelerator allows the compression or decompression algorithm to be applied to individual data fragments, rather than the entire disk file to be processed. Furthermore, the embodiments are not limited to a specific compression or decompression algorithm, but can be flexibly selected and adapted according to the actual application scenario, compression ratio requirements, processing speed requirements, and the types of algorithms supported by the hardware accelerator. Commonly available algorithms include, but are not limited to: gzip / zlib algorithms are streaming compression algorithms with high compression ratios, but they are inefficient in traditional serial processing. In this embodiment, data can be divided into independent fragments and submitted concurrently to a hardware accelerator. Each fragment can be independently compressed or decompressed using gzip / zlib, thereby significantly improving processing speed while maintaining a high compression ratio.

[0091] The snappy / lz4 algorithm is a block compression algorithm with a relatively low compression ratio but extremely fast processing speed. It also supports independent processing of blocks and is well adapted to the fragmentation concurrency mechanism of this application embodiment.

[0092] Dedicated algorithms natively supported by hardware accelerators: Hardware accelerators from different manufacturers usually have specific optimized compression algorithms built in. In this embodiment, by calling the hardware driver interface, the fragmented data can be directly submitted to the hardware accelerator, and the hardware firmware or circuitry will perform the compression / decompression operations natively supported by it.

[0093] Taking the export scenario as an example, when compressing the disk data to be processed, each data fragment can be regarded as an independent compression unit. After receiving the fragment, the hardware accelerator executes the compression algorithm internally and outputs the compressed data block. The length of the compressed data block may be smaller than the original fragment length. The hardware accelerator also returns the actual length and status information of the compressed data. The system collects the compression results of all fragments and merges them according to the sequence identifier, finally generating a complete compressed disk file. In addition, preset fragment separators can be inserted between each compressed data block in the compressed file for identifying fragment boundaries during subsequent decompression.

[0094] Taking the import scenario as an example, when decompressing compressed disk data, the system first parses the compressed file, identifies and extracts each independent compressed data block (for example, by splitting it according to a preset split delimiter); submits each compressed data block as an independent fragment to the hardware accelerator; the hardware accelerator executes the corresponding decompression algorithm to restore the original data block; the system merges all the restored data blocks in order to reconstruct the complete original disk data.

[0095] It should be noted that the embodiments of this application do not rely on improvements to a specific compression algorithm. Through a scheduling mechanism of dynamic fragmentation, concurrent submission, and result merging, the hardware accelerator can execute its supported compression / decompression algorithms in the optimal manner. The embodiments of this application maintain good openness and adaptability at the algorithm level, and can flexibly select the optimal compression / decompression algorithm according to hardware capabilities and business needs. At the same time, with the help of the fragmentation concurrency mechanism, even for stream compression algorithms that are traditionally regarded as "serial inefficient" (such as gzip), near-linear performance improvements can be achieved with hardware acceleration support, thereby achieving a better balance between compression ratio and processing speed.

[0096] Figure 5 This is a flowchart illustrating the interaction between software threads, accelerator drivers, and hardware accelerators during data processing, as provided in the embodiments of this application. (Combined with...) Figure 5 As shown, the process mainly includes two threads working together: a data submission thread and a data receiving thread. The two threads interact with the underlying accelerator hardware through the accelerator driver to jointly complete the accelerated data processing.

[0097] The data submission thread is responsible for preprocessing and task scheduling: first, it queries the available resources and buffer size of the accelerator; then, based on the query results, it divides the data to be processed into appropriately sized fragments and controls the amount of data in the submission queue to ensure it does not exceed the hardware processing capacity. Processing tasks are asynchronously submitted to the hardware for execution via the accelerator driver, which is responsible for issuing specific compression or decompression commands to the accelerator hardware.

[0098] The data receiving thread is responsible for result collection and post-processing: it continuously listens for the processing results returned by the accelerator hardware, and after receiving the results, it merges the data according to the order information carried in the results, and finally reconstructs them into complete and ordered output data.

[0099] After receiving instructions from the accelerator driver, the accelerator hardware performs the actual compression or decompression calculations and returns the processed results to the accelerator driver, which is then received by the data receiving thread.

[0100] This process achieves efficient parallel execution and optimized resource utilization of data processing tasks through dual-thread collaboration, resource-aware task scheduling, and asynchronous hardware processing mechanisms, thereby improving processing throughput while ensuring the integrity and correct order of the result data.

[0101] It should be noted that, in Figure 5 In the illustrated embodiment, the specific process by which the data receiving thread listens for and merges the returned results is as follows: (1) Implementation method of listening to the returned result.

[0102] Completion status awareness mechanism: The data receiving thread continuously monitors the processing completion status of the hardware accelerator through the interface provided by the accelerator driver. This is usually achieved in two ways: (a) Polling query: periodically read the status register or completion queue maintained by the accelerator hardware or driver to check if any new processing results are ready; (b) Event notification: register a callback function or wait for kernel events. When the accelerator completes the processing of any data slice, the driver actively triggers an interrupt or sends an asynchronous notification to wake up the receiving thread and obtain the results in a timely manner.

[0103] Result data acquisition: Once a processed shard is confirmed, the thread reads the corresponding result data and associated metadata (including the unique sequence identifier attached to the shard when it was submitted, processing status, result length, etc.) through the driver interface, and temporarily stores it in the thread's receive buffer or ordered queue.

[0104] (2) Implementation method of merging processing result data.

[0105] Reorganization by Identifier: The thread maintains a reorganization buffer (such as an array or mapping structure) organized by sequential identifiers. Each received result fragment is placed in the corresponding position in the buffer according to its sequential identifier, ensuring that the fragments are logically aligned according to their original submission order.

[0106] Streaming merging and output: Threads check the buffer in identifier order (starting from the smallest sequence number). If the result of a fragment for a certain sequence number is ready, its data content is immediately appended to the final output file or data stream, and the buffer entry is released; if a discontinuous sequence number is detected (e.g., a fragment has not yet returned), the thread can temporarily store the subsequent ready fragments, wait for the missing fragments to arrive, and then continue outputting in order, or perform timeout processing according to the configured policy.

[0107] Integrity guarantee: After all the fragment results have been received and merged, the thread verifies the integrity and consistency of the output data, finally generates a complete processed data file, and notifies the upstream module that the processing flow has ended.

[0108] This thread obtains hardware processing results in real time through a combination of active listening and event-driven methods, and reassembles the fragmented data in memory in an orderly manner with the help of preset sequence identifiers, so as to achieve efficient and accurate streaming merging, thereby minimizing the additional latency caused by result return and merging while ensuring data integrity.

[0109] It should be understood that the technical solutions described in the foregoing embodiments have good scalability and adaptability in terms of application scenarios and implementation methods.

[0110] In terms of application scenarios, the embodiments provided in this application are also applicable to various private clouds, industry clouds, and hybrid cloud platforms. In these scenarios, the rapid import, export, migration, and backup of virtual machine disks are also critical requirements. The hardware-accelerated dynamic sharding concurrent processing mechanism provided in the embodiments of this application can significantly improve data flow efficiency and reduce host load, thereby providing a universal disk processing acceleration solution for a wider range of cloud environments.

[0111] At the hardware implementation level, the hardware accelerators referred to in this application are not limited to dedicated coprocessors built into the CPU. They are also applicable to independent hardware accelerator cards or dedicated compression cards connected to the server via an expansion bus (such as PCIe). These devices typically have stronger processing power, higher concurrency, and richer support for compression algorithms. Their interaction with the host system, task submission, and result return mechanisms are similar in principle to integrated accelerators, and can all be incorporated into the "detection-segmentation-concurrent processing-merging" process framework of this application's embodiments through corresponding device drivers to achieve hardware offloading and acceleration of compression / decompression tasks.

[0112] In summary, the embodiments of this application aim to construct a general acceleration framework that integrates hardware awareness, dynamic scheduling, concurrent processing, and result reorganization. This framework can be horizontally extended to various virtualization and cloud environments, and vertically adapted to different forms of dedicated processing units, from integrated coprocessors to standalone accelerator cards, thus maintaining broad applicability and vitality in technological evolution and market applications.

[0113] By dividing each function into corresponding functional modules, this application provides a virtual machine disk data import / export device, which can be a chip used in a server. Figure 6 A schematic block diagram of the functional modules of a virtual machine disk data import / export device provided for an exemplary embodiment of this application. For example... Figure 6 As shown, the disk data import / export device for this virtual machine includes: The acceleration processing module 61 is used to divide the disk data to be processed into multiple data fragments according to the processing capacity of the hardware accelerator when the server detects that there is an available hardware accelerator. The first data processing module 62 is used to compress or decompress multiple data fragments through a hardware accelerator, and to merge the compressed or decompressed data fragments to generate complete processed data.

[0114] The second data processing module 63 is used to perform pure software compression or decompression of the disk data to be processed by the CPU when it is detected that no available hardware accelerator is available on the server, and to generate complete processed data.

[0115] This embodiment detects available hardware accelerators on the server, segments the data based on their processing capabilities, and processes multiple data segments in parallel using the hardware accelerators. This efficiently offloads compression or decompression tasks during virtual machine disk import / export to dedicated hardware. When no available hardware accelerators are detected on the server, the CPU performs pure software compression or decompression on the disk data to be processed. This device not only significantly improves disk format conversion and data processing speed but also greatly reduces the computational burden on the host server, effectively supporting rapid disk migration, backup, and sharing operations in various virtualization environments.

[0116] In another embodiment provided in this application, the acceleration processing module 61 is specifically used for: Get the size of the data processing window for the hardware accelerator; Based on the data processing window size, the disk data to be processed is divided into multiple data fragments; the size of each data fragment is an integer multiple of the data processing window size.

[0117] This embodiment ensures optimal matching between data fragments and hardware processing units by directly aligning the data partitions to integer multiples of the data processing window size of the hardware accelerator. This not only significantly improves the utilization efficiency of hardware resources and single-processing performance, but also provides a prerequisite for achieving high-concurrency, low-latency pipelined compression / decompression processing, thus laying the foundation for accelerated processing.

[0118] In another embodiment provided in this application, when exporting disk data to be processed, the acceleration processing module 61 is further configured to: Get the buffer size of the hardware accelerator; The size of each data slice is adjusted according to the buffer size of the hardware accelerator; wherein the adjusted size of each data slice does not exceed the buffer size.

[0119] This embodiment effectively adjusts the data fragments by introducing buffer size constraints in the export compression scenario, ensuring that each data fragment strictly matches the input cache capacity of the hardware accelerator. This not only effectively prevents boundary anomalies during data transmission and processing, but also further optimizes the utilization efficiency of hardware resources and the reliability of task scheduling. It is one of the key guarantees for achieving high-speed and stable disk export compression.

[0120] In another embodiment provided in this application, the acceleration processing module 61 is further used for: Identify the fragment delimiters between compressed data blocks in the disk data to be processed; The disk data to be processed is divided into multiple independent data fragments based on the fragment delimiter.

[0121] This embodiment achieves lossless identification and restoration of the original concurrent compression structure by directly parsing and using the pre-set fragment delimiters in the compressed file during the decompression scenario. The device not only has accurate and efficient segmentation and avoids additional computational overhead, but also provides a suitable data unit for submitting each independent compressed block directly and in parallel to the hardware accelerator for decompression processing.

[0122] In another embodiment provided in this application, the first data processing module 62 is specifically used for: Queries the amount of available computing resources for the hardware accelerator; Determine the concurrency level for concurrent processing based on the amount of available computing resources; Based on the concurrency level, multiple data fragments are submitted concurrently to the hardware accelerator for compression or decompression.

[0123] This embodiment achieves precise matching and dynamic balance between hardware parallelism and software task scheduling by using a dynamic concurrency control mechanism to sense the available computing resources of the hardware accelerator in real time and determine and adjust the task concurrency accordingly. This not only significantly improves the utilization efficiency of hardware resources and the throughput of task processing, but also effectively avoids processing delays or resource idleness caused by overload or underload, thereby ensuring the efficiency and stability of virtual machine disk import and export processing at the system level.

[0124] In another embodiment provided in this application, when multiple data fragments are submitted concurrently to a hardware accelerator for processing, a unique sequence identifier is attached to each data fragment.

[0125] By attaching a unique sequence identifier to each concurrently submitted data fragment, this embodiment provides an accurate and reliable basis for reassembling potentially out-of-order processing results returned by the hardware accelerator. This mechanism effectively solves the problem of result order uncertainty in high-concurrency processing, ensuring that subsequent merge processing can quickly and error-free restore all fragment results into a complete and continuous data stream. Thus, while fully leveraging the advantages of hardware parallel performance, it strictly guarantees the integrity and correctness of the final output data.

[0126] In another embodiment provided in this application, the first data processing module 62 is further configured to: Listen for the processing results returned by the hardware accelerator, including compression or decompression results; Based on the sequence identifier carried by the processing results, all processing results are merged in order to reconstruct complete compressed or decompressed data.

[0127] This embodiment achieves accurate and efficient reassembly of the result data stream under high-concurrency processing mode by actively listening to hardware responses and strictly sorting and merging according to preset order identifiers. This device fully retains the processing speed advantages brought by hardware parallelism while completely solving the data misalignment problem caused by out-of-order responses, ensuring the integrity and logical correctness of the final output data.

[0128] In another embodiment provided in this application, the above-mentioned hardware accelerator is hardware for data compression and decompression processing, and the hardware accelerator includes at least one of the following: a coprocessor integrated into the central processing unit or a separate hardware accelerator card.

[0129] By explicitly supporting various hardware implementations such as integrated coprocessors and independent hardware accelerator cards, this embodiment enables the acceleration device to flexibly adapt to hardware platforms of different architectures and manufacturers, greatly improving the versatility and deployability of the technical solution. It ensures that existing or dedicated hardware acceleration resources can be effectively utilized in various virtualization environments, thereby achieving stable and efficient virtual machine disk processing acceleration in a wide range of practical application scenarios.

[0130] In another embodiment provided in this application, the second data processing module 63 is specifically used for: Use a disk format conversion tool to convert the disk data to the target format; During the conversion process, software compression algorithms are used to compress the disk data to be processed, or to decompress the already compressed disk data to be processed.

[0131] This embodiment introduces a complete software backup processing path, enabling the disk import / export device to possess strong adaptability and high reliability independent of specific hardware. This design ensures that the system can automatically degrade and complete core processing tasks in various deployment environments (including traditional servers without hardware accelerators), thereby greatly expanding the device's applicability and deployment flexibility, and providing users with continuous and reliable service assurance.

[0132] This application also provides a computing device, such as... Figure 7 As shown, Figure 7 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. The computing device can specifically be the server mentioned above, including a processor 1901, a communication interface 1902, a memory 1903, and a communication bus 1904. The processor 1901, the communication interface 1902, and the memory 1903 communicate with each other through the communication bus 1904.

[0133] Memory 1903 is used to store computer programs; The processor 1901, when executing the program stored in the memory 1903, implements the method described above in the embodiments of this application.

[0134] The communication bus mentioned in the computing device above can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0135] The communication interface is used for communication between the aforementioned computing device and other devices.

[0136] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0137] The processors mentioned above can be general-purpose processors, including CPUs, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0138] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the methods described above in the embodiments of this application.

[0139] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute the methods described above in the embodiments of this application.

[0140] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0141] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0142] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, computing devices, and computer-readable storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0143] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for importing and exporting disk data in a virtual machine, characterized in that, The method includes: When a usable hardware accelerator is detected on the server, the disk data to be processed is divided into multiple data fragments according to the processing capability of the hardware accelerator. The hardware accelerator is used to compress or decompress the multiple data fragments, and the compressed or decompressed data fragments are merged to generate complete processed data. When no available hardware accelerator is detected on the server, the CPU performs pure software compression or decompression on the disk data to be processed, and generates complete processed data.

2. The method according to claim 1, characterized in that, The step of dividing the disk data to be processed into multiple data fragments based on the processing capability of the hardware accelerator includes: Obtain the size of the data processing window of the hardware accelerator; Based on the data processing window size, the disk data to be processed is divided into multiple data fragments; wherein the size of each data fragment is an integer multiple of the data processing window size.

3. The method according to claim 2, characterized in that, When exporting the disk data to be processed, the step of dividing the disk data into multiple data fragments according to the processing capability of the hardware accelerator further includes: Obtain the buffer size of the hardware accelerator; The size of each data slice is adjusted according to the buffer size of the hardware accelerator; wherein the adjusted size of each data slice does not exceed the buffer size.

4. The method according to claim 1, characterized in that, When importing the disk data to be processed, the step of dividing the disk data into multiple data fragments according to the processing capability of the hardware accelerator further includes: Identify the fragmentation delimiters between compressed data blocks in the disk data to be processed; The disk data to be processed is divided into multiple independent data fragments according to the fragment delimiter.

5. The method according to claim 1, characterized in that, The compression or decompression process of the multiple data fragments using the hardware accelerator includes: Query the amount of available computing resources for the hardware accelerator; The concurrency level of concurrent processing is determined based on the amount of available computing resources. According to the concurrency level, multiple data fragments are submitted concurrently to the hardware accelerator for compression or decompression processing.

6. The method according to claim 5, characterized in that, When the multiple data shards are submitted concurrently to the hardware accelerator for processing, a unique sequence identifier is attached to each data shard.

7. The method according to claim 6, characterized in that, The merging process of the various data fragments after compression or decompression includes: Listen to the processing results returned by the hardware accelerator, including compression results or decompression results; Based on the sequence identifier carried by the processing results, all processing results are merged in order to reconstruct complete compressed or decompressed data.

8. The method according to claim 1, characterized in that, The hardware accelerator is hardware used for data compression and decompression processing, and the hardware accelerator includes at least one of the following: a coprocessor integrated into the central processing unit or a separate hardware accelerator card.

9. The method according to claim 1, characterized in that, The process of compressing or decompressing the disk data to be processed via CPU using pure software includes: The disk data to be processed is converted to the target format using a disk format conversion tool. During the conversion process, software compression algorithms are used to compress the disk data to be processed, or the already compressed disk data to be processed is decompressed.

10. A computing device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-9.

Images