Data copying method and related device

Abstract

A data copying method is applied to execution equipment capable of being externally connected with an external storage medium. In the process of executing data copying aiming at the external storage medium, data is copied from a local storage medium to a cache region of a memory, and after the data which is cached in the memory and is to be copied to the external storage medium reaches a certain data volume, a write-back thread corresponding to the external storage medium is started at the same time; therefore, the process of copying the data from the local storage medium to the memory and the process of copying the data from the memory to the external storage medium can be executed in parallel, the copying speed of the data from the local storage medium to the external storage medium is increased, and the data copying performance is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data copying method and related apparatus. Background Technology

[0002] With the development of computer technology, personal computers and laptops have become indispensable in people's work and life. In daily life, people usually use computers to handle work tasks, surf the Internet, or play entertainment games.

[0003] Generally, computers use local storage media such as hard disk drives (HDDs) or solid-state drives (SSDs) to persistently store user data. However, with the rise of mobile work, users often need to continue processing or storing data on other computers. In these cases, users often need to copy data from their computers to more portable storage media (such as external hard drives or memory cards) so that they can access the data anywhere using these portable storage media.

[0004] Currently, external storage media such as portable hard drives or memory cards are connected to computers via a Universal Serial Bus (USB) interface. Furthermore, due to limited hardware bandwidth, copying local data from the computer to external storage media often takes a considerable amount of time, potentially impacting the user experience. Summary of the Invention

[0005] This application provides a data copying method and related apparatus, which can improve the performance of copying data to an external storage medium.

[0006] In a first aspect, a data copying method is provided, applied to an execution device capable of connecting to an external storage medium. The data copying method includes: the execution device acquiring a data copying request, the data copying request instructing the copying of target data from a local storage medium to an external storage medium.

[0007] Then, based on the data copy request, the execution device copies the data from the local storage medium to the cache area in memory. The cache area is a region allocated by the operating system for external storage media (e.g., a file cache area) to cache data waiting to be copied to the external storage medium.

[0008] Secondly, when the amount of data cached in the cache area reaches the first threshold, the execution device will start the write-back thread corresponding to the external storage medium, and write the data in the cache area to the external storage medium through the write-back thread. Notably, writing the data in the cache area to the external storage medium through the write-back thread will not block the copying of data from the local storage medium to the cache area in memory.

[0009] In this solution, during the data copying process for the external storage medium, the data is first copied from the local storage medium to the memory cache area. After the amount of data to be copied to the external storage medium cached in memory reaches a certain amount, the write-back thread corresponding to the external storage medium is started simultaneously. This allows the processes of copying data from the local storage medium to memory and from memory to the external storage medium to be executed in parallel, thereby improving the data copying speed from the local storage medium to the external storage medium and enhancing data copying performance.

[0010] In one possible implementation, the execution device copies data from the local storage medium to the memory cache area in parallel with the write-back thread writing data from the cache area to the external storage medium. That is, after the execution device begins writing data from the cache area to the external storage medium via the write-back thread, it continues copying data from the local storage medium to the memory cache area, thus making the processes of copying data from the local storage medium to memory and from memory to the external storage medium parallel.

[0011] In one possible implementation, when the amount of data cached in the cache region but not written to the external storage medium reaches a second threshold, the execution device suspends copying data from the target data from the local storage medium to the cache region of memory, wherein the second threshold is greater than or equal to the first threshold.

[0012] In other words, when the amount of data to be copied cached in the cache area reaches a certain amount, it can be considered that enough data to be copied has been cached in memory. The execution device will find it difficult to quickly copy the data already cached in memory to the external storage medium. At this time, the execution device can temporarily pause the copying of data from the target data from the local storage medium to the cache area of ​​memory in order to avoid caching too much data to be copied in memory at once.

[0013] In one possible implementation, after pausing the copying of data from the target data from the local storage medium to the cache area for a preset time, the execution device continues to copy the uncopied data from the target data from the local storage medium to the cache area.

[0014] In other words, the execution device only briefly pauses the copying of data from the target data from the local storage medium to the cache area. The execution device will then continue to copy the uncopyable data from the target data from the local storage medium to the cache area, thereby ensuring that the data copying process can continue to execute and avoiding affecting the normal copying of data to the external storage medium.

[0015] In one possible implementation, the execution device pauses copying data from the local storage medium to the cache area for a preset duration, which is related to the speed at which data in the cache area is written to the external storage medium.

[0016] In this solution, by setting the preset duration for pausing data copying on the execution device to be related to the speed at which data in the cache area is written to the external storage medium, it can be ensured that the execution device pauses for an appropriate duration each time, avoiding the impact of pausing data copying for too long on the normal data copying or the excessive performance overhead caused by pausing data copying for too short a time.

[0017] In one possible implementation, the second threshold for triggering a pause in data copying is determined based on a pre-configured threshold and / or the speed at which data in the cache area is written to the external storage medium.

[0018] In one possible implementation, after copying all data from the target data from the local storage medium to the cache area, the execution device writes the data in the cache area to the external storage medium by executing the fsync system call. Furthermore, after writing all data from the cache area to the external storage medium, the execution device returns a success message to the initiator of the fsync system call.

[0019] In one possible implementation, the initiator of the fsync system call initiates the fsync system call only after all data in the target data has been copied from the local storage medium to the cache area.

[0020] In one possible implementation, the write-back thread is configured separately for the external storage medium when it is accessed.

[0021] In one possible implementation, the external storage medium includes a portable hard drive, a Universal Serial Bus (USB) flash drive, or a memory card.

[0022] Secondly, a data copying device is provided, comprising: an acquisition module for acquiring a data copying request, the data copying request indicating that target data in a local storage medium be copied to an external storage medium; a processing module for copying data from the target data from the local storage medium to a cache area in memory based on the data copying request, the cache area being used to cache the data to be copied to the external storage medium; the processing module is further configured to, when the amount of data cached in the cache area reaches a first threshold, start a write-back thread corresponding to the external storage medium, and write the data in the cache area to the external storage medium through the write-back thread.

[0023] In one possible implementation, the processing module copies data from the target data from the local storage medium to the cache area in memory, which is performed in parallel with the write-back thread writing the data in the cache area to the external storage medium.

[0024] In one possible implementation, the processing module is further configured to: when the amount of data cached in the cache area and not written to the external storage medium reaches a second threshold, suspend copying data from the target data from the local storage medium to the cache area in memory, wherein the second threshold is greater than or equal to the first threshold.

[0025] In one possible implementation, the processing module is further configured to: after pausing the copying of data from the target data from the local storage medium to the cache area for a preset time, continue copying the uncopyable data from the target data from the local storage medium to the cache area.

[0026] In one possible implementation, the preset duration is related to the speed at which data in the cache area is written to the external storage medium.

[0027] In one possible implementation, the second threshold is determined based on a pre-configured threshold and / or the speed at which data in the cache region is written to the external storage medium.

[0028] In one possible implementation, the processing module is further configured to: after copying all data in the target data from the local storage medium to the cache area, write the data in the cache area to the external storage medium by executing the fsync system call; and after writing all data in the cache area to the external storage medium, return a system call success message to the initiator of the fsync system call.

[0029] In one possible implementation, the initiator only initiates the fsync system call after copying all data from the target data from the local storage medium to the cache area.

[0030] In one possible implementation, the write-back thread is configured separately for the external storage medium when it is accessed.

[0031] A third aspect of this application provides a data copying apparatus, which may include a processor and a memory coupled together. The memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method described in the first aspect or any implementation thereof is implemented. For details regarding the steps in the various possible implementations of the first aspect executed by the processor, please refer to the first aspect; further details will not be repeated here.

[0032] The fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method of any implementation of the first aspect described above.

[0033] The fifth aspect of this application provides a circuit system including a processing circuit configured to perform the method of any implementation of the first aspect described above.

[0034] The sixth aspect of this application provides a computer program product that, when run on a computer, causes the computer to perform any implementation of the first aspect described above.

[0035] A seventh aspect of this application provides a chip system including a processor for implementing the functions involved in any implementation of the first aspect, such as processing data and / or information involved in the methods described above. In one possible design, the chip system further includes a memory for storing necessary program instructions and data. This chip system may be composed of chips or may include chips and other discrete devices.

[0036] The beneficial effects of the second to seventh aspects mentioned above can be referred to the introduction of the first aspect above, and will not be repeated here. Attached Figure Description

[0037] Figure 1 This is a schematic diagram illustrating how external storage media handle I / O requests in related technologies.

[0038] Figure 2 This application provides a schematic diagram of the structure of an execution device 101;

[0039] Figure 3 A flowchart illustrating a data copying method provided in this application;

[0040] Figure 4 A schematic diagram illustrating the copying of data from a local storage medium to an external storage medium, as provided in this application;

[0041] Figure 5 A schematic diagram illustrating a method for pausing the copying of data from local storage medium to memory, as provided in this application;

[0042] Figure 6 A schematic diagram illustrating a method for recovering data copied from local storage medium to memory, as provided in this application;

[0043] Figure 7 A schematic diagram illustrating a data copying process provided in this application;

[0044] Figure 8 This application provides a schematic diagram of a process for copying data from a local storage medium to an external storage medium.

[0045] Figure 9 A schematic diagram of a data copying device provided in this application;

[0046] Figure 10 A schematic diagram of the structure of an execution device provided in this application;

[0047] Figure 11 This is a schematic diagram of the structure of a computer-readable storage medium provided in this application. Detailed Implementation

[0048] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0049] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein.

[0050] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or apparatus.

[0051] For ease of understanding, some technical terms involved in the embodiments of this application will be introduced below.

[0052] (1) Dirty Page

[0053] In operating system memory management, a dirty page refers to a memory page that has been modified but has not yet been written to disk or other persistent storage media. Generally, when the data in a memory page is modified, the kernel marks the memory page as "dirty" (i.e., the dirty bit is set to 1) through the page table or related data structures, indicating that the data in the memory page is inconsistent with the original data on the disk.

[0054] (2) Memory Page

[0055] In a computer, the operating system typically divides memory into fixed-size blocks, such as 4 kilobytes (KB). These individual blocks of memory are called memory pages and are used for data exchange with the disk.

[0056] (3) Write-back thread

[0057] The write-back thread is a background thread maintained by the operating system kernel that periodically or based on memory status writes data from dirty pages in memory to disk. For example, the kernel can automatically trigger the write-back thread based on preset policies (such as scheduled tasks, insufficient memory, or the proportion of dirty pages exceeding a threshold).

[0058] Furthermore, the write-back thread operates in a non-blocking manner, meaning the write-back process is handled asynchronously by the kernel and does not block the user process; this is a delayed persistence mechanism.

[0059] (4) fsync system call

[0060] The fsync system call is a system call in the operating system used to force the synchronization of file contents to physical storage devices. Its core function is to ensure that the data written to disk by the application is truly persistent, rather than just remaining in memory.

[0061] Specifically, to improve I / O efficiency, operating systems typically cache data requested for write operations in memory, waiting for a certain period to be asynchronously written to disk in batches. However, caching data in memory can lead to a situation where, after an application successfully calls a write operation, the data may not actually be written to disk. If the device crashes or loses power, the cached data will be lost. Therefore, the fsync system call forces the operating system to flush all dirty pages and metadata (such as file size and modification time) corresponding to the specified file descriptor to disk and waits for the physical write to complete before returning, ensuring that data is not lost in the event of a system crash.

[0062] It should be noted that the fsync system call is a synchronous call and will block the current thread until the disk operation is complete.

[0063] (5) Input / Output (IO) Request

[0064] An I / O request is an instruction sent by the operating system kernel to a storage device (such as a hard drive or USB flash drive) to perform a read or write operation on data. An I / O request typically includes: the destination address (i.e., the logical block address of the device or the offset in the file system), the data size (i.e., the amount of data transferred), and the operation type (i.e., read or write).

[0065] (6) Memory card

[0066] Memory cards are independent storage media used in mobile phones, digital cameras, laptops, and other digital products. They are generally in the form of cards, hence the general term "memory card." For example, a memory card can specifically be a Secure Digital (SD) card.

[0067] (7) Universal Serial Bus flash disk (USB flash drive)

[0068] A USB flash drive is a miniature, high-capacity portable storage product that uses a USB interface and requires no physical drive. It connects to a computer via the USB interface, enabling plug-and-play functionality.

[0069] (8) External hard drive

[0070] A portable hard drive is a portable data storage device that connects to a computer via a USB interface. It can be plugged in or unplugged at any time and can transfer data with the computer at a high speed.

[0071] (9) File Explorer

[0072] File Explorer is a system service responsible for managing persistent or continuous data in databases, persistent message queues, or transactional file systems. It stores data and performs fault recovery. Based on File Explorer, users can perform various operations on files, such as opening, copying, and moving them.

[0073] The applicant's research revealed that in related technologies, when copying local data stored on a computer to external storage media, upper-level applications (such as file explorers) typically first copy a portion of the local data to the computer's memory for caching, and then use the `fsync` system call to copy the cached data from memory to the external storage media. After successfully copying the cached data from memory to the external storage media, the file explorer continues to copy data from the local storage media to the computer's memory for caching, so that subsequent calls to the `fsync` system call can be made to copy data from memory to the external storage media. For example, after copying 32 megabytes (MB) of data from the local storage media to memory, the file explorer calls the `fsync` system call once to copy this 32MB of data from memory to the external storage media; after 32MB of data is successfully copied to the external storage media, the file explorer continues to copy another 32MB of data from the local storage media to memory, and so on, repeating the process of copying data from local storage media to memory and then from memory to external storage media.

[0074] However, because the fsync system call is a synchronous call, it blocks the file explorer currently calling the fsync system call until the disk operation is complete. Therefore, while copying data from memory to external storage via the fsync system call, the file explorer cannot copy data from local storage to its memory cache. In other words, copying data from local storage to memory and copying cached data from memory to external storage are performed serially.

[0075] For example, please refer to Figure 1 , Figure 1 This is a schematic diagram illustrating the handling of I / O requests by external storage media in related technologies. For example... Figure 1 As shown, in related technologies, the number of I / O requests processed by the external storage medium during the data copying process can be collected to obtain the I / O request processing status of the external storage medium over a period of time. Clearly, the external storage medium exhibits intermittent I / O request processing, meaning there are periods when the external storage medium experiences no I / O request processing. Specifically, the period when the external storage medium centrally processes I / O requests is actually the period when the file explorer calls the fsync system call to copy data from memory to the external storage medium. This is because after the file explorer calls the fsync system call, it triggers the operating system kernel to issue an I / O request to the external storage medium. When the fsync system call returns successfully, the file explorer continues to copy data from the local storage medium to memory for caching. During this time, the external storage medium does not process any I / O requests; that is, the data cached in memory is not simultaneously copied to the external storage medium.

[0076] Based on this, this application provides a data copying method. Referring to the idea of ​​configuring write-back threads for local storage media in operating systems, a corresponding write-back thread is also configured for external storage media connected to the local device. Furthermore, during the data copying process for the external storage media, data is first copied from the local storage media to the memory cache area. Once the amount of data to be copied to the external storage media cached in memory reaches a certain level, the corresponding write-back thread for the external storage media is simultaneously started to write the data back to the external storage media. This allows the processes of copying data from the local storage media to memory and from memory to the external storage media to execute in parallel, thereby improving the data copying speed from the local storage media to the external storage media and enhancing data copying performance. In addition, this solution is essentially based on the existing write-back thread concept of the operating system, with additional write-back logic configured for the external storage media. This ensures that data to be copied from memory to the external storage media can be copied to the external storage media in a timely manner, thus improving the data copying speed of the external storage media while minimizing modifications to existing technologies.

[0077] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of an execution device 101 provided in this application. Figure 2 As shown, the data copying method provided in this application can be applied to the execution device 101. The execution device 101 is, for example, a personal computer, a laptop computer, or a server.

[0078] Specifically, execution device 101 includes processor 103, which is coupled to system bus 105. Processor 103 can be one or more processors, each of which can include one or more processor cores. A video adapter 107 drives a display 109, which is coupled to system bus 105. System bus 105 is coupled to input / output (I / O) bus via bus bridge 111. I / O interface 115 is coupled to the I / O bus. I / O interface 115 communicates with various I / O devices, such as input device 117 (e.g., touchscreen), external storage 121 (e.g., hard disk, floppy disk, optical disk, or USB flash drive), multimedia interface, transceiver 123 (capable of sending and / or receiving radio communication signals), camera 155 (capable of capturing still and moving digital video images), and external USB port 125. Optionally, the interface connected to I / O interface 115 can be a USB interface.

[0079] The processor 103 can be any conventional processor, including a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, or a combination thereof. Optionally, the processor can be a special-purpose device such as an ASIC.

[0080] The execution device 101 can communicate with the software deployment server 149 via network interface 129. Exemplarily, network interface 129 is a hardware network interface, such as a network interface card (NIC). Network 127 can be an external network, such as the Internet, or an internal network, such as Ethernet or a virtual private network (VPN). Optionally, network 127 can also be a wireless network, such as a WiFi network or a cellular network.

[0081] Hard disk drive interface 131 is coupled to system bus 105. Hardware driver interface is connected to hard disk drive 133. Internal memory 135 is coupled to system bus 105. Data running in internal memory 135 may include operating system (OS) 137, applications 143, and schedules of execution device 101.

[0082] An operating system consists of the Shell 139 and the kernel 141. The Shell 139 is an interface between the user and the operating system kernel. The shell is the outermost layer of the operating system. The shell manages the interaction between the user and the operating system: waiting for user input, interpreting user input for the operating system, and processing various operating system outputs.

[0083] Kernel 141 consists of the parts of the operating system used to manage memory, files, peripherals, and system resources. Kernel 141 interacts directly with the hardware. The operating system kernel typically runs processes and provides inter-process communication, CPU time-slice management, interrupts, memory management, I / O management, and so on.

[0084] Please see Figure 3 , Figure 3 This is a flowchart illustrating a data copying method provided in this application. Figure 3 As shown, the data copying method includes the following steps 301-303.

[0085] Step 301: Obtain a data copy request. The data copy request is used to instruct the target data in the local storage medium to be copied to the external storage medium.

[0086] In this application, the execution device is equipped with a local storage medium, which may be a persistent storage medium such as a solid-state drive or a hard disk drive. Furthermore, the execution device is connected to an external storage medium, which may include portable persistent storage media such as external hard drives, USB flash drives, or memory cards.

[0087] When a user needs to copy data from the local storage medium of the execution device to an external storage medium, the user can send a data copy request to the execution device. This data copy request instructs the target data to be copied from the local storage medium to the external storage medium. Specifically, the target data can be, for example, one or more files stored on the local storage medium. Therefore, the user can perform operations such as copy and paste, or cut on the execution device to send a data copy request to an upper-level application (such as a file explorer) on the execution device.

[0088] Step 302: Based on the data copy request, copy the data in the target data from the local storage medium to the cache area in memory. The cache area is used to cache the data to be copied to the external storage medium.

[0089] Upon receiving a data copy request, the file explorer on the device triggers the copying of data from the local storage medium to the memory cache area based on the data copy request. The memory cache area is a region allocated by the operating system for external storage media (e.g., a file cache area) to cache data waiting to be copied to the external storage medium.

[0090] It should be noted that after the execution device copies the data from the target data from the local storage medium to the memory cache area, it marks the memory page in memory that stores the data from the target data as a dirty page, and marks the data on that memory page as needing to be copied to the external storage medium.

[0091] Step 303: When the amount of data cached in the cache area reaches the first threshold, start the write-back thread corresponding to the external storage medium, and write the data in the cache area to the external storage medium through the write-back thread.

[0092] As the file resource management in the execution device continuously copies data from the local storage medium to the memory cache area, the amount of data cached in the cache area gradually increases. When the amount of data cached in the cache area reaches a first threshold (i.e., the amount of data is greater than or equal to the first threshold), it means that the cache area has cached more data than the first threshold waiting to be copied to the external storage medium. At this time, the operating system in the execution device starts the write-back thread corresponding to the external storage medium. This write-back thread belongs to the operating system kernel level. After being started, the write-back thread will continuously write the data to be copied from the cache area to the external storage medium. For example, the write-back thread identifies memory pages in memory that are marked as needing to be written to the external storage medium, and can then send IO requests to the external storage medium to instruct that the data on these memory pages be written to the external storage medium.

[0093] The aforementioned first threshold can be a pre-set threshold, such as 16MB or 32MB, and this application does not specifically limit it. Generally, by setting a first threshold, it can be ensured that the write-back thread configured by the execution device for the external storage medium is in a dormant state when idle, that is, it will only be awakened when the data cached in the cache area reaches the first threshold, thereby reducing performance consumption. In addition, in order to ensure that the write-back thread is awakened in a timely manner to improve the data copying speed, the size of the first threshold can be determined based on the speed at which data is copied from the local storage medium to memory. For example, when the speed at which data is copied from the local storage medium to memory is high, the first threshold can be set to a larger value; when the speed at which data is copied from the local storage medium to memory is low, the first threshold can be set to a smaller value.

[0094] Furthermore, the execution device copies data from the target data from the local storage medium to the memory cache area, while the write-back thread writes the data from the cache area to the external storage medium in parallel.

[0095] In other words, once the amount of data cached in the cache area reaches a certain threshold, the execution device will start writing the data in the cache area to the external storage medium through the write-back thread. At this time, the execution device will continue to copy the data in the target data from the local storage medium to the cache area in memory, so that the process of copying data from the local storage medium to memory and from memory to the external storage medium is parallel.

[0096] For example, please refer to Figure 4 , Figure 4 This application provides a schematic diagram illustrating the copying of data from a local storage medium to an external storage medium. For example... Figure 4As shown, after receiving a data copy request, the execution device first copies the data from the local storage medium to the cache area in memory. Once the data cached in the memory area reaches a first threshold, the execution device starts a write-back thread, thereby continuously copying data from the local storage medium to the memory cache area while simultaneously writing the data in the memory cache area back to the external storage medium, thus achieving data copying to the external storage medium.

[0097] It should be noted that in related technologies, after a certain amount of data is copied from the local storage medium to memory, the file explorer calls the fsync system call to copy the cached data in memory to the external storage medium. However, in this application, since the copying of data from the memory cache to the external storage medium is performed by a write-back thread, the file explorer in the execution device no longer needs to trigger the fsync system call after copying a certain amount of data from the local storage medium to memory. Instead, it continuously copies data from the local storage medium to memory, thereby ensuring that the processes of copying data from the local storage medium to memory and from memory to the external storage medium can be executed in parallel.

[0098] Optionally, the write-back thread for the external storage medium is configured separately for the external storage medium when it is connected to the execution device. For example, when the external storage medium is connected to the execution device via a USB interface or other interface, the operating system, after recognizing the connection, configures a separate write-back thread for the external storage medium so that data copying can be performed based on the write-back thread when data copying occurs on the external storage medium.

[0099] Specifically, operating systems typically run a device-level write-back thread. This thread is primarily used to write cached data (i.e., data on dirty pages) back to local storage when certain conditions are met. However, the write-back thread for this device is usually only triggered when the cached data in memory reaches a certain threshold of the total memory capacity (e.g., 5% of a 32GB memory capacity), or when the cached data has been cached in memory for a relatively long time (e.g., 5 or 30 seconds). This application borrows the idea of ​​the existing write-back thread, configuring a separate write-back thread for externally connected storage media, and configuring separate write-back logic for data write-back to the external storage media. That is, when the amount of data cached in memory to be copied (written) to the external storage media reaches a first threshold (a lower threshold), the corresponding write-back thread for the external storage media will be triggered to write the data to the external storage media, thereby achieving timely data copying to the external storage media.

[0100] Furthermore, by configuring a separate write-back thread and data write-back logic for the external storage medium, the upper-layer application (i.e., the file explorer) no longer needs to trigger the fsync system call after copying a certain amount of data to memory. Instead, the write-back thread asynchronously copies the data from memory to the external storage medium, thereby ensuring that the upper-layer application can continuously copy data from the local storage medium to the cache area of ​​memory.

[0101] Understandably, in most cases, the speed at which a device copies data from local storage to memory is higher than the speed at which it copies data from memory to external storage. When the amount of data to be copied is large, although copying data from local storage to memory and from memory to external storage are performed in parallel, the higher speed of copying data from local storage to memory results in an increasing amount of data cached in memory, gradually consuming a significant amount of memory space. Therefore, this application can intermittently pause the copying process from local storage to memory cache based on the amount of data already cached in the memory cache, thereby ensuring that excessive amounts of data to be copied are not cached in memory, thus further saving memory space.

[0102] For example, when the amount of data that has been cached in the cache area but not written to the external storage medium reaches a second threshold, the execution device suspends copying data from the target data from the local storage medium to the cache area in memory, wherein the second threshold is greater than or equal to the first threshold.

[0103] In other words, when the amount of data to be copied cached in the cache area reaches a certain amount, it can be considered that enough data to be copied has been cached in memory. The execution device will find it difficult to quickly copy the data already cached in memory to the external storage medium. At this time, the execution device can temporarily pause the copying of data from the target data from the local storage medium to the cache area of ​​memory in order to avoid caching too much data to be copied in memory at once.

[0104] For example, please refer to Figure 5 , Figure 5 This application provides a schematic diagram illustrating the suspension of data copying from local storage media to memory. For example... Figure 5As shown, during the normal data copying process, the execution device copies data from the local storage medium to the cache area in memory, while simultaneously writing back the cached data in memory to the external storage medium using a write-back thread. Because the speed of copying data from the local storage medium to memory is faster than the speed of writing data from memory back to the external storage medium, the amount of data cached in memory increases. When the amount of data cached in memory and awaiting writing to the external storage medium reaches a second threshold, the execution device pauses copying data from the local storage medium to the cache area in memory. At this point, the execution device will continue to write data from the cache area in memory back to the external storage medium, thereby continuously consuming the cached data in memory.

[0105] Furthermore, in order to ensure that the data copying process can continue to execute and avoid affecting the normal copying of data to the external storage medium, after pausing the copying of data from the target data from the local storage medium to the cache area for a preset time, the execution device continues to copy the uncopyable data from the target data from the local storage medium to the cache area.

[0106] Specifically, during the process where the execution device pauses copying data from the local storage medium to the cache area, the write-back thread continues to write data from the cache area to the external storage medium. Therefore, the amount of data in the cache area that has been cached but not yet written to the external storage medium will gradually decrease. If the execution device pauses copying data from the local storage medium to the cache area for too long, all the data in the cache area that has not yet been written to the external storage medium may be consumed, thus affecting the normal copying of data to the external storage medium. Therefore, after each pause (i.e., the aforementioned preset duration), the execution device resumes the process of copying data from the local storage medium to the cache area, ensuring that there is always data in the cache area waiting to be written to the external storage medium until the data copying is complete.

[0107] For example, please refer to Figure 6 , Figure 6 This application provides a schematic diagram illustrating the process of recovering data copied from local storage media to memory. For example... Figure 6 As shown, when the amount of data cached in memory and awaiting writing to the external storage medium reaches a second threshold, the execution device pauses copying data from the local storage medium to the memory cache area. At this time, the execution device only performs write-back of data from the memory cache area to the external storage medium, thereby continuously consuming the cached data in memory. After a preset time has elapsed since the execution device paused copying data from the local storage medium to the memory cache area, the execution device continues to copy the uncopyable data from the target data to the memory cache area to ensure that a certain amount of data awaiting writing to the external storage medium is always cached in the cache area.

[0108] It should be noted that after the execution device begins writing data from the memory cache to the external storage medium via the write-back thread, it will continuously execute the process of writing data from the cache to the external storage medium until all data has been written (i.e., until the data copying process is complete). During this period, the execution device will determine whether to pause the data copying from the local storage medium to the cache region for a preset duration based on whether the amount of data cached in the cache region but not yet written to the external storage medium reaches a second threshold. Therefore, during the entire data copying process to the external storage medium, the execution device may pause the data copying process from the local storage medium to the cache region multiple times, but this will not affect the normal data copying process because the write-back thread is continuously copying data from the cache region to the external storage medium.

[0109] For example, please refer to Figure 7 , Figure 7 This is a schematic diagram illustrating a data copying process provided in this application. Figure 7 As shown, Figure 7 The CCP demonstrated the data copying process between local storage media, memory, and external storage media at five time points, T1-T5.

[0110] At time T1, the execution device copies data from the local storage medium to the memory cache area. However, because the data cached in the memory cache area has not yet reached the first threshold, the execution device has not yet started the write-back thread to write the data in the cache area to the external storage medium.

[0111] At time T2, the execution device continues copying data from the local storage medium to the memory cache area. Furthermore, since the data cached in the memory cache area has reached the first threshold, the execution device has initiated a write-back thread to write the data in the cache area to the external storage medium. That is, at this point, the execution device is simultaneously copying data from the local storage medium to the memory cache area and also copying the cached data from the cache area to the external storage medium.

[0112] At time point T3, because the amount of data cached in the memory cache region that has not yet been written to the external storage medium has reached the second threshold, the execution device pauses copying data from the local storage medium to the memory cache region. At this time, the execution device still writes the data in the cache region to the external storage medium by executing the write-back thread.

[0113] At time T4, after a preset timeout in copying data from local storage to the memory cache, the execution device resumes copying the uncopyable data from local storage to the memory cache. Simultaneously, the execution device continuously writes data from the cache to the external storage medium via write-back threads.

[0114] At time point T5, the execution device has already copied all the data in the target data from the local storage medium to the memory cache area. Therefore, the execution device no longer executes the process of copying data from the local storage medium to the memory cache area. However, since there is still some data in the memory cache area that has not been written to the external storage medium, the execution device continues to write the data in the cache area to the external storage medium by executing the write-back thread.

[0115] Depend on Figure 7 As can be seen, after the execution device starts the write-back thread, it continuously copies data from the cache area to the external storage medium until the target data stored on the local storage medium is completely written to the external storage medium. However, during the entire data copying process, the execution device may pause the copying of data from the local storage medium to the cache area once or multiple times to ensure that the data cached in the cache area remains stable below the specified data amount (i.e., the second threshold).

[0116] Optionally, the preset duration for each pause in copying data from the local storage medium to the cache area is related to the speed at which data in the cache area is written to the external storage medium. Specifically, if the speed at which data in the cache area is written to the external storage medium is higher, then the data in the cache area waiting to be written to the external storage medium is consumed faster, and the execution device needs to resume data copying faster. Therefore, the preset duration for pausing data copying is shorter. Conversely, if the speed at which data in the cache area is written to the external storage medium is slower, then the data in the cache area waiting to be written to the external storage medium is consumed slower, and the preset duration for pausing data copying can be longer.

[0117] In practical applications, if the preset duration for pausing data copying by the execution device is too long, the data in the cache area may be completely consumed without being effectively replenished, thus affecting the normal data copying process to the external storage medium. If the preset duration for pausing data copying by the execution device is too short, the execution device will need to frequently pause and resume the process of copying data from the local storage medium to memory, resulting in significant performance overhead.

[0118] Based on this, the execution device can determine the preset duration for pausing the copying of data from the local storage medium to the cache region by measuring the speed at which data is written from the cache region to the external storage medium. Alternatively, the execution device can determine the preset duration by continuously adjusting the pause duration of data copying and observing the amount of data remaining in the cache region after each data copying is resumed. For example, when the execution device pauses data copying for the first time, the pause duration can be a pre-configured duration; if, after resuming data copying, the execution device finds that there is still a significant amount of data in the cache region that has not been written to the external storage medium, then the execution device will extend the pause duration for the next data copying; if, after resuming data copying, the execution device finds that there is a small amount of data in the cache region that has not been written to the external storage medium, then the execution device will shorten the pause duration for the next data copying. In this way, by observing the amount of data remaining in the cache region after each data copying is resumed and adjusting the pause duration for the next pause, the execution device can determine a stable pause duration to ensure that the amount of data remaining in the cache region that has not been written to the external storage medium is as small as possible and will not affect the normal data copying process each time data copying is resumed.

[0119] In summary, by setting the preset duration for pausing data copying on the execution device to be related to the speed at which data in the cache area is written to the external storage medium, it is possible to ensure that the execution device pauses for an appropriate duration each time, avoiding the impact of pausing data copying for too long on the normal data copying process or the excessive performance overhead caused by pausing data copying for too short a time.

[0120] Optionally, the second threshold for triggering a pause in data copying can be determined based on a pre-configured threshold and / or the speed at which data in the cache area is written to the external storage medium. For example, the second threshold can be a pre-configured threshold (e.g., 32MB, 64MB, 128MB, etc.).

[0121] For example, the second threshold can be a threshold determined based on the speed at which data in the cache area is written to the external storage medium. Specifically, assuming the speed at which data in the cache area is written to the external storage medium is N bytes per second, then the second threshold can be N*X bytes. That is, the second threshold can be a product of the speed at which data in the cache area is written to the external storage medium and a specified duration.

[0122] Generally, execution devices can determine the speed at which data is written from the cache to the external storage medium by collecting the number of I / O requests completed by the external storage medium per unit of time, such as the number of I / O operations per second (IOPS). For example, assuming that the external storage medium completes 10,000 I / O requests per second (i.e., 10,000 IOPS), and each I / O request indicates the writing of 4KB of data, then the speed at which data is written from the cache to the external storage medium is 4KB * 10,000 per second, or 40MB / s.

[0123] For example, the second threshold could be a threshold determined by combining a pre-configured threshold and the speed at which data is written from the cache region to the external storage medium (e.g., the second threshold is the smaller of the pre-configured threshold and the threshold determined by the speed at which data is written from the cache region to the external storage medium). For instance, if the pre-configured threshold is 64MB and the threshold determined by the speed at which data is written from the cache region to the external storage medium is 40MB, the second threshold could be set to 40MB.

[0124] Understandably, in the execution device, the upper-layer application, such as the file explorer mentioned above, typically interacts with the user and receives data copy requests. The file explorer triggers the data copy process by receiving user requests and needs to provide feedback to the user upon completion, allowing the user to stay informed of the copy progress. However, in practical applications, when copying data from the cache to external storage media using a write-back process, the write-back process does not provide feedback to the file explorer. Therefore, the file explorer is unaware of the actual progress of the data copying to the external storage media. Generally, the file explorer displays the data copy progress on its interface to inform the user of the current copy status. When the file explorer cannot ascertain the actual progress of the data copying to the external storage media, it typically displays the progress of copying data from the local storage media to the memory cache as the data copy progress. In this way, by setting the second threshold mentioned above and continuously pausing the File Explorer's copying of data from the local storage medium to the memory cache area, this solution can ensure that the data copying progress presented to the user by the File Explorer is as close as possible to the actual data copying progress without affecting the overall data copying progress, thus ensuring that the user is shown the most accurate data copying progress possible.

[0125] Furthermore, in this application, the file explorer can trigger the fsync system call during the process of copying data from the local storage medium to the cache area, so as to know in a timely manner when the entire data copying process is completed based on the return information of the fsync system call.

[0126] For example, after copying all data in the target data from the local storage medium to the cache area, the file explorer can call the fsync system call to actively trigger the copying of the data in the cache area to the external storage medium.

[0127] Therefore, after copying all data from the target data from the local storage medium to the cache area, the operating system kernel in the execution device can write the data in the cache area to the external storage medium by executing the fsync system call. Furthermore, after writing all data from the cache area to the external storage medium, the operating system kernel can return a success message to the initiator of the fsync system call (i.e., the file explorer mentioned above). In this way, the file explorer can display a message on the user interface indicating that the data copy is complete, thus informing the user that all the target data has been copied to the external storage medium.

[0128] Optionally, since the execution of the fsync system call will block the initiator, thus preventing data from being copied from the local storage medium to the cache area, the initiator of the fsync system call will only initiate the fsync system call after all the data in the target data has been copied from the local storage medium to the cache area. This ensures that the process of copying data from the local storage medium to the cache area will not be blocked, thus ensuring the overall data copying efficiency.

[0129] In other words, the file explorer does not trigger the fsync system call during the process of copying data from the local storage medium to the cache area in memory, thus ensuring that the process of copying data from the local storage medium to the cache area is not blocked and stopped by the fsync system call.

[0130] To facilitate understanding, the following will detail the process of copying data from local storage media to external storage media in practical applications, using concrete examples. Please refer to... Figure 8 , Figure 8 This application provides a schematic diagram of a process for copying data from a local storage medium to an external storage medium. For example... Figure 8 As shown, the process of copying data from local storage medium to external storage medium includes the following steps 801-809.

[0131] Step 801: External storage medium access is detected, and a write-back thread is configured for the external storage medium.

[0132] When a user connects an external storage medium to the execution device, the operating system on the execution device recognizes the connection and then configures a separate write-back thread for the external storage medium. At this time, the write-back thread configured by the operating system for the external storage medium is still in a dormant state and has not yet started execution.

[0133] Step 802: Obtain a data copy request.

[0134] After the user sends a data copy request to the execution device, the upper-layer application on the execution device (such as File Explorer) receives the data copy request and determines that the target data in the local storage medium needs to be copied to the external storage medium.

[0135] Step 803: Copy the data from the target data from the local storage medium to the cache area of ​​memory.

[0136] Based on data copy requests, File Explorer can divide the data in the target data into blocks and continuously copy these blocks from local storage to a cache area in memory. Furthermore, File Explorer allocates a corresponding storage address on the external storage medium for each data block copied to memory, ensuring that the target data is ultimately copied efficiently to the external storage medium.

[0137] Step 804: Determine whether the amount of cached data in the cache area exceeds the first threshold.

[0138] After File Explorer begins copying data from the target data from the local storage medium to the memory cache area, the operating system continuously monitors and determines whether the amount of cached data in the cache area corresponding to the external storage medium exceeds a first threshold.

[0139] Step 805: If the amount of cached data in the cache area exceeds the first threshold, execute the write-back thread to write the data in the cache area to the external storage medium.

[0140] If the amount of cached data in the cache area exceeds the first threshold when the write-back thread is not woken up, the operating system will wake up and execute the write-back thread, thereby writing the data in the cache area to the external storage medium through the write-back thread.

[0141] If the amount of cached data in the cache area does not exceed the first threshold, the operating system will not wake up the write-back thread. Instead, the file explorer will continuously copy the data from the target data from the local storage medium to the cache area in memory.

[0142] Step 806: Determine whether all data in the target data has been copied to the memory cache area.

[0143] During the data copying process, File Explorer will determine whether all data in the target data has been copied to the memory cache area.

[0144] Step 807: If all data in the target data has been copied to the cache area in memory, then call the fsync system call to actively trigger the writing of the data in the cache area to the external storage medium.

[0145] If all data in the target dataset has been copied to the memory cache, the file explorer calls the fsync system call, thus actively triggering the writing of the data in the cache to the external storage medium. At this point, the operating system kernel executes the fsync system call to issue an I / O request to the external storage medium, thereby continuing to write the remaining data in the cache to the external storage medium.

[0146] Furthermore, after the fsync system call completes, the operating system kernel returns the return value of the fsync system call to the file explorer (e.g., a value of 0 indicating the completion of the fsync system call), indicating that the fsync system call completed successfully. In this way, the file explorer can notify the user that the data copy is complete, allowing the user to promptly unplug the external storage medium.

[0147] Step 808: If not all data in the target data has been copied to the memory cache area, determine whether the amount of data cached in the cache area and not written to the external storage medium exceeds the second threshold.

[0148] If not all data in the target data is copied to the memory cache area, the file explorer will continuously determine whether the amount of data cached in the cache area but not written to external storage media exceeds a second threshold. The method for determining the second threshold can be found in the above-described embodiment and will not be repeated here.

[0149] Step 809: If the amount of data cached in the cache area but not written to the external storage medium exceeds the second threshold, pause copying data from the local storage medium to the cache area in memory for a preset time, and then resume copying data from the local storage medium to the cache area in memory.

[0150] If the amount of data cached in the cache area but not yet written to the external storage medium exceeds the second threshold, File Explorer can pause the copying of data from the local storage medium to the cache area in memory for a preset time. Of course, after pausing the copying of data from the local storage medium to the cache area in memory for the preset time, File Explorer resumes the copying of data from the local storage medium to the cache area in memory and proceeds to step 806 above until the target data is copied to the external storage medium.

[0151] The method provided in this application has been described in detail above. Next, the device provided in this application for performing the above method will be described.

[0152] Please see Figure 9 , Figure 9 This is a schematic diagram of a data copying device provided in this application. Figure 9 As shown, the data copying device includes: an acquisition module 901, used to acquire a data copying request, the data copying request being used to instruct the copying of target data from a local storage medium to an external storage medium; a processing module 902, used to copy data from the target data from the local storage medium to a cache area in memory based on the data copying request, the cache area being used to cache the data to be copied to the external storage medium; the processing module 902 is further used to start a write-back thread corresponding to the external storage medium when the amount of data cached in the cache area reaches a first threshold, and write the data in the cache area to the external storage medium through the write-back thread.

[0153] In one possible implementation, the processing module 902 copies data from the target data from the local storage medium to the cache area in memory, which is performed in parallel with writing the data in the cache area to the external storage medium via a write-back thread.

[0154] In one possible implementation, the processing module 902 is further configured to: when the amount of data cached in the cache area but not written to the external storage medium reaches a second threshold, pause copying data from the target data from the local storage medium to the cache area of ​​memory, wherein the second threshold is greater than or equal to the first threshold.

[0155] In one possible implementation, the processing module 902 is further configured to: after pausing the copying of data from the target data from the local storage medium to the cache area for a preset time, continue copying the uncopyable data from the target data from the local storage medium to the cache area.

[0156] In one possible implementation, the preset duration is related to the speed at which data in the cache area is written to the external storage medium.

[0157] In one possible implementation, the second threshold is determined based on a pre-configured threshold and / or the speed at which data in the cache region is written to the external storage medium.

[0158] In one possible implementation, the processing module 902 is further configured to: after copying all data in the target data from the local storage medium to the cache area, write the data in the cache area to the external storage medium by executing the fsync system call; and after writing all data in the cache area to the external storage medium, return a system call success message to the initiator of the fsync system call.

[0159] In one possible implementation, the initiator only initiates the fsync system call after copying all data from the target data from the local storage medium to the cache area.

[0160] In one possible implementation, the write-back thread is configured separately for the external storage medium when it is accessed.

[0161] Figure 10 This is a schematic diagram of the structure of an execution device provided in this application. Figure 10 As shown, the execution device 1000 is equipped with the aforementioned message detection device. The execution device 1000 is implemented using a general bus architecture.

[0162] The execution device 1000 includes at least one processor 1001, a communication bus 1002, a memory 1003, and at least one communication interface 1004.

[0163] Optionally, the processor 1001 is a general-purpose CPU, NP, microprocessor, or one or more integrated circuits for implementing the solutions of this application, such as application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0164] The communication bus 1002 is used to transmit information between the aforementioned components. The communication bus 1002 includes an address bus, a data bus, and a control bus. For ease of representation, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus.

[0165] Optionally, memory 1003 is read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions. Alternatively, memory 1003 is random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions. Alternatively, memory 1003 may be electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited to these. Optionally, memory 1003 exists independently and is connected to processor 1001 via communication bus 1002. Optionally, memory 1003 and processor 1001 are integrated together.

[0166] Communication interface 1004 uses any transceiver-like device for communicating with other devices or communication networks. Communication interface 1004 includes a wired communication interface. Optionally, communication interface 1004 also includes a wireless communication interface. The wired communication interface is, for example, an Ethernet interface. An Ethernet interface is an optical interface, an electrical interface, or a combination thereof. The wireless communication interface is a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof, etc.

[0167] In a specific implementation, as one example, the processor 1001 includes one or more CPUs, such as... Figure 10 CPU0 and CPU1 are shown in the diagram.

[0168] In a specific implementation, as one example, the execution device 1000 includes multiple processors, such as... Figure 10 The processors 1001 and 1005 shown are illustrated. Each of these processors is either a single-core processor (SCPU) or a multi-core processor (Multi-CPU). Here, a processor refers to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0169] In some embodiments, the memory 1003 is used to store program code 1006 for executing the scheme of this application, and the processor 1001 executes the program code 1006 stored in the memory 1003. That is, the execution device 1000 implements the above-described method embodiment through the processor 1001 and the program code 1006 in the memory 1003.

[0170] Please refer to Figure 11 , Figure 11 This is a schematic diagram of a computer-readable storage medium provided in this application. This application also provides a computer-readable storage medium in some embodiments, wherein the above-described... Figure 3 The disclosed method can be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art.

[0171] Figure 11 A conceptual partial view of an example computer-readable storage medium arranged according to at least some of the embodiments shown herein is illustrated schematically. The example computer-readable storage medium includes a computer program for executing computer processes on a computing device.

[0172] In one embodiment, the computer-readable storage medium 1100 is provided using a signal bearer medium 1101. The signal bearer medium 1101 may include one or more program instructions 1102, which, when executed by one or more processors, can provide the above-mentioned... Figure 3 The described function or part of the function.

[0173] In some examples, the signal carrying medium 1101 may include a computer-readable medium 1103, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital magnetic tape, a memory, ROM, or RAM, etc.

[0174] In some embodiments, the signal carrying medium 1101 may comprise a computer-recordable medium 1104, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, etc. In some embodiments, the signal carrying medium 1101 may comprise a communication medium 1105, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). Therefore, for example, the signal carrying medium 1101 may be transmitted by a wireless form of communication medium 1105 (e.g., a wireless communication medium conforming to the IEEE 802.X standard or other transmission protocols).

[0175] One or more program instructions 1102 may be, for example, computer-executable instructions or logical implementation instructions. In some examples, the computing device may be configured to provide various operations, functions, or actions in response to one or more program instructions 1102 conveyed to the computing device via a computer-readable medium 1103, a computer-recordable medium 1104, and / or a communication medium 1105.

[0176] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the accompanying drawings of the device embodiments provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0177] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, and of course, it can also be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods of the various embodiments of this application.

[0178] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0179] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, computer instructions may be transferred from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A data copying method, characterized in that, include: Obtain a data copy request, the data copy request being used to instruct the target data in the local storage medium to be copied to the external storage medium; Based on the data copy request, the data in the target data is copied from the local storage medium to the cache area in memory, and the cache area is used to cache the data to be copied to the external storage medium; When the amount of data cached in the cache area reaches a first threshold, the write-back thread corresponding to the external storage medium is started, and the data in the cache area is written to the external storage medium through the write-back thread.

2. The method according to claim 1, characterized in that, The process of copying data from the target data from the local storage medium to the memory cache area is performed in parallel with the process of writing data from the cache area to the external storage medium through the write-back thread.

3. The method according to claim 1 or 2, characterized in that, The method further includes: When the amount of data cached in the cache area but not written to the external storage medium reaches a second threshold, the copying of data from the target data from the local storage medium to the cache area of ​​memory is paused, wherein the second threshold is greater than or equal to the first threshold.

4. The method according to claim 3, characterized in that, The method further includes: After pausing the copying of data from the target data from the local storage medium to the cache area for a preset time, the copying of the uncopyable data from the target data from the local storage medium to the cache area continues.

5. The method according to claim 4, characterized in that, The preset duration is related to the speed at which data in the cache area is written to the external storage medium.

6. The method according to any one of claims 3-5, characterized in that, The second threshold is determined based on a pre-configured threshold and / or the speed at which data in the cache area is written to the external storage medium.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: After copying all data from the local storage medium to the cache area, the data in the cache area is written to the external storage medium by executing the fsync system call; After writing all the data in the cache area to the external storage medium, a successful system call message is returned to the initiator of the fsync system call.

8. The method according to claim 7, characterized in that, The initiator initiates the fsync system call only after copying all data in the target data from the local storage medium to the cache area.

9. The method according to any one of claims 1-8, characterized in that, The write-back thread is configured separately for the external storage medium when it is accessed.

10. The method according to any one of claims 1-9, characterized in that, The external storage medium includes a portable hard drive, a Universal Serial Bus (USB) flash drive, or a memory card.

11. A data copying device, characterized in that, include: The acquisition module is used to acquire a data copy request, which is used to instruct the target data in the local storage medium to be copied to the external storage medium; The processing module is used to copy data from the local storage medium to a cache area in memory based on the data copy request, wherein the cache area is used to cache data to be copied to the external storage medium; The processing module is further configured to, when the amount of data cached in the cache area reaches a first threshold, start the write-back thread corresponding to the external storage medium, and write the data in the cache area to the external storage medium through the write-back thread.

12. The apparatus according to claim 11, characterized in that, The processing module copies data from the local storage medium to the cache area of ​​memory from the target data, which is performed in parallel with the write-back thread writing data from the cache area to the external storage medium.

13. The apparatus according to claim 11 or 12, characterized in that, The processing module is further configured to: When the amount of data cached in the cache area but not written to the external storage medium reaches a second threshold, the copying of data from the target data from the local storage medium to the cache area of ​​memory is paused, wherein the second threshold is greater than or equal to the first threshold.

14. The apparatus according to claim 13, characterized in that, The processing module is further configured to: After pausing the copying of data from the target data from the local storage medium to the cache area for a preset time, the copying of the uncopyable data from the target data from the local storage medium to the cache area continues.

15. The apparatus according to claim 14, characterized in that, The preset duration is related to the speed at which data in the cache area is written to the external storage medium.

16. The apparatus according to any one of claims 13-15, characterized in that, The second threshold is determined based on a pre-configured threshold and / or the speed at which data in the cache area is written to the external storage medium.

17. The apparatus according to any one of claims 11-16, characterized in that, The processing module is further configured to: After copying all data from the local storage medium to the cache area, the data in the cache area is written to the external storage medium by executing the fsync system call; After writing all the data in the cache area to the external storage medium, a successful system call message is returned to the initiator of the fsync system call.

18. The apparatus according to claim 17, characterized in that, The initiator initiates the fsync system call only after copying all data in the target data from the local storage medium to the cache area.

19. The apparatus according to any one of claims 11-18, characterized in that, The write-back thread is configured separately for the external storage medium when it is accessed.

20. A data copying device, characterized in that, The device includes a memory and a processor; the memory stores code, and the processor is configured to execute the code, wherein when the code is executed, the device performs the method as described in any one of claims 1 to 10.

21. A computer storage medium, characterized in that, The computer storage medium stores instructions that, when executed by the computer, cause the computer to perform the method according to any one of claims 1 to 10.

22. A computer program product, characterized in that, The computer program product stores instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1 to 10.

Images