Large file copy method and apparatus, electronic device, storage medium, and program

Abstract

The present application provides a large file copy method and apparatus, an electronic device, a computer-readable storage medium, and a computer program. The large file copy method comprises: when the size of a file to be processed is greater than a preset threshold, invoking a preset kernel command; in response to the kernel command, and when it is determined that a source file address and a destination file address of said file satisfy a preset address condition, acquiring a queue state of a preset circular queue; and when the queue state satisfies a first state condition, sequentially extracting file content of said file from the source file address and sequentially storing same into the circular queue, and when the queue state satisfies a second state condition, sequentially extracting file content stored in the circular queue and sequentially writing same into the destination file address. The present application does not relate to the interaction between a kernel layer and an application layer, thus reducing system overhead. In addition, the copying of large files also eliminates the process of caching data of the large files into memories, thereby improving the stability of file caching systems.

Description

Large file copying method and device, electronic equipment, storage medium and program

[0001] Cross-reference of related applications

[0002] The present application claims priority to the Chinese patent application No. 202411124925.7, filed on August 16, 2024, entitled "Large file copying method, device, electronic equipment and storage medium", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to the technical field of computer, in particular to a large file copying method and device, electronic equipment, computer readable storage medium and computer program. BACKGROUND

[0004] File copying is an important function of computer system, and how to efficiently complete file copying is an important factor to improve computer performance.

[0005] At present, when the file system performs file copying, the data content of the file can be copied to the application layer through the read interface of the kernel first, and the data content of the file is cached to the memory at the same time, then the write interface of the kernel is called to write the data content of the application layer to the destination address, and the read-write process is repeated to finally complete the copying of the file.

[0006] However, in the current scheme, when the size of the file to be copied is large, the data needs to be copied back and forth between the kernel and the application layer during the execution process, which leads to large system overhead, and because the capacity of the memory is limited, the caching action of the large file during the copying process easily affects the stability of the file cache. SUMMARY

[0007] Embodiments of the present application provide a large file copying method and device to solve the problems in the prior art.

[0008] In a first aspect, the embodiments of the present application provide a large file copying method, which comprises:

[0009] In the case that the file size of the file to be processed is greater than a preset threshold, a preset kernel command is called;

[0010] In response to the kernel command, and in the case that the source file address and the destination file address of the file to be processed meet a preset address condition, the queue state of a preset ring queue is obtained; the queue state is used to represent the storage state of the ring queue;

[0011] extract the file content of the to-be-processed file from the source file address and sequentially store the file content in the ring queue in the case that the queue state satisfies a first state condition, and sequentially extract the file content stored in the ring queue and sequentially write the file content to the destination file address in the case that the queue state satisfies a second state condition;

[0012] copy the to-be-processed file in the case that the queue state satisfies a termination condition.

[0013] In a second aspect, an embodiment of the present application provides a large file copying device, the device comprising:

[0014] a calling module configured to call a preset kernel command in the case that the file size of the to-be-processed file is greater than a preset threshold value;

[0015] a state module configured to, in response to the kernel command, acquire a queue state of a preset ring queue in the case that a source file address and a destination file address of the to-be-processed file satisfy a preset address condition; the queue state is used to represent the storage condition of the ring queue;

[0016] a copying module configured to extract the file content of the to-be-processed file from the source file address and sequentially store the file content in the ring queue in the case that the queue state satisfies a first state condition, and sequentially extract the file content stored in the ring queue and sequentially write the file content to the destination file address in the case that the queue state satisfies a second state condition;

[0017] a termination module configured to copy the to-be-processed file in the case that the queue state satisfies a termination condition.

[0018] In a third aspect, an embodiment of the present application further provides an electronic device comprising a processor;

[0019] a memory configured to store instructions executable by the processor;

[0020] The processor is configured to execute the instructions to implement the method of the first aspect.

[0021] In a fourth aspect, an embodiment of the present application further provides a computer readable storage medium, when instructions in the computer readable storage medium are executed by a processor of an electronic device, the electronic device can execute the method of the first aspect.

[0022] In a fifth aspect, an embodiment of the present application provides a computer program comprising computer readable code, when the computer readable code is executed on a computing processing device, the computing processing device is caused to execute the method of the first aspect.

[0023] In the case that the file size of the to-be-processed file is greater than a preset threshold, the preset kernel command is executed, and in the case that the source file address and the destination file address of the to-be-processed file satisfy a preset address condition, the queue state of a preset ring queue is acquired, in the case that the queue state satisfies a first state condition, the file content of the to-be-processed file is extracted from the source file address in sequence and stored in the ring queue in sequence, and in the case that the queue state satisfies a second state condition, the file content stored in the ring queue is extracted in sequence and written into the destination file address in sequence, and the read-write process is repeatedly executed to complete the copying of the to-be-processed file. The entire large file copying process of the embodiment of the present application is implemented in the kernel layer, and the interaction between the kernel layer and the application layer is not involved, so that the system overhead is reduced. In addition, the process of buffering the data of the large file to the memory is also saved for the copying of the large file, the pressure of the file buffering system is reduced, and the stability of the file buffering system is improved.

[0024] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the content of the specification can be implemented, and in order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0026] Fig. 1 is a step flow chart of a large file copying method provided by an embodiment of the present application;

[0027] Fig. 2 is a specific step flow chart of a large file copying method provided by an embodiment of the present application;

[0028] Fig. 3 is a schematic diagram of a ring queue provided by an embodiment of the present application;

[0029] Fig. 4 is a parallel processing flow chart provided by an embodiment of the present application;

[0030] Fig. 5 is a flow chart of a large file copying process provided by an embodiment of the present application;

[0031] Fig. 6 is a serial processing flow chart provided by an embodiment of the present application;

[0032] Fig. 7 is a block diagram of a large file copying device provided by an embodiment of the present application;

[0033] Fig. 8 is a block diagram of a device of the present application;

[0034] FIG. 9 is a structural diagram of a server in some embodiments of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0036] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally a class, and are not limited to the number of objects, for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects. The term "multiple" in the embodiments of the present application means two or more, and other quantifiers are similar.

[0037] FIG. 1 is a step flowchart of a large file copying method provided by the embodiments of the present application. As shown in FIG. 1, the method can include:

[0038] In step 101, a preset kernel command is called when the file size of the to-be-processed file is greater than a preset threshold.

[0039] In the related art, data needs to be copied back and forth between the kernel and the application layer during the file copying process, and the data caching operation is involved. This results in a large system overhead and memory instability when copying large files. The embodiments of the present application aim to optimize the copying process of large files and solve the above problems when copying large files.

[0040] Firstly, the embodiment of the present application can analyze the to-be-processed file waiting for copying to determine whether the to-be-processed file is a large file. Specifically, the embodiment of the present application can set a preset threshold and acquire the file size of the to-be-processed file, and in the case that the file size of the to-be-processed file is greater than the preset threshold, it is determined that the to-be-processed file is a large file. The preset threshold can be set according to actual needs, for example, the preset threshold can be set to 6 GB, and when the file size of the to-be-processed file is greater than 6 GB, the to-be-processed file can be determined as a large file.

[0041] Further, the embodiment of the present application develops a new kernel command for copying a large file: copy_file_direct, which can directly implement the copying of a large file from the kernel layer, eliminating the data interaction between the application layer and the kernel layer, and in the process of executing the command to copy a large file, the cache system of the file system can not be passed through, that is, the data content of the large file is not cached to the memory.

[0042] Specifically, the calling of the kernel command can be the calling of an application binary interface (ABI, Application Binary Interface), which is a standard defining the interface between an application program and an operating system or hardware.

[0043] Step 102, in response to the kernel command, and in the case that the source file address and the destination file address of the to-be-processed file meet the preset address condition, acquiring the queue state of the preset ring queue; the queue state is used to represent the storage state of the ring queue.

[0044] In the embodiment of the present application, the source file address of the to-be-processed file represents the position of the to-be-processed file before copying, and the destination file address of the to-be-processed file represents the position of the to-be-processed file after copying. The preset address condition represents that the source file address and the destination file address of the to-be-processed file meet certain special conditions, and when the condition is established, the subsequent parallel read / write operation can be used to speed up the copying process without negative effects.

[0045] Specifically, the embodiment of the present application can set a ring queue in the kernel to replace the application layer to maintain the data used in the read / write parallel execution process. The embodiment of the present application can acquire the queue state of the ring queue in response to the kernel command and in the case that the source file address and the destination file address of the to-be-processed file meet the preset address condition, the queue state is used to represent the storage state of the ring queue, and then the read / write process can control itself to be in a dormant or working state according to the queue state until the parallel copying of the file is completed.

[0046] Optionally, the preset address condition is that the source file address and the destination file address are not in the same disk.

[0047] In one implementation, the preset address condition can represent that the source file address and the destination file address are in the same disk, because the head of the mechanical disk has the overhead of seeking, i.e., it takes a certain moving time for the head to move to a specific position. If the read-write parallel copy strategy is adopted in the case where the source file address and the destination file address are in the same disk, the head of the disk needs to move back and forth repeatedly, which causes a performance decline.

[0048] However, in the case where the read-write parallel copy strategy is adopted when the source file address and the destination file address are not in the same disk, e.g., in the case where the source file address and the destination file address are in two different disks respectively, the heads of the two disks can work in parallel, because the heads of the two disks move independently, and there is no mutual interference between them, thus meeting the requirement of parallel execution of read-write operations, and effectively improving the copy efficiency of large files.

[0049] In step 103, the file content of the to-be-processed file is sequentially extracted from the source file address and sequentially stored in the ring queue when the queue state satisfies a first state condition, and the file content stored in the ring queue is sequentially extracted and sequentially written into the destination file address when the queue state satisfies a second state condition.

[0050] In the embodiment, the ring queue can be composed of a plurality of memory pages, each memory page representing a piece of memory area, and the memory pages are used to store data content. The role of the read process is to sequentially extract the file content of the to-be-processed file from the source file address and sequentially store it in the ring queue, and the read process can work as long as there is an empty memory page (a memory page without data) in the ring queue; the role of the write process is to extract the file content stored in the ring queue and sequentially write it into the destination file address, and the write process can work as long as there is a non-empty memory page (a memory page with data) in the ring queue, and the two processes work in parallel to complete the copy of the file.

[0051] Specifically, the queue state satisfying the first state condition can represent that the state of the ring queue at this time can meet the work of the read process, i.e., there is an empty memory page in the ring queue; and the queue state satisfying the second state condition can represent that the state of the ring queue at this time can meet the work of the write process, i.e., there is a non-empty memory page in the ring queue. The read process and the write process can independently work in parallel according to the respective state conditions they meet, to realize the copy of data.

[0052] For example, the reading process can read data content such as several pages from the start position of the large file and store the read data content into the start page of the ring queue in sequence; the writing process can extract the data content stored in the memory pages of the ring queue in sequence from the start page of the ring queue and write the data content into the destination file address in sequence, and the above process is repeatedly executed to complete the copying of the file.

[0053] Step 104, in the case where the queue state satisfies the termination condition, the copying of the to-be-processed file is completed.

[0054] In the embodiment of the present application, the termination condition indicates that the data content of the to-be-processed file is completely copied from the source file address to the destination file address, so that a new file is generated at the destination file address after copying, and the copying of the to-be-processed file is completed. The entire large file copying process of the embodiment of the present application is implemented in the kernel layer, and does not involve the interaction between the kernel layer and the application layer. In addition, the process of buffering the data of the large file to the memory is also omitted for the copying of the large file, which reduces the pressure on the file buffering system and improves the stability of the file buffering system.

[0055] In summary, in the case where the file size of the to-be-processed file is greater than the preset threshold, the preset kernel command is executed, and in the case where the source file address and the destination file address of the to-be-processed file satisfy the preset address condition, the queue state of the preset ring queue is obtained; in the case where the queue state satisfies the first state condition, the file content of the to-be-processed file is extracted from the source file address in sequence and stored in the ring queue in sequence, and in the case where the queue state satisfies the second state condition, the file content stored in the ring queue is extracted in sequence and written into the destination file address in sequence, and the reading and writing process is repeatedly executed to complete the copying of the to-be-processed file. The entire large file copying process of the embodiment of the present application is implemented in the kernel layer, and does not involve the interaction between the kernel layer and the application layer, so as to reduce the system overhead. In addition, the process of buffering the data of the large file to the memory is also omitted for the copying of the large file, which reduces the pressure on the file buffering system and improves the stability of the file buffering system.

[0056] FIG. 2 is a specific step flowchart of a large file copying method provided by the embodiment of the present application, as shown in FIG. 2, the method can include:

[0057] Step 201, in the case where the file size of the to-be-processed file is greater than a preset threshold, a preset kernel command is called.

[0058] This step can be specifically referred to step 101, which will not be described here.

[0059] Optionally, the preset threshold is the product of the total capacity of the memory and a preset coefficient, and the preset coefficient can be 0.7.

[0060] In the embodiments of the present application, generally, the memory of the device is a fixed element, and the total capacity of the memory is a fixed value. Since caching large files can affect the working stability of the memory, the product of the total capacity of the memory and a preset coefficient can be used as a preset threshold, so that the to-be-processed file with a file size greater than the preset threshold can be identified as a large file. Preferably, the preset coefficient can be 0.7. Of course, the value of the preset coefficient can also be adjusted according to actual needs, such as 0.6, 0.8, etc.

[0061] In step 202, in response to the kernel command, and in the case where it is determined that the source file address and the destination file address of the to-be-processed file satisfy a preset address condition, a queue state of a preset ring queue is obtained; the queue state is used to represent the storage condition of the ring queue.

[0062] This step can refer to step 102 for specific implementation, and details are not described herein.

[0063] In step 203, in the case where the queue state represents that there is an empty memory page in the ring queue, the read process is used to sequentially extract the file content of the to-be-processed file from the source file address and sequentially store the file content into the empty memory page; the empty memory page represents a memory page without storing data content.

[0064] In the embodiments of the present application, the ring queue can be composed of a plurality of memory pages; the read process is used to sequentially extract the file content of the to-be-processed file from the source file address and sequentially store the file content into the ring queue.

[0065] In step 204, in the case where the queue state represents that there is no empty memory page in the ring queue, the read process is controlled to sleep.

[0066] In the embodiments of the present application, for steps 203-204, the ring queue can be composed of a plurality of memory pages, each memory page represents a memory area, and the memory page is used to store data content. Referring to FIG. 3, which shows a schematic diagram of a ring queue, assuming that the ring queue includes 8 memory pages, the read process is used to write the read data in the order of memory pages 1-8; and the write process is used to extract data in the order of memory pages 1-8.

[0067] In the embodiments of the present application, the queue state represents the data storage condition of the memory pages in the ring queue, and the queue state can reflect whether the ring queue includes an empty memory page (a memory page without storing data) and whether the ring queue includes a non-empty memory page (a memory page storing data).

[0068] In one case, one memory page can be 4k in size, and the ring queue can be 128M in size.

[0069] For the read process, as long as the queue state represents that there is an empty memory page in the ring queue (i.e. there is still free capacity in the ring queue), the read process can work normally to perform the operation of writing the read data content to the empty memory page. When the queue state represents that there is no empty memory page in the ring queue, the read process cannot work normally, and the read process can enter a sleep state.

[0070] The read process can specifically extract the file content of the to-be-processed file from the source file address in sequence through a block device interface provided by the kernel. The block device interface is an interface provided for a storage device such as a hard disk or an optical drive, and it is a bottom-layer interface for directly reading and writing file data without passing through an application layer.

[0071] For example, referring to FIG. 3, the first flag (HEAD) reflects the memory page in which the read process currently writes data, i.e. the read process writes the read data content in the memory page 1 in sequence until the data content in the memory page 5 is written. At this time, the queue state can represent that the memory pages 6, 7, and 8 do not store data, and thus the read request can continue to work to write data in the memory pages 6, 7, and 8 in sequence, until the queue state represents that there is no empty memory page, and the read request enters a sleep state.

[0072] Further referring to FIG. 4, a read-write parallel processing flowchart of an embodiment of the present application is shown:

[0073] S1, a read process is created.

[0074] S2, the read process uses a block device interface to extract the data content of the to-be-processed file from a source file address in sequence, and stores the read data content in memory pages in sequence from the start of the ring queue.

[0075] S3, the write thread reads the data content stored in the memory pages in sequence from the start of the ring queue, and uses a block device interface to write the data content to a destination file address in sequence.

[0076] The step S2 and the step S3 are executed in parallel, thereby improving the file copying efficiency. It is tested that in the process of copying a large file from a disk to a U disk, the read-write parallel execution can save 30% of the time.

[0077] Step 205, in a case where the queue state represents that there is a non-empty memory page in the ring queue, the write process extracts the file content stored in the non-empty memory page in sequence and writes the file content to the destination file address in sequence, the non-empty memory page representing a memory page that stores data content.

[0078] The ring queue includes a plurality of memory pages; the write process is used to extract the file content stored in the ring queue in sequence and write into the destination file address in sequence.

[0079] Step 206, in the case that the queue state represents that there is no non-empty memory page in the ring queue, the write process is controlled to sleep.

[0080] For the write process, as long as the queue state represents that there is a non-empty memory page in the ring queue (i.e. there is data content in the ring queue), the write process can work normally and perform the operation of extracting data content from the non-empty memory page. When the queue state represents that there is no non-empty memory page in the ring queue, the write process cannot work normally and can enter sleep state.

[0081] The write process can specifically write the file content into the destination file address through a block device interface provided by the kernel. The block device interface is an interface provided for hard disks, optical drives and other storage devices, which is a bottom interface and is used to read and write file data directly without passing through the application layer.

[0082] For example, referring to FIG. 3, the second flag (TAIL) reflects the memory page in the ring queue from which the write process currently extracts data content, i.e. the write process extracts data content from memory page 1 in sequence until the data content of memory page 2 is extracted. At this time, the queue state can represent that there are memory pages 3, 4 and 5 storing data, and the write request can continue to work and extract data from memory pages 3, 4 and 5 in sequence until the queue state represents that there is no non-empty memory page at present, and the write request enters sleep state.

[0083] Step 207, in the case that the read process generates an end flag bit, the write process writes the file content stored in the remaining non-empty memory pages in the ring queue into the destination file address, and completes the copying of the to-be-processed file. The non-empty memory page represents a memory page storing data content.

[0084] The read process is used to extract the file content of the to-be-processed file from the source file address in sequence and store into the ring queue in sequence, and the write process is used to extract the file content stored in the ring queue in sequence and write into the destination file address in sequence; the end flag bit is generated by the read process when the last data page in the source file address is read.

[0085] In the embodiment of the present application, the to-be-processed file comprises a plurality of data pages, a reading process reads the data content of the to-be-processed file from the first data page in sequence until the last data page is read, and the reading process generates an end flag. The writing process can identify the end flag, and the writing process can write the file content stored in all the remaining non-empty memory pages in the ring queue into the destination file address according to the end flag, and determine that the copying of the to-be-processed file is completed, that is, all the data pages of the to-be-processed file are copied from the source file address to the destination file address.

[0086] Optionally, the method can further comprise:

[0087] Step 208, in a case where the source file address and the destination file address of the to-be-processed file do not satisfy the preset address condition, applying for a memory buffer area of a preset size.

[0088] Step 209, sequentially extracting the file content of the to-be-processed file from the source file address and sequentially storing the file content in the memory buffer area.

[0089] Step 210, in a case where the memory buffer area is full, sequentially extracting the file content stored in the memory buffer area and sequentially writing the file content into the destination file address.

[0090] Step 211, in a case where the memory buffer area does not store the file content, entering the step of sequentially extracting the file content of the to-be-processed file from the source file address and sequentially storing the file content in the memory buffer area until all the file content of the source file address is extracted.

[0091] In the embodiment of the present application, for steps 208 to 211, the source file address of the to-be-processed file represents the position of the to-be-processed file before copying, and the destination file address of the to-be-processed file represents the position of the to-be-processed file after copying. The preset address condition represents that the source file address and the destination file address of the to-be-processed file satisfy a certain special condition, and when the condition is established, the subsequent parallel reading and writing operations can be used to speed up the copying process without negative effects.

[0092] Specifically, the preset address condition is that the source file address and the destination file address are not in the same disk. When the source file address and the destination file address of the to-be-processed file satisfy the preset address condition, the read-write operation can be executed in parallel, thereby improving the copying efficiency of the large file. However, since the head of the mechanical disk has a seek overhead, that is, a certain moving time is required for the head to move to a specific position, when the source file address and the destination file address of the to-be-processed file do not satisfy the preset address condition, that is, the source file address and the destination file address are in the same disk, if the read-write parallel copying strategy is used, the head of the disk needs to move back and forth repeatedly, which causes a performance decline.

[0093] To solve the problem, the embodiment of the present application can use the read-write serial copying strategy when the source file address and the destination file address of the to-be-processed file do not satisfy the preset address condition. As shown in FIG. 5:

[0094] M1, determine whether the to-be-processed file is a large file.

[0095] M2, call the kernel command.

[0096] M3, determine whether the source file address and the destination file address satisfy the preset address condition.

[0097] If yes, execute M4, execute the parallel strategy.

[0098] If no, execute M5, execute the serial strategy.

[0099] Specifically, the serial copying strategy is to first apply a memory buffer of a preset size for use. The form of the memory buffer can be a ring queue or a non-ring queue. Then, the processing process extracts the file content of the to-be-processed file from the source file address in sequence and stores the file content in the memory buffer in sequence. When the memory buffer is full, the processing process extracts the file content stored in the memory buffer in sequence and writes the file content to the destination file address in sequence. Then, when all the file content in the memory buffer is taken out, the processing process extracts the file content of the to-be-processed file from the source file address in sequence and stores the file content in the memory buffer in sequence. The process is repeated until all the file content of the source file address is copied to the destination file address.

[0100] The entire serial execution process can refer to FIG. 6, which includes:

[0101] N1, apply a memory buffer of a preset size.

[0102] N2, in the case that the memory buffer is empty, the processing process extracts the file content of the to-be-processed file from the source file address in sequence and stores the file content in the memory buffer in sequence.

[0103] N3, in the case that the memory buffer is full, the processing process extracts the file content stored in the memory buffer in turn and writes into the destination file address in turn.

[0104] N4, judging whether all file content of the source file address is read, if yes, exiting, if not, entering N2.

[0105] In summary, in the case that the file size of the to-be-processed file is greater than the preset threshold, the preset kernel command is executed, and in the case that the source file address and the destination file address of the to-be-processed file meet the preset address condition, the queue state of the preset ring queue is acquired, in the case that the queue state meets the first state condition, the file content of the to-be-processed file is extracted from the source file address in turn and stored in the ring queue in turn, and in the case that the queue state meets the second state condition, the file content stored in the ring queue is extracted in turn and written into the destination file address in turn, and the read-write process is repeatedly executed to complete the copy of the to-be-processed file. The whole large file copy process of the embodiment of the application is implemented in the kernel layer, and the interaction between the kernel layer and the application layer is not involved, so the system overhead is reduced. In addition, the process of buffering the data of the large file to the memory is also saved for the copy of the large file, the pressure of the file buffering system is reduced, and the stability of the file buffering system is improved.

[0106] Fig. 7 is a block diagram of a large file copy device provided by an embodiment of the application, which comprises:

[0107] The calling module 301 is configured to call the preset kernel command in the case that the file size of the to-be-processed file is greater than the preset threshold.

[0108] The state module 302 is configured to acquire the queue state of the preset ring queue in response to the kernel command and in the case that the source file address and the destination file address of the to-be-processed file meet the preset address condition; the queue state is used to represent the storage state of the ring queue.

[0109] The copy module 303 is configured to extract the file content of the to-be-processed file from the source file address in turn and store in the ring queue in turn in the case that the queue state meets the first state condition, and extract the file content stored in the ring queue in turn and write into the destination file address in turn in the case that the queue state meets the second state condition.

[0110] The termination module 304 is configured to complete the copy of the to-be-processed file in the case that the queue state meets the termination condition.

[0111] Optionally, the device further comprises:

[0112] The application module is configured to apply for a memory buffer of a preset size when it is determined that the source file address and the destination file address of the to-be-processed file do not satisfy a preset address condition.

[0113] The first processing module is configured to sequentially extract file content of the to-be-processed file from the source file address and sequentially store the file content in the memory buffer.

[0114] The second processing module is configured to sequentially extract the file content stored in the memory buffer and sequentially write the file content into the destination file address when the memory buffer is full.

[0115] The third processing module is configured to enter the step of sequentially extracting file content of the to-be-processed file from the source file address and sequentially storing the file content in the memory buffer when the memory buffer does not store the file content, until all the file content of the source file address is extracted.

[0116] Optionally, the preset address condition is that the source file address and the destination file address are not in the same disk.

[0117] Optionally, the ring queue includes a plurality of memory pages; and the reading process is configured to sequentially extract file content of the to-be-processed file from the source file address and sequentially store the file content in the ring queue.

[0118] The copy module 303 includes:

[0119] The reading submodule is configured to sequentially extract file content of the to-be-processed file from the source file address and sequentially store the file content in the empty memory page through the reading process when the queue state indicates that there is an empty memory page in the ring queue; the empty memory page indicates a memory page that does not store data content.

[0120] The first sleeping submodule is configured to control the reading process to sleep when the queue state indicates that there is no empty memory page in the ring queue.

[0121] Optionally, the ring queue includes a plurality of memory pages; and the writing process is configured to sequentially extract file content stored in the ring queue and sequentially write the file content into the destination file address.

[0122] The copy module 303 includes:

[0123] The writing submodule is configured to sequentially extract file content stored in the non-empty memory page through the writing process and sequentially write the file content into the destination file address when the queue state indicates that there is a non-empty memory page in the ring queue; the non-empty memory page indicates a memory page that stores data content.

[0124] a second dormancy sub-module, configured to control the write process to be dormant in a case that the queue state indicates that there is no non-empty memory page in the ring queue.

[0125] Optionally, the ring queue comprises a plurality of memory pages; and the termination module 304 comprises:

[0126] a termination sub-module, configured to, in a case that the read process generates an end flag, complete the copying of the to-be-processed file after the write process writes the file content stored in the remaining non-empty memory pages in the ring queue into the destination file address; the non-empty memory page represents a memory page that stores data content.

[0127] Optionally, the read process is configured to sequentially extract the file content of the to-be-processed file from the source file address and sequentially store the file content in the ring queue; and the write process is configured to sequentially extract the file content stored in the ring queue and sequentially write the file content into the destination file address; the end flag is generated by the read process when the read process reads the last data page in the source file address.

[0128] Optionally, the preset coefficient is any one of 0.6, 0.7 and 0.8.

[0129] Optionally, the kernel command is an ABI command.

[0130] Optionally, the size of the memory page is 4k.

[0131] Optionally, the size of the ring queue is 128M.

[0132] Optionally, the read process is configured to sequentially extract the file content of the to-be-processed file from the source file address through a block device interface provided by a kernel.

[0133] Optionally, the write process is configured to write the file content into the destination file address through a block device interface provided by a kernel.

[0134] Optionally, the read process and the write process are executed in parallel.

[0135] Optionally, the end flag is generated by the read process when the read process reads the last data page in the source file address.

[0136] Optionally, the preset threshold is 6GB.

[0137] Optionally, the preset threshold is a product of a total capacity of the memory and a preset coefficient, and the preset coefficient is 0.7.

[0138] In summary, in the case that the file size of the to-be-processed file is greater than the preset threshold, the preset kernel command is executed, and in the case that the source file address and the destination file address of the to-be-processed file satisfy the preset address condition, the queue state of the preset ring queue is acquired, in the case that the queue state satisfies the first state condition, the file content of the to-be-processed file is sequentially extracted from the source file address and sequentially stored in the ring queue, and in the case that the queue state satisfies the second state condition, the file content stored in the ring queue is sequentially extracted and sequentially written into the destination file address, and the read-write process is repeatedly executed to complete the copying of the to-be-processed file. The entire large file copying process of the embodiment of the present application is implemented in the kernel layer, and does not involve the interaction between the kernel layer and the application layer, so as to reduce the system overhead. In addition, the process of buffering the data of the large file to the memory is also saved for the copying of the large file, the pressure of the file buffering system is reduced, and the stability of the file buffering system is improved.

[0139] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts are described in the part of the method embodiment.

[0140] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts between the embodiments can be referred to each other.

[0141] For the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment related to the method, and will not be described in detail here.

[0142] The embodiment of the present application provides a large file copying device, which includes a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs contain a method for performing the method described in one or more embodiments.

[0143] FIG. 8 is a block diagram of a large file copying device 800 according to an exemplary embodiment. For example, the device 800 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0144] Referring to FIG. 8, the device 800 can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0145] The processing component 802 generally controls the overall operations of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions. In addition, the processing component 802 can include one or more modules to facilitate interaction with other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0146] The memory 804 is configured to store various types of data to support operations of the device 800. Examples of such data include instructions for any applications or methods operating on the device 800, contact data, phonebook data, messages, pictures, videos, and so on. The memory 804 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks, or optical disks.

[0147] The power component 806 provides power to the various components of the device 800. The power component 806 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0148] The multimedia component 808 includes a screen providing an output interface between the device 800 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensors can not only sense a boundary of a touching or sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. The front and / or rear camera can receive external multimedia data when the device 800 is in an operation mode, such as a shooting mode or a video mode. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0149] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive an external audio signal when the device 800 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0150] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which can include a keypad, a click wheel, buttons, and so on. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0151] The sensor component 814 includes one or more sensors for providing status assessments of various aspects of the device 800. For example, the sensor component 814 can detect an open / closed position of the device 800, relative positioning of components, such as a display and a keypad of the device 800, a change of position of the device 800 or a component of the device 800, presence or absence of user contact with the device 800, changes in orientation or acceleration / deceleration of the device 800, and temperature changes of the device 800, among other possibilities. The sensor component 814 can include proximity sensor configured to detect presence of an object in proximity to the device 800 without any physical touch. The sensor component 814 can also include a light sensor (e.g., a CMOS or CCD image sensor) configured to work in conjunction with an imaging application. In some embodiments, the sensor component 814 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0152] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a corresponding communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an example embodiment, the communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 816 can further include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0153] In exemplary embodiments, the apparatus 800 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic devices, to perform the above method.

[0154] In exemplary embodiments, a non-transitory computer readable storage medium including instructions, such as the memory 804 including instructions, is also provided, which can be executed by the processor 820 of the apparatus 800 to complete the above method. For example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0155] FIG. 9 is a schematic diagram of a structure of a server in some embodiments of the present application. The server 1900 can have a large difference due to different configurations or performances, and can include one or more central processing units (CPUs) 1922 (for example, one or more processors) and a memory 1932, one or more storage media 1930 (for example, one or more mass storage devices) storing application programs 1942 or data 1944. The memory 1932 and the storage media 1930 can be temporary storage or persistent storage. The programs stored in the storage media 1930 can include one or more modules (not shown in FIG. 9), each of which can include a series of instruction operations in the server. Further, the central processing unit 1922 can be configured to communicate with the storage media 1930 and execute a series of instruction operations in the storage media 1930 on the server 1900.

[0156] The server 1900 can also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input / output interfaces 1958, one or more keyboards 1956, and / or one or more operating systems 1941, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0157] Embodiments of the present application also provide a computer program product including a computer program, which, when executed by a processor, implements a large file copying method.

[0158] As used in this description, the terms "one embodiment", "an embodiment” or "one or more embodiments” mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment.

[0159] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been described in detail in order not to obscure the understanding of this description.

[0160] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. It is understood that the application can be implemented by means of both hardware and software, and that the application can be implemented in either hardware or software alone. In a unitary or integrated claim, items such as multiple parts, devices, apparatus, units, means, components and the like can be both reference to units and to materials. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. The

[0161] The above embodiments are only used to illustrate the technical solutions of the present application, not limit the technical solutions of the present application; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: still can be modified to the technical solutions recorded in the foregoing embodiments, or equivalent replacement is made to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

[0162] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present application cover any and all variations of the application that come within the scope of the claims and their equivalents. It is intended that the specification and examples be considered exemplary only, with the true scope and spirit of the application indicated by the following claims.

[0163] It is understood that the application is not limited to the precise construction and compositions disclosed herein and as such various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application should only be limited by the claims appended hereto.

[0164] The above merely preferred embodiments of the present application are not intended to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0165] The above describes in detail a large file copying method, device, electronic equipment, computer readable storage medium and computer program provided by the present application. The principles and implementation manners of the present application are described by applying specific examples. The above embodiment description is only used to help understand the method and core idea of the present application. Meanwhile, for those skilled in the art, the specific implementation manner and application range will be changed according to the idea of the present application. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A large file copying method, wherein, The method comprises: In the case that the file size of the file to be processed is greater than a preset threshold, a preset kernel command is invoked; In response to the kernel command, and in the case that the source file address and the destination file address of the file to be processed satisfy a preset address condition, a queue state of a preset ring queue is acquired; the queue state is used to represent the case that the ring queue stores data; In the case that the queue state satisfies a first state condition, the file content of the file to be processed is sequentially extracted from the source file address and sequentially stored in the ring queue, and in the case that the queue state satisfies a second state condition, the file content stored in the ring queue is sequentially extracted and sequentially written into the destination file address; In the case that the queue state satisfies a termination condition, the copying of the file to be processed is completed.

2. The large file copying method of claim 1, wherein, The method further comprises: In the case that the source file address and the destination file address of the file to be processed do not satisfy the preset address condition, a memory buffer area with a preset size is applied for; The file content of the file to be processed is sequentially extracted from the source file address and sequentially stored in the memory buffer area; In the case that the memory buffer area is full, the file content stored in the memory buffer area is sequentially extracted and sequentially written into the destination file address; In the case that the memory buffer area does not store the file content, the step of sequentially extracting the file content of the file to be processed from the source file address and sequentially storing the file content in the memory buffer area is entered until all the file content of the source file address is extracted.

3. The large file copying method according to claim 1 or 2, wherein, The preset address condition is that the source file address and the destination file address are not in the same disk.

4. The large file copying method of claim 1, wherein, The ring queue comprises a plurality of memory pages; a read process is used to sequentially extract the file content of the file to be processed from the source file address and sequentially store the file content in the ring queue; The step of sequentially extracting the file content of the file to be processed from the source file address and sequentially storing the file content in the ring queue in the case that the queue state satisfies the first state condition comprises: In the case that the queue state represents that there is an empty memory page in the ring queue, the read process is used to sequentially extract the file content of the file to be processed from the source file address and sequentially store the file content in the empty memory page; the empty memory page represents a memory page that does not store data content; In the case that the queue state represents that there is no empty memory page in the ring queue, the read process is controlled to sleep.

5. The large file copying method of claim 1, wherein, The ring queue comprises a plurality of memory pages; a write process is used to sequentially extract the file content stored in the ring queue and sequentially write the file content into the destination file address; The step of sequentially extracting the file content stored in the ring queue and sequentially writing the file content into the destination file address in the case that the queue state satisfies the second state condition comprises: In the case that the queue state represents that there is a non-empty memory page in the ring queue, the write process is used to sequentially extract the file content stored in the non-empty memory page and sequentially write the file content into the destination file address; the non-empty memory page represents a memory page that stores data content; In a case where the queue state indicates that the ring queue does not contain the non-empty memory page, the write process is controlled to sleep.

6. The large file copying method of claim 1, wherein, The ring queue includes a plurality of memory pages. In a case where the queue state satisfies a termination condition, copying of the to-be-processed file is completed. In a case where the read process generates an end flag, the write process completes copying of the to-be-processed file after writing file content stored in the remaining non-empty memory pages in the ring queue to the destination file address; the non-empty memory page indicates a memory page containing data content. The read process is configured to sequentially extract file content of the to-be-processed file from the source file address and sequentially store the file content in the ring queue, and the write process is configured to sequentially extract file content stored in the ring queue and sequentially write the file content to the destination file address.

7. The large file copying method of claim 1, wherein, The preset threshold is a product of a total capacity of the memory and a preset coefficient.

8. The large file copying method of claim 7, wherein, The preset coefficient is any one of 0.6, 0.7, and 0.

8.

9. The large file copying method of claim 1, wherein, The kernel command is an ABI command.

10. The method of copying a large file according to any one of claims 4-6, wherein, The size of the memory page is 4k.

11. The method of copying a large file according to any one of claims 4-6, wherein, The size of the ring queue is 128M.

12. The large file copying method according to claim 4 or 6, wherein, The read process is configured to sequentially extract file content of the to-be-processed file from the source file address through a block device interface provided by the kernel.

13. The large file copying method according to claim 5 or 6, wherein, The write process is configured to write file content to the destination file address through the block device interface provided by the kernel.

14. The large file copying method of claim 6, wherein, The read process and the write process are executed in parallel.

15. The large file copying method of claim 6, wherein, The end flag is generated by the read process when the read process reads a last data page in the source file address.

16. The large file copying method of claim 1, wherein, The preset threshold is 6GB.

17. A large file copying apparatus, wherein, The apparatus includes: a calling module configured to call a preset kernel command in a case where a file size of a to-be-processed file is greater than a preset threshold; a state module configured to, in response to the kernel command, acquire a queue state of a preset ring queue in a case where a source file address and a destination file address of the to-be-processed file satisfy a preset address condition; the queue state is used to indicate a storage state of the ring queue; a copying module configured to, in a case where the queue state satisfies a first state condition, sequentially extract file content of the to-be-processed file from the source file address and sequentially store the file content in the ring queue, and in a case where the queue state satisfies a second state condition, sequentially extract file content stored in the ring queue and sequentially write the file content to the destination file address; a termination module configured to, in a case where the queue state satisfies a termination condition, complete copying of the to-be-processed file.

18. An electronic device, comprising: includes: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method of any one of claims 1-16.

19. A computer readable storage medium, wherein, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the method of any one of claims 1-16.

20. A computer program comprising computer readable code which, when run on computing processing device, causes the computing processing device to perform the method of any one of claims 1-16.

Images