Cloud file export method and device

Abstract

The embodiment of the present application provides a method and device for exporting cloud files, wherein the method includes: obtaining a file export request, wherein the file export request is used to request to export the cloud file to a target location; responding to the file export request, executing a file acquisition task and executing a file export task, wherein the file acquisition task is a task for acquiring resource data of the cloud file, and the file export task is a task for exporting the resource data of the cloud file to a target location, and the file acquisition task and the file export task are in an asynchronous execution relationship. Through the present application, the problem of low export efficiency of cloud files is solved, thereby achieving the effect of improving the export efficiency of cloud files.

Description

Technical Field

[0001] The embodiments of the present application relate to the computer field, and more specifically, to a method and device for exporting cloud files. Background Art

[0002] As the scale of cloud resource usage grows, the amount of data is also increasing exponentially. Taking virtual machines as an example, in a large data center, the number of virtual machines can be tens of thousands or even hundreds of thousands.

[0003] The cloud file export method of the related technology usually adopts a synchronous method, that is, after the client initiates the export request, it needs to wait for the server to complete the processing before receiving the result. This method will cause the client to wait for a long time when there are many files, the file size is large, or the network bandwidth is limited, which seriously affects the user experience and leads to the problem of low efficiency in exporting cloud files. Therefore, there is a problem of low efficiency in exporting cloud files.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method and device for exporting cloud files, so as to at least solve the problem of low efficiency in exporting cloud files in the related art.

[0006] According to one embodiment of the present application, a method for exporting cloud files is provided, comprising: obtaining a file export request, wherein the file export request is used to request to export the cloud file to a target location; responding to the file export request, executing a file acquisition task and executing a file export task, wherein the file acquisition task is a task for acquiring resource data of the cloud file, and the file export task is a task for exporting the resource data of the cloud file to the target location, and the file acquisition task and the file export task are in an asynchronous execution relationship.

[0007] According to another embodiment of the present application, a cloud file exporting device is provided, comprising: a first acquisition unit, used to acquire a file export request, wherein the file export request is used to request to export the cloud file to a target location; a first execution unit, used to respond to the file export request, execute a file acquisition task and execute a file export task, wherein the file acquisition task is a task of acquiring resource data of the cloud file, and the file export task is a task of exporting the resource data of the cloud file to the target location, and there is an asynchronous execution relationship between the file acquisition task and the file export task.

[0008] In an exemplary embodiment, the above-mentioned first execution unit includes: a first execution module, used to utilize a distributed service architecture to respond to the above-mentioned file export request, execute the above-mentioned file acquisition task and the above-mentioned file export task, wherein the above-mentioned distributed service architecture is configured with multiple copies, and the above-mentioned copies are used to respond to the above-mentioned file migration request, and the above-mentioned file migration request includes the above-mentioned file export request.

[0009] In an exemplary embodiment, the first execution module includes: an acquisition submodule, used to acquire the first thread of the main service copy, wherein the main service copy is a copy used to respond to the file export request among the multiple copies, the multiple copies include the main service copy, the first thread is a thread configured by the main service copy and is in an idle state, the main service copy is configured with at least two service threads, the at least two service threads are used to respond to concurrent file migration requests, the idle state is a thread state waiting to respond to the file migration request, the service thread is the first thread or the second thread, the second thread is a thread in a working state, and the working state is a thread state responding to the file migration request; an execution submodule, used to utilize the first thread to respond to the file export request and execute the file acquisition task and the file export task.

[0010] In an exemplary embodiment, the acquisition submodule includes: a first determination subunit, used to determine a master service copy from the multiple copies as a copy among the multiple copies for responding to the file export request, wherein the master service copy is the first copy among the multiple copies to obtain the dominant service lock, and each copy among the multiple copies is set to attempt to obtain the dominant service lock after receiving the file migration request.

[0011] In an exemplary embodiment, the device further includes: a query subunit, which is used to query the data volume corresponding to the resource data of the cloud file after the master service copy is determined from the multiple copies as the copy for responding to the file export request among the multiple copies; a second determination subunit, which is used to segment the resource data of the cloud file according to the data volume to obtain multiple resource sub-data after the master service copy is determined from the multiple copies as the copy for responding to the file export request among the multiple copies; a calling subunit, which is used to segment the resource data of the cloud file according to the data volume to obtain multiple resource sub-data after the master service copy is determined from the multiple copies as the copy for responding to the file export request among the multiple copies; and After the copy, at least one copy other than the above-mentioned main service copy is called from the above-mentioned multiple copies, and each sub-copy in the above-mentioned at least one copy is indicated with an offset corresponding to the resource sub-data in the above-mentioned multiple resource sub-data that needs to be processed, wherein the above-mentioned offset is used to indicate that each of the above-mentioned sub-copies exports the above-mentioned resource sub-data corresponding to each of them; a merging sub-unit is used to, after the main service copy is determined from the above-mentioned multiple copies as the copy in the above-mentioned multiple copies for responding to the above-mentioned file export request, merge the resource sub-data corresponding to the exports of each of the above-mentioned sub-copies through the above-mentioned main service copy to obtain complete resource data, wherein the resource data of the above-mentioned cloud file includes the above-mentioned complete resource data.

[0012] In an exemplary embodiment, the second determination sub-unit includes: an acquisition component for acquiring the number of copies currently allowed to be called among the multiple copies, wherein the number of copies is the number of copies in the at least one copy; a combining component for combining the number of copies and the amount of data to segment the resource data of the cloud file to obtain the multiple resource sub-data.

[0013] In an exemplary embodiment, the above-mentioned device also includes: an acquisition sub-unit, which is used to acquire the above-mentioned corresponding exported resource sub-data transmitted to the above-mentioned master service copy before merging the resource sub-data corresponding to each of the above-mentioned sub-copies through the above-mentioned master service copy to obtain the complete resource data, wherein the resource transmission address of the above-mentioned master service copy is pre-stored in the service database, and the above-mentioned service database is used to provide corresponding resource transmission addresses for each of the above-mentioned sub-copies.

[0014] In an exemplary embodiment, the above-mentioned device also includes: a second execution unit, which is used to execute an exception handling task when export exception information is obtained in the process of responding to the above-mentioned file export request, executing the file acquisition task and executing the file export task, wherein the above-mentioned export exception information is used to indicate that there is an abnormal situation during the execution of the above-mentioned file acquisition task or the above-mentioned file export task, and the above-mentioned exception handling task is used to handle the above-mentioned abnormal situation.

[0015] In an exemplary embodiment, the above-mentioned second execution unit includes at least one of the following: a second execution module, used to re-execute the task in an abnormal state or the above-mentioned file export task; a third execution module, used to roll back the corresponding data of the above-mentioned task in an abnormal state or the above-mentioned file export task to before the above-mentioned abnormal situation occurs.

[0016] In an exemplary embodiment, the above-mentioned device also includes: a second acquisition unit, used to acquire the above-mentioned exported exception information before the above-mentioned exception handling task is executed; a classification unit, used to classify the above-mentioned exported exception information before the above-mentioned exception handling task is executed, and obtain the classified exported exception information, wherein different types of the above-mentioned abnormal situations correspond to different types of the above-mentioned exported exception information; the above-mentioned second execution unit includes: a fourth execution module, used to execute the above-mentioned exception handling task according to the classified exported exception information, wherein different types of the above-mentioned exported exception information correspond to different types of the above-mentioned exception handling tasks.

[0017] In an exemplary embodiment, the above-mentioned device also includes at least one of the following: a first display unit, used to display first task information corresponding to the above-mentioned file acquisition task in the process of responding to the above-mentioned file export request, executing the file acquisition task and executing the file export task, wherein the above-mentioned first task information is used to indicate the execution status of the above-mentioned file acquisition task; a second display unit, used to display second task information corresponding to the above-mentioned file acquisition task in the process of responding to the above-mentioned file export request, executing the file acquisition task and executing the file export task, wherein the above-mentioned second task information is used to indicate the execution progress of the above-mentioned file acquisition task; a third display unit, used to display third task information corresponding to the above-mentioned file export task in the process of responding to the above-mentioned file export request, executing the file acquisition task and executing the file export task, wherein the above-mentioned third task information is used to indicate the execution status of the above-mentioned file export task; a fourth display unit, used to display fourth task information corresponding to the above-mentioned file export task in the process of responding to the above-mentioned file export request, executing the file acquisition task and executing the file export task, wherein the above-mentioned fourth task information is used to indicate the execution progress of the above-mentioned file export task.

[0018] In an exemplary embodiment, the above-mentioned device also includes: a first adjustment unit, which is used to adjust or manage the execution status of the above-mentioned file acquisition task according to the above-mentioned first task information after the above-mentioned first task information corresponding to the above-mentioned file acquisition task is displayed; the above-mentioned device also includes: a second adjustment unit, which is used to adjust or manage the execution progress of the above-mentioned file acquisition task according to the above-mentioned second task information after the above-mentioned second task information corresponding to the above-mentioned file acquisition task is displayed; the above-mentioned device also includes: a third adjustment unit, which is used to adjust or manage the execution status of the above-mentioned file export task according to the above-mentioned third task information after the above-mentioned third task information corresponding to the above-mentioned file export task is displayed; the above-mentioned device also includes: a fourth adjustment unit, which is used to adjust or manage the execution progress of the above-mentioned file export task according to the above-mentioned fourth task information after the above-mentioned fourth task information corresponding to the above-mentioned file export task is displayed.

[0019] In an exemplary embodiment, the above-mentioned device also includes: a verification unit, which is used to perform a permission check on the above-mentioned file export request after the above-mentioned file export request is obtained; a first setting unit, which is used to set the above-mentioned file export request to a prohibited response if the above-mentioned permission check fails after the above-mentioned file export request is obtained; and a second setting unit, which is used to set the above-mentioned file export request to a permitted response if the above-mentioned permission check succeeds after the above-mentioned file export request is obtained.

[0020] In an exemplary embodiment, the above-mentioned device also includes: a third setting unit, which is used to set the above-mentioned file export request as an allowed response after the above-mentioned file export request is obtained, if the request identifier corresponding to the above-mentioned file export request is in the white list; a fourth setting unit, which is used to set the above-mentioned file export request as a prohibited response after the above-mentioned file export request is obtained, if the request identifier corresponding to the above-mentioned file export request is in the black list; the above-mentioned device also includes: a fifth setting unit, which is used to place the request identifier corresponding to the above-mentioned file export request in the above-mentioned black list during the process of setting the above-mentioned file export request to a prohibited response when the above-mentioned permission check fails.

[0021] In an exemplary embodiment, the above-mentioned device also includes: a third acquisition unit, which is used to, in the process of responding to the above-mentioned file export request, executing the file acquisition task and executing the file export task, in the case that other file migration requests other than the above-mentioned file export request are acquired, obtain the first response priority corresponding to the above-mentioned other file migration request; a sixth setting unit, which is used to, in the process of responding to the above-mentioned file export request, executing the file acquisition task and executing the file export task, in the case that the first response priority is lower than the second response priority corresponding to the above-mentioned file export request, set the above-mentioned other file migration request to be responded after the above-mentioned file export request response; a seventh setting unit, which is used to, in the process of responding to the above-mentioned file export request, executing the file acquisition task and executing the file export task, in the case that the first response priority is higher than the second response priority corresponding to the above-mentioned file export request, give priority to responding to the above-mentioned other file migration request.

[0022] In an exemplary embodiment, the above-mentioned device also includes: a first deletion unit, which is used to delete the request information corresponding to the above-mentioned file export request after responding to the above-mentioned file export request, executing the file acquisition task and executing the file export task, and when the task completion information is obtained, wherein the above-mentioned task completion information is used to indicate that the above-mentioned file acquisition task and the above-mentioned file export task are completed.

[0023] In an exemplary embodiment, the above-mentioned device also includes: a second deletion unit, which is used to delete temporary data in the process of executing the above-mentioned file acquisition task and the above-mentioned file export task after responding to the above-mentioned file export request, executing the file acquisition task and executing the file export task, when the task completion information is obtained, wherein the above-mentioned task completion information is used to indicate that the above-mentioned file acquisition task and the above-mentioned file export task are completed.

[0024] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when running.

[0025] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0026] Through this application, asynchronously executed file acquisition tasks and file export tasks are adopted. Specifically, when a file export request is received, the file acquisition task is executed first, that is, the file resource data required to be exported is obtained from the cloud. This process is asynchronous, which means that the client can continue with other operations without waiting for the task to be completed. At the same time, the file acquisition task will run independently in the background and will not block the execution of other tasks. During the execution of the file acquisition task, the file export task will also be executed synchronously to export the acquired cloud file resource data to the target location. Since the file acquisition task and the file export task are executed asynchronously, the file export task will not be affected by the execution time of the file acquisition task, which can solve the problem of low export efficiency of cloud files, thereby achieving the technical effect of improving the export efficiency of cloud files. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a schematic diagram of an application environment of a method for exporting cloud files according to an embodiment of the present application;

[0028] Figure 2 is a flow chart of a method for exporting cloud files according to an embodiment of the present application;

[0029] Figure 3 is a schematic diagram of a method for exporting cloud files according to an embodiment of the present application;

[0030] Figure 4 It is a structural block diagram of a cloud file export device according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0033] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a server device of a cloud file export method according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0034] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for exporting cloud files in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] In this embodiment, a method for exporting cloud files is provided. Figure 2 is a flow chart of a method for exporting cloud files according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:

[0037] Step S202, obtaining a file export request, wherein the file export request is used to request to export a cloud file to a target location;

[0038] In an exemplary embodiment, a file export request may be an operation request initiated by a user or a system, which is intended to retrieve and transfer a specific file from a cloud storage system to a local storage or other remote storage location specified by the user. This instruction may come from a user or other system component, and the purpose is to request that the file data stored in the cloud be exported to a specified location. In other words, the user or the system hopes to migrate the cloud file to a specific location through this request.

[0039] For example, a company uses a cloud storage service to store a large amount of documents and data. When an employee needs to download these materials to a local computer for further analysis or editing, he will initiate a "file export request" through a cloud service platform or application. This request will clearly specify which files need to be exported and to which specific folder or storage device these files should be exported.

[0040] In an exemplary embodiment, a file export request may include various parameters, such as a file identifier (such as a file name, file ID, etc.), path information of a target storage location, format requirements for the exported file (such as whether format conversion is required), security or permission verification information, etc. After receiving the request, the system will perform a series of verification and processing procedures to ensure that the request is legal and that the export operation is accurately performed according to the parameters in the request.

[0041] After receiving the file export request, the system will start to execute the export task according to the detailed information in the request. During this process, users will feel that the system begins to respond to their operation needs and will see relevant progress prompts or feedback. Once the export is completed, users can find the files they requested to export in the specified target location for subsequent work or analysis. The whole process is transparent and efficient for users, greatly improving work efficiency and user experience.

[0042] Step S204, responding to the file export request, executing the file acquisition task and the file export task, wherein the file acquisition task is a task of acquiring resource data of the cloud file, and the file export task is a task of exporting the resource data of the cloud file to the target location, and the file acquisition task and the file export task are in an asynchronous execution relationship.

[0043] In an exemplary embodiment, when the system "responds to a file export request", it can be used to indicate that the system has begun processing a user's request to export a cloud file to a specific location. During this process, the system will "execute a file acquisition task", that is, obtain the file data requested by the user from the cloud, and will "execute a file export task" to export this data to the target location specified by the user. Importantly, the "file acquisition task" and the "file export task" are not performed in a strict order, but rather use an "asynchronous execution relationship" to indicate that the two tasks can be performed in parallel without affecting each other, thereby improving overall processing efficiency.

[0044] By responding to file export requests and asynchronously executing file acquisition tasks and file export tasks, the system can handle users' export needs more efficiently. Instead of waiting for a long time for the files to be fully prepared before starting to download, users can start receiving file data almost immediately. This processing method significantly improves the user experience, especially when dealing with large or large files. At the same time, it also optimizes the system's resource utilization, allowing the server to handle multiple concurrent requests more efficiently.

[0045] Optionally, responding to a file export request can be understood as the system's reaction and processing of an operation request issued by a user or other system component to export a cloud file to a specified location.

[0046] Optionally, the file acquisition task can be understood as a task in which the system retrieves and obtains file resource data requested to be exported by the user from the cloud storage.

[0047] Optionally, the file export task can be understood as a task in which the system transfers the acquired cloud file resource data to a target location specified by the user.

[0048] Optionally, the asynchronous execution relationship can be understood as, during the file export process, the file acquisition task and the file export task can be performed independently without waiting for each other to complete. This parallel processing relationship is called an asynchronous execution relationship.

[0049] Through the above steps, asynchronously executed file acquisition tasks and file export tasks are adopted. Specifically, when a file export request is received, the file acquisition task is executed first, that is, the file resource data required to be exported is obtained from the cloud. This process is asynchronous, which means that the client can continue with other operations without waiting for the task to be completed. At the same time, the file acquisition task will run independently in the background and will not block the execution of other tasks. During the execution of the file acquisition task, the file export task will also be executed synchronously to export the acquired cloud file resource data to the target location. Since the file acquisition task and the file export task are executed asynchronously, the file export task will not be affected by the execution time of the file acquisition task, and the problem of low export efficiency of cloud files can be solved, thereby achieving the technical purpose of making full use of the multi-core processor and parallel computing capabilities of the server, and then achieving the technical effect of improving the export efficiency of cloud files.

[0050] The execution subject of the above steps may be a server, a terminal, etc., but is not limited thereto.

[0051] The execution order of step S202 and step S204 is interchangeable, that is, step S204 may be executed first, and then step S202.

[0052] As an optional solution, responding to a file export request, executing a file acquisition task and executing a file export task include:

[0053] The distributed service architecture is used to respond to file export requests and execute file acquisition tasks and file export tasks, wherein the distributed service architecture is configured with multiple copies, and the copies are used to respond to file migration requests, and the file migration requests include file export requests.

[0054] In an optional embodiment, the distributed service architecture can be a system architecture that distributes services to multiple nodes or servers, and each node can process requests independently. This architecture improves the scalability, fault tolerance and performance of the system.

[0055] In an optional embodiment, the replica may refer to redundant nodes or service instances in a distributed service architecture, which store the same data or services to ensure high availability and load balancing of data.

[0056] It should be noted that this embodiment describes the operation flow of a system or platform when processing a file export request. Specifically, when the system receives a file export request, it will perform two main tasks: file acquisition and file export. These two tasks are completed through a distributed service architecture.

[0057] The distributed service architecture not only improves the efficiency and speed of file processing, but also ensures the high availability of the system. Even if a node or server fails, other nodes or servers can still continue to provide services, ensuring that tasks such as file export are not affected.

[0058] In addition, the existence of multiple copies also enhances data security. Even if one copy is damaged or lost, other copies still contain complete data, thus reducing the risk of data loss.

[0059] To further illustrate, let's assume that there is a large enterprise whose data is stored on different servers and nodes. When an employee wants to export an important file, he sends a file export request to the system. At this time, a node in the distributed service architecture will receive the request, and then, based on the details of the request, it will get the file from the corresponding node or replica and finally export it to the user.

[0060] Processing file export requests through a distributed service architecture can greatly improve processing speed and efficiency, ensuring that users can quickly and accurately obtain the required files. At the same time, the configuration of multiple copies also enhances system stability and data security, providing users with a reliable and efficient file export service.

[0061] As an optional solution, a distributed service architecture is used to respond to file export requests and execute file acquisition tasks and file export tasks, including:

[0062] S1-1, obtaining the first thread of the primary service copy, wherein the primary service copy is a copy used to respond to file export requests among multiple copies, the multiple copies include the primary service copy, the first thread is a thread configured by the primary service copy and in an idle state, the primary service copy is configured with at least two service threads, the at least two service threads are used to respond to concurrent file migration requests, the idle state is a thread state to be responded to file migration requests, the service thread is the first thread or the second thread, the second thread is a thread in a working state, and the working state is a thread state that is responding to file migration requests;

[0063] S1-2, using the first thread, responding to the file export request, executing the file acquisition task and the file export task.

[0064] In an optional embodiment, the master service copy may be a service instance among multiple copies that is specifically used to respond to file export requests.

[0065] In an optional embodiment, the service thread may be a thread configured in the primary service copy for processing file migration requests.

[0066] In an optional embodiment, the first thread may refer to a thread in the primary service copy that is in an idle state and is waiting to respond to a file migration request.

[0067] In an optional embodiment, the idle state can be used to indicate that the thread is not currently executing a task and is waiting for a new request; the working state can be used to indicate that the thread is processing a request.

[0068] It should be noted that this embodiment describes how to use the master service copy in the distributed service architecture to respond to file export requests and execute file acquisition and file export tasks.

[0069] In a distributed system, the design of the master service replica is crucial because it directly affects the performance and response speed of the system. By properly configuring the number of service threads, the system can handle concurrent file migration requests more efficiently, thereby improving the user experience. In addition, this architecture can also dynamically adjust the state of threads according to actual needs to achieve optimal utilization of resources.

[0070] To further illustrate, consider a large library system that uses a distributed service architecture to manage book borrowing. When a reader wants to export his or her borrowing record, the system will find the master service replica that is responsible for handling such requests. This master service replica has multiple replica threads, and one of the idle threads (i.e., the first thread) will be selected to respond to the export request. This thread will perform necessary operations, such as obtaining the reader's borrowing record from the database and exporting it to the format required by the reader.

[0071] Using the master service copy in the distributed service architecture to respond to file export requests can significantly improve the system's processing power and response speed. By properly scheduling idle threads (first threads) to process requests, the system can perform file acquisition and file export tasks more efficiently, thereby improving the user experience. At the same time, this architecture also has good scalability and flexibility, and can be adjusted and optimized according to actual needs.

[0072] As an optional solution, obtaining the first thread of the primary service copy includes:

[0073] A primary service copy is determined from multiple copies as the copy used to respond to file export requests, wherein the primary service copy is the first copy among the multiple copies to obtain the dominant service lock, and each copy among the multiple copies is set to try to obtain the dominant service lock after receiving the file migration request.

[0074] In an optional embodiment, the dominant service lock can be a synchronization mechanism used to determine a primary service replica among multiple replicas. When a replica receives a file migration request, it will try to acquire this lock. The first replica that successfully acquires the lock will become the primary service replica.

[0075] It should be noted that this embodiment describes how to determine a primary service copy from multiple copies in a distributed service architecture and obtain the first thread in the primary service copy to process a file export request. The key lies in the mechanism of the "dominant service lock", which is used to determine a primary service copy among multiple copies.

[0076] The dominant service lock mechanism ensures that only one replica is used as the primary service replica to process requests at the same time, which avoids conflicts and data inconsistencies caused by multiple replicas processing the same request at the same time. This mechanism is a common synchronization strategy in distributed systems to ensure data consistency and system stability.

[0077] To further illustrate, let's assume that there are five service replicas, all waiting to process a file migration request. When a request arrives, all five replicas will try to acquire the dominant service lock. Assuming replica A is the first replica to successfully acquire the lock, replica A becomes the primary service replica and is responsible for processing this file export request.

[0078] By using the dominant service lock to determine the primary service replica, the system can ensure that only one replica is responsible for processing file export requests at a time, thus avoiding resource waste and potential conflicts. This mechanism improves system reliability and data consistency, providing users with a stable and efficient service. At the same time, it also makes the system more flexible, and can easily expand or reduce the number of service replicas to adapt to different workloads.

[0079] As an optional solution, after determining a primary service replica from the multiple replicas as a replica for responding to the file export request among the multiple replicas, the method further includes:

[0080] S2-2, query the data volume corresponding to the resource data of the cloud file;

[0081] S2-3, according to the data volume, the resource data of the cloud file is divided into data slices to obtain multiple resource sub-data;

[0082] S2-4, calling at least one replica other than the main service replica from the multiple replicas, and indicating the offset corresponding to the resource sub-data in the multiple resource sub-data that each sub-replica in the at least one replica needs to process, wherein the offset is used to indicate each sub-replica to export the corresponding resource sub-data;

[0083] S2-5, through the main service copy, the resource sub-data corresponding to the exports of each sub-copy are merged to obtain complete resource data, wherein the resource data of the cloud file includes the complete resource data.

[0084] In an optional embodiment, the data volume may refer to the size of a cloud file or the total amount of data contained therein.

[0085] In an optional embodiment, data sharding may be dividing a large file into multiple smaller data blocks (resource sub-data) to facilitate parallel processing and transmission.

[0086] In an optional embodiment, the offset may indicate the starting position of each resource sub-data in the original large file, ensuring that each sub-copy can accurately process the corresponding part.

[0087] It should be noted that this embodiment describes a strategy for processing large file exports in a distributed service architecture. When a large file needs to be exported from the cloud, the primary service copy is first determined, and then a series of operations are performed to ensure efficient and stable export of the file. The specific steps include querying the data volume of the file, data sharding, calling other copies to assist in processing, and finally merging the data.

[0088] This embodiment makes full use of the parallel processing capability of the distributed system. By dividing a large file into small pieces and assigning them to multiple copies for processing, the speed and efficiency of file export can be significantly improved. At the same time, the integrity and accuracy of the data are ensured through the precise control of the offset.

[0089] To further illustrate, optionally assume that there is a 1GB cloud file that needs to be exported. The system first determines a primary service replica and then queries the size of the file. Next, the system may split the file into 10 100MB resource sub-data. In addition to the primary service replica, the system will call the other 9 replicas, each of which processes a resource sub-data and exports the corresponding part based on the offset. Finally, the primary service replica merges all exported resource sub-data into a complete 1GB file.

[0090] Through data sharding and distributed processing, the performance bottleneck problem encountered when exporting large files is effectively solved. It not only improves the speed of file export, but also ensures the integrity and accuracy of the exported files. In addition, this method has good scalability and flexibility, and can dynamically adjust the number and size of data shards, as well as the number of copies involved in processing, according to the size of the file and system resources.

[0091] As an optional solution, the resource data of the cloud file is sharded according to the data volume to obtain multiple resource sub-data, including:

[0092] S3-1, obtaining the number of replicas currently allowed to be called among multiple replicas, wherein the number of replicas is the number of replicas in at least one replica;

[0093] S3-2, based on the number of copies and the amount of data, the resource data of the cloud file is sharded to obtain multiple resource sub-data.

[0094] In an optional embodiment, the number of replicas may be the number of replicas that are currently available to process tasks in a distributed service architecture.

[0095] It should be noted that this embodiment describes how to perform data sharding according to the data volume of the cloud file and the number of currently available copies to obtain multiple resource sub-data. Data sharding is to divide a large data file into multiple smaller data blocks for parallel processing in a distributed system.

[0096] Data sharding is a common strategy for distributed systems to process large data files. By splitting large files into small pieces, the system can use multiple nodes or replicas to process these data pieces in parallel, thereby improving processing speed and efficiency. In addition, data sharding also helps improve the fault tolerance of the system, because even if a replica fails during processing, it will only affect a small part of the data without affecting the processing of the entire file.

[0097] To further illustrate, let's assume that there is a 500MB cloud file that needs to be exported, and there are 5 available copies in the current distributed system. According to the method described in this paragraph, the system will first obtain the number of these 5 available copies, and then combine the file's data size (500MB) to split the file into data slices. For example, each resource sub-data can be 100MB, so the original file is split into 5 100MB resource sub-data.

[0098] By sharding data based on the amount of data and the number of available copies, the system can more effectively utilize distributed resources and improve the parallelism and speed of file processing. This approach not only optimizes resource utilization, but also enhances the fault tolerance and scalability of the system.

[0099] As an optional solution, before merging the resource sub-data corresponding to the exports of each sub-copy through the master service copy to obtain the complete resource data, the method further includes:

[0100] The resource sub-data corresponding to the export are obtained and transmitted to the main service copy, wherein the resource transmission address of the main service copy is pre-stored in the service database, and the service database is used to provide the corresponding resource transmission address for each sub-copy.

[0101] In an optional embodiment, resource sub-data may refer to the portion of data processed by each sub-replica after data sharding.

[0102] In an optional embodiment, the service database may be a database for storing and managing service-related information, specifically a database storing the resource transfer address of the primary service copy.

[0103] In an optional embodiment, the resource transfer address may be a network address used by the primary service replica to receive resource sub-data.

[0104] It should be noted that this embodiment describes a key step in the distributed service architecture before each sub-copy transmits the processed resource sub-data to the main service copy. It involves the process of obtaining and transmitting these resource sub-data to the main service copy, and the role of the service database in this process.

[0105] The existence of the service database makes the system more flexible and configurable. By centrally managing information such as resource transfer addresses, the system can easily cope with changes or expansions of the primary service replica. In addition, this design also improves the security of the system, because only the child replica that knows the correct resource transfer address can communicate with the primary service replica.

[0106] To further illustrate, in a distributed file processing system, suppose there are 5 child replicas that process 5 parts of a large file respectively. After processing, these child replicas need to send the processed data parts (resource sub-data) to the main service replica for merging. In order to complete this transmission, each child replica will query the service database, find the resource transmission address of the main service replica, and then send the data to this address.

[0107] By using the service database to store and provide the resource transfer address of the primary service copy, the system can ensure that the resource sub-data is accurately and efficiently transferred to the primary service copy. This approach not only simplifies the data transfer process, but also improves the scalability and security of the system.

[0108] As an optional solution, in the process of responding to the file export request, executing the file acquisition task and executing the file export task, the method further includes:

[0109] When the export exception information is obtained, an exception handling task is executed, wherein the export exception information is used to indicate that an exception exists during the execution of the file acquisition task or the file export task, and the exception handling task is used to handle the exception.

[0110] In an optional embodiment, the export exception information may refer to information generated by an abnormal situation detected by the system during the execution of a file acquisition task or a file export task.

[0111] In an optional embodiment, the exception handling task may refer to a series of operations performed by the system after an abnormal situation is detected, which is intended to resolve or mitigate the impact of the abnormal situation and ensure the stability of the system and the integrity of the data.

[0112] It should be noted that this embodiment describes how the system handles exceptions encountered during the process of responding to file export requests, executing file acquisition tasks, and file export tasks. Exceptions can occur during file acquisition or file export, at which point the system generates export exception information and triggers exception handling tasks to handle these exceptions.

[0113] Among them, exception handling is a very important part of distributed systems. Since distributed systems involve multiple nodes and complex network communications, various abnormal situations are more likely to occur. A robust distributed system should be able to detect these abnormal situations and trigger corresponding exception handling tasks in a timely manner to ensure the stability and availability of the system.

[0114] To further illustrate, let's assume that during the file export process, the file acquisition fails due to a network failure. At this time, the system will generate an export exception message, indicating that there is an exception during the execution of the file acquisition task. Subsequently, the system will trigger an exception handling task, such as retrying to obtain the file, recording an error log, notifying the administrator, etc., to resolve this exception.

[0115] By introducing an exception handling mechanism in the process of responding to file export requests, the system can more robustly cope with various abnormal situations. This not only improves the reliability of the system, but also ensures the consistency and integrity of the data.

[0116] As an optional solution, performing an exception handling task includes at least one of the following:

[0117] S4-1, re-execute the task or file export task in the abnormal state;

[0118] S4-2, rolling back the corresponding data of the task or file export task in the abnormal state to the time before the abnormal situation occurs.

[0119] In an optional embodiment, re-executing a task may mean that when a task fails due to an abnormal situation, the system attempts to re-execute the task in the hope of successfully completing it.

[0120] In an optional embodiment, data rollback may refer to restoring the data corresponding to the task to a state before the abnormal situation occurs to ensure the consistency and integrity of the data.

[0121] It should be noted that this embodiment describes the specific operations that can be included in "executing an exception handling task". When the system encounters an abnormal situation when executing a task (such as a file acquisition task or a file export task), the exception handling task will be triggered to try to recover or correct this abnormal state.

[0122] To further illustrate, suppose that during the file export process, the file transfer is interrupted due to temporary network fluctuations. At this time, the system will detect the exception and trigger the exception handling task. As part of the exception handling, the system may try to re-execute the file export task to ensure that the file can be successfully exported. In other words, if the file export task modifies some key data during execution, but is then interrupted due to an exception, the system may choose to roll back the data to the state before the exception occurred to avoid data inconsistency or corruption.

[0123] By performing exception handling tasks, such as re-executing tasks or rolling back data, the system can respond quickly and effectively when encountering abnormal situations. This not only improves the fault tolerance and stability of the system, but also ensures the integrity and consistency of the data.

[0124] As an optional solution, before executing the exception handling task, the method further includes:

[0125] S5-1, obtain export exception information;

[0126] S5-2, classifying the export exception information to obtain classified export exception information, wherein different types of exception situations correspond to different types of export exception information;

[0127] Executing the exception handling task includes: executing the exception handling task according to the classified exported exception information, wherein different types of exported exception information correspond to different types of exception handling tasks.

[0128] In an optional embodiment, the classified exported exception information may refer to the information obtained after classifying the original exported exception information according to its nature or cause, and further executing corresponding processing measures based on the classified exported exception information to solve or alleviate the impact of the abnormal situation.

[0129] It should be noted that this embodiment describes that before executing the exception handling task, the system needs to first obtain the export exception information and classify it according to the type of this information. Different types of exception situations will generate different types of export exception information, and each type of export exception information will correspond to different types of exception handling tasks.

[0130] Classifying the exported exception information and executing the corresponding exception handling tasks is the key to ensuring that the system can effectively respond to different types of exceptions. Through refined exception handling strategies, the system can more efficiently resolve exceptions and improve system stability and reliability.

[0131] To further illustrate, it is optional to assume that during the file export process, two abnormal situations are encountered: one is the network connection abnormality, and the other is the file reading and writing abnormality. When the system detects these abnormalities, it will generate corresponding export abnormality information. For network connection abnormalities, the system will classify them as "network type abnormalities", and for file reading and writing abnormalities, they will be classified as "file operation type abnormalities". Based on these classified export abnormality information, the system will perform different types of exception handling tasks. For example, for "network type abnormalities", the system will try to reconnect to the network or switch to a backup network line; and for "file operation type abnormalities", the system will try to repair the file or re-read the file.

[0132] By classifying the exported exception information and executing the corresponding type of exception handling tasks according to the classification results, the system can more accurately deal with different types of exceptions. This refined exception handling method not only improves the stability and reliability of the system, but also ensures that the exceptions can be resolved in a timely and effective manner.

[0133] As an optional solution, in the process of responding to the file export request, executing the file acquisition task and executing the file export task, the method further includes at least one of the following:

[0134] S6-1, displaying first task information corresponding to the file acquisition task, wherein the first task information is used to indicate the execution status of the file acquisition task;

[0135] S6-2, displaying second task information corresponding to the file acquisition task, wherein the second task information is used to indicate the execution progress of the file acquisition task;

[0136] S6-3, displaying third task information corresponding to the file export task, wherein the third task information is used to indicate the execution status of the file export task;

[0137] S6-4, displaying fourth task information corresponding to the file export task, wherein the fourth task information is used to indicate the execution progress of the file export task.

[0138] It should be noted that this embodiment describes the task information and execution progress feedback mechanism that the system may provide in the process of responding to the file export request and executing the file acquisition task and the file export task. Specifically, the system will display the execution status and execution progress information related to the file acquisition task and the file export task.

[0139] Providing task execution status and progress information is very important for users, because it can help users understand the completion status of the current task, estimate how long it will take to wait, and respond in time when encountering problems. This feedback mechanism also enhances the interactivity and user experience of the system.

[0140] Further exemplification, for example, when a user requests to export a large file, the system starts to execute the file acquisition task. During this process, the system interface may display the first task information such as "File acquisition in progress", indicating the execution status of the file acquisition task.

[0141] At the same time, a progress bar may also be displayed on the interface, which gradually fills up as the file acquisition progresses. This is an example of the second task information, indicating the specific progress of the file acquisition.

[0142] When the file acquisition is completed, the system starts to execute the file export task. At this time, the interface may be updated to "File export in progress", which is an example of the third task information, reflecting the execution status of the file export task.

[0143] Similar to the file acquisition task, the file export task can also be accompanied by a progress bar to display the progress of the export, which is the practical application of the fourth task information.

[0144] By displaying the execution status and progress information of file acquisition tasks and file export tasks, the system provides users with a transparent operating environment. Users can arrange their time reasonably based on this information and learn about the progress of tasks in a timely manner, thereby improving work efficiency and satisfaction.

[0145] As an optional solution, after displaying the first task information corresponding to the file acquisition task, the method further includes: adjusting or managing the execution state of the file acquisition task according to the first task information;

[0146] After displaying the second task information corresponding to the file acquisition task, the method further includes: adjusting or managing the execution progress of the file acquisition task according to the second task information;

[0147] After displaying the third task information corresponding to the file export task, the method further includes: adjusting or managing the execution state of the file export task according to the third task information;

[0148] After displaying the fourth task information corresponding to the file export task, the method further includes: adjusting or managing the execution progress of the file export task according to the fourth task information.

[0149] It should be noted that this embodiment describes how the system adjusts or manages the execution of tasks based on the information after displaying the execution status and progress information of the file acquisition task and the file export task. Specifically, the system will adjust or manage the execution status and execution progress of the file acquisition task and the file export task respectively based on the displayed first task information, second task information, third task information, and fourth task information.

[0150] This dynamic adjustment mechanism based on task information and progress information is an important means to improve the intelligence and adaptive capabilities of the system. Through real-time monitoring and feedback adjustment, the system can more efficiently deal with various complex situations and ensure the smooth completion of the task.

[0151] For further example, optionally, after displaying the first task information of the file acquisition task (such as "File acquisition in progress"), the system may determine whether the task is progressing normally based on this information. If the system detects that the task is stagnant or abnormal, it may automatically restart the task or make other adjustments to ensure that the file acquisition can be completed smoothly.

[0152] When displaying the second task information (i.e., progress information) of the file acquisition task, the system may dynamically adjust resource allocation according to the speed of the progress. For example, if the progress is slow, the system may increase resource investment to speed up the task completion.

[0153] For the third task information of the file export task (such as "File export in progress"), the system can monitor whether the export process is smooth. If a problem is detected during the export process, the system can take corresponding measures, such as retrying the export or notifying the administrator.

[0154] After displaying the fourth task information (progress information) of the file export task, the system can adjust the export strategy according to the actual progress. For example, if the export progress is far below expectations, the system can optimize the export algorithm or increase processing power to improve efficiency.

[0155] By adjusting or managing the execution status and progress of file acquisition tasks and file export tasks according to the displayed task information and progress information, the system can achieve more flexible and efficient task processing. This dynamic adjustment mechanism not only improves the system's response speed and task completion efficiency, but also enhances the system's stability and reliability.

[0156] As an optional solution, after obtaining the file export request, the method further includes:

[0157] S7-1, perform permission check on the file export request;

[0158] S7-2, in case of permission verification failure, setting the file export request to a prohibited response;

[0159] S7-3, when the permission check succeeds, the file export request is set to allow response.

[0160] In an optional embodiment, permission verification may refer to a process in which the system performs identity verification and permission judgment on the user who initiates the request.

[0161] In an optional embodiment, prohibiting a response can be understood as, when the permission check fails, the system will not perform the file export operation, that is, will not respond to the request.

[0162] In an optional embodiment, the permission response can be understood as that, if the permission verification is successful, the system will continue to process the file export request and execute subsequent file acquisition and file export tasks.

[0163] It should be noted that this embodiment describes a process in which the system first performs a permission check after receiving a file export request. The purpose of this check is to confirm whether the user who initiated the request has the authority to perform the file export operation. Based on the result of the check, the system will decide whether to respond to the request. If the permission check fails, the system will prohibit responding to the request; if the check succeeds, the response is allowed.

[0164] Permission verification is an important part of information system security, which ensures that only authorized users can access or operate specific resources. In sensitive operations such as file export, permission verification can effectively prevent unauthorized data leakage or abuse.

[0165] As an example, let's assume that a company's file management system receives a file export request initiated by an employee. The system will first perform a permission check on the request to check whether the employee has permission to export the file. If the employee is not authorized to perform the file export operation (for example, he / she is not the administrator of the file or has not obtained the relevant permission), then the permission check will fail, and the system will set the file export request to a prohibited response, that is, the export operation will not be performed. On the contrary, if the employee has the corresponding permission and the permission check succeeds, the system will set the request to an allowed response and continue to execute the subsequent file export process.

[0166] By performing permission verification on file export requests and setting the response status of the request (prohibited or allowed) based on the verification results, the system can ensure that only users with corresponding permissions can successfully export files. This mechanism enhances the security of the system, prevents unauthorized file access and operation, and protects the security and integrity of data.

[0167] As an optional solution, after obtaining the file export request, the method further includes:

[0168] S8-1, when the request identifier corresponding to the file export request is in the whitelist, setting the file export request to an allow response;

[0169] S8-2, when the request identifier corresponding to the file export request is in the blacklist, setting the file export request to a prohibited response;

[0170] In the case where the permission check fails, in the process of setting the file export request as a prohibited response, the method further includes: placing a request identifier corresponding to the file export request in a blacklist.

[0171] In an optional embodiment, the whitelist can be understood as a list containing request identifiers that are explicitly allowed by the system to perform specific operations (file export in this case).

[0172] In an optional embodiment, the blacklist may be understood as a list containing request identifiers for which specific operations are explicitly prohibited by the system.

[0173] In an optional embodiment, the request identifier may be an identifier used to uniquely identify a file export request, which may be a user ID, an IP address, a session token, or the like.

[0174] It should be noted that this embodiment further refines the permission control mechanism of the system when processing file export requests. The system decides whether to allow the response to the file export request by checking whether the request identifier (which can be a user ID, session ID or other unique identifier) ​​is in the whitelist or blacklist. The whitelist contains request identifiers that are explicitly allowed to perform file export operations, while the blacklist contains request identifiers that are prohibited from performing such operations. If the request identifier is in the blacklist or if the permission check fails, the system will prohibit the response to the request and add the request identifier to the blacklist to prevent future illegal access.

[0175] As a further example, optionally assume that the backend management system of an online education platform receives a request from a teacher to export a file. The system first checks whether the request identifier of the request (such as the ID of the teacher's account) is in the whitelist. If so, the system allows the request and continues to process the file export. If the request identifier is in the blacklist, for example, because the account has previously attempted illegal access, the system will reject the request. In addition, if the teacher account fails the regular permission check, the system will not only reject the request, but also add its request identifier to the blacklist to prevent future attempts at illegal access.

[0176] By using whitelists and blacklists to control responses to file export requests, the system is able to efficiently handle large volumes of requests while ensuring that only trusted entities can access sensitive resources. In the event of a permission check failure, adding the request identifier to the blacklist further enhances the security of the system and prevents potential future threats.

[0177] As an optional solution, in the process of responding to the file export request, executing the file acquisition task and executing the file export task, the method further includes:

[0178] S9-1, when other file migration requests other than the file export request are obtained, obtain the first response priority corresponding to the other file migration request;

[0179] S9-2, when the first response priority is lower than the second response priority corresponding to the file export request, setting other file migration requests to be responded to after the file export request response;

[0180] S9-3, when the first response priority is higher than the second response priority corresponding to the file export request, other file migration requests are responded to first.

[0181] In an optional embodiment, the first response priority may refer to the processing priority corresponding to other file migration requests except the file export request.

[0182] In an optional embodiment, the second response priority may refer to the processing priority corresponding to the file export request currently being processed.

[0183] It should be noted that the present embodiment describes the logic of how to arrange task responses and execution order according to task priorities in a multi-task processing environment. Specifically, while responding to file export requests, executing file acquisition and executing file export tasks, if the system also receives other types of file migration requests, the system will evaluate the priorities of these requests and decide whether to process the current file export request first or interrupt the current processing to respond to the high-priority file migration request first based on the priorities.

[0184] The priority of a task can be set based on a variety of factors, including but not limited to the importance of the task, urgency, user authority level, system resource status, etc. By dynamically adjusting the response priority of tasks, the system can manage concurrent tasks more flexibly and efficiently, ensuring that critical tasks are responded to and processed in a timely manner.

[0185] To further illustrate, for example, a system is processing a regular file export request (second response priority), and at this time, the system receives an urgent file migration request to restore important data that can be lost due to a failure (first response priority is very high). According to the logic described in this paragraph, the system will determine that the priority of this urgent file migration request is higher than the current file export request, so the system will suspend or postpone the processing of the file export task and instead give priority to executing this urgent file migration request.

[0186] By comparing and evaluating the response priorities of different tasks and dynamically adjusting the processing order of tasks according to the priorities, the system can achieve more intelligent and efficient task management in a multi-task environment. This strategy not only improves the system's ability to respond to urgent or important tasks, but also ensures the rational allocation and utilization of system resources, thereby improving the overall performance and reliability of the system.

[0187] As an optional solution, after responding to the file export request, executing the file acquisition task and executing the file export task, the method further includes:

[0188] When task completion information is obtained, the request information corresponding to the file export request is deleted, wherein the task completion information is used to indicate that the file acquisition task and the file export task are completed.

[0189] In an optional embodiment, the task completion information may be a signal or a message for explicitly informing the system that a specific file acquisition task and a file export task have been completed.

[0190] In an optional embodiment, the request information may be data and metadata related to a specific file export request, which may include the source, time, content, etc. of the request. Such information is necessary during the request processing, but once the request processing is completed, it is no longer needed.

[0191] It should be noted that the description of this embodiment refers to that after the file export request is responded to and the file acquisition task and the file export task are completed, the system will perform a cleanup step. Specifically, when the system obtains task completion information indicating that the file acquisition task and the file export task have been completed, it will delete the request information related to the file export request. This process is to free up system resources and keep the system clean and efficient.

[0192] By promptly cleaning up data that is no longer needed, the system can maintain its efficient operation and avoid unnecessary resource occupation. In addition, from a security and privacy perspective, deleting old request information also helps reduce potential data leakage risks.

[0193] To further illustrate, optionally assume that a user submits a file export request through the system. The system receives and processes the request, completing the acquisition and export of the file. Once both tasks are completed, the system will receive a task completion message. At this point, the system knows that the file export request has been fully processed, so it deletes all request information related to the request to free up storage space and processing resources.

[0194] By deleting the relevant request information after the task is completed, the system can effectively manage its storage and processing resources. This strategy not only helps improve the overall performance and responsiveness of the system, but also enhances the system's security and privacy protection capabilities.

[0195] As an optional solution, after responding to the file export request, executing the file acquisition task and executing the file export task, the method further includes:

[0196] When task completion information is obtained, temporary data in the process of executing the file acquisition task and the file export task is deleted, wherein the task completion information is used to indicate that the file acquisition task and the file export task are completed.

[0197] In an optional embodiment, the task completion information may refer to a signal or notification received by the system, indicating that a specific file acquisition and file export task has been completed.

[0198] In an optional embodiment, temporary data may refer to some temporary data files, caches or logs that may be generated by the system when executing file acquisition and file export tasks. These data are usually no longer needed after the task is completed, so they are called temporary data.

[0199] It should be noted that this embodiment describes a system operation after completing a file export request, executing a file acquisition task and a file export task. Specifically, when the system obtains task completion information indicating that the file acquisition and file export tasks have been completed, it deletes temporary data generated during the execution of these tasks.

[0200] Deleting temporary data not only helps save storage space, but also improves system security and privacy protection. In some cases, temporary data may contain sensitive information, which may be accessed by unauthorized personnel if not deleted in time. In addition, regular cleaning of temporary data is also part of system maintenance to ensure system stability and performance.

[0201] To further illustrate, optionally assume that a user requests to export a large amount of data. When processing this request, the system may create some temporary data tables, indexes, or log files to assist in processing. Once the data is successfully exported and the user receives the file, the system will receive a task completion signal. At this time, in order to save storage space and keep the system clean, the system will delete the temporary data created during the export process.

[0202] By deleting temporary data generated during the execution of file acquisition and file export tasks, the system can manage its storage space more effectively and reduce unnecessary resource waste. At the same time, this operation also enhances data security and reduces the risk of sensitive information leakage.

[0203] As an optional solution, for ease of understanding, the above-mentioned cloud file export method is applied to the distributed cloud resource file export scenario, aiming to improve the efficiency and reliability of file export, so as to meet the file export requirements in various cloud computing environments. Specifically, this embodiment provides a distributed cloud resource file export solution based on dual asynchrony, which realizes parallel execution and asynchronous processing by decomposing the file export task into two asynchronous processes, namely resource data acquisition and export processing, thereby reducing the time consumption and resource occupation of the export process. At the same time, by introducing a fault handling mechanism and a status monitoring function, the reliability and stability of the export task are improved, and the integrity and consistency of the data are guaranteed.

[0204] It should be noted that this embodiment proposes an innovative dual-asynchronous distributed cloud resource file export implementation method. Its core feature is that through the dual-asynchronous mechanism, it realizes the effective shielding of abnormal requests, the balanced load of exported resources, and the efficient file export capability. The following are the detailed technical implementation steps:

[0205] First, the system receives and verifies the user request. This step is completed by calling the cloud service API or other data acquisition interface to ensure fast data extraction and stable transmission. This method not only improves the efficiency of data processing, but also ensures the security of data transmission.

[0206] Secondly, once the resource data is acquired, the system starts the export processing program. The program will perform in-depth processing and conversion on the acquired data, such as compression, encryption or format conversion, to meet the diverse needs of users. The flexibility of this step enables this embodiment to be widely adapted to different application scenarios.

[0207] Next is the parallel execution phase of export. In this phase, the acquisition of resource data and the export processing are carried out in parallel, which realizes the efficient use of system resources, thereby improving the concurrent processing capability and overall efficiency of the export task. In particular, the microservice architecture provides strong support for this embodiment. When the system receives an export request, the back-end service will start an asynchronous processing thread, which is the first layer of asynchronous processing logic of this embodiment. If a large number of export file requests are encountered, each request will trigger a new thread for parallel processing. The second layer of asynchronous processing logic is more complicated: multiple copies of the back-end will perform a master selection operation, and the master service will be determined by trying to obtain the leader lock of Redis. The selected master service stores its own IP and port information in Redis, and is responsible for querying the total amount of data to be exported, performing data sharding, and calling other slave services for collaborative processing. When all services complete the data export, the master service will be responsible for merging all exported files. This process is implemented using the apache.poi toolkit, and various operations of the table can be completed without relying on third-party Excel software.

[0208] In addition, this embodiment also introduces a complete fault handling mechanism. This mechanism can capture and handle abnormal situations that occur during the export process, such as automatically performing error retry and data rollback operations, thereby ensuring the reliability and stability of the export task.

[0209] Finally, this embodiment provides a real-time task execution status and progress monitoring function. Users can query the execution status of tasks at any time through a friendly interface and adjust and manage them according to actual needs. This function greatly improves the user experience, allowing users to understand the task progress more intuitively and make corresponding adjustments in a timely manner.

[0210] Further examples are given to illustrate that, in order to realize the purpose of quickly exporting large files in a distributed system, this embodiment aims to provide a cloud resource file export solution based on dual asynchronous operation. The flowchart is as follows: Figure 3 The figure shows the whole process from export request to Excel document merging and finally storing in Redis database, as follows:

[0211] The client first initiates an export request, which is initially processed by the gateway and then received by the backend main service to generate a preliminary Excel document, which is then passed to the backend slave service for further processing. The slave service makes necessary modifications to the document and returns it to the main service, which generates the final version and stores it in the Redis database. Next, the system retrieves relevant documents from Redis and merges them to generate a complete Excel document, which is then sent to the database for persistent storage. Throughout the process, the dual asynchronous processing mechanism is used to achieve efficient execution of export tasks and improve system throughput. At the same time, this embodiment also introduces a fault handling mechanism to ensure the stability and reliability of the export process. In addition, this embodiment supports multiple export methods and processing algorithms to meet the personalized needs of users, and realizes flexible expansion and maintenance of backend services based on the microservice architecture.

[0212] In an optional embodiment, for efficient acquisition of resource data, at the beginning of the export task, the system uses an asynchronous method to quickly acquire resource data of the file to be exported, covering key information such as file content and metadata. This process uses cloud service API interfaces or other advanced data acquisition technologies to ensure rapid data extraction and transmission, laying a solid foundation for subsequent export processing.

[0213] For example, when the cloud platform receives an export request from a user, it will first go through the gateway's strict permission verification mechanism. This step is crucial because it can effectively prevent unauthorized users from accessing and exporting confidential data, thereby ensuring data security. If the user fails to pass the permission verification, the request will be immediately intercepted and processed, and the system will return a clear prompt message to the front end to inform the user that he or she does not have permission to perform the export operation. This design not only complies with data security best practices, but also avoids the risk of data leakage caused by misoperation or malicious attempts.

[0214] On the contrary, if the user successfully passes the authorization authentication, the system will continue to execute the subsequent export process. The design of this process fully considers the importance of data protection and the compliance of operations, ensuring that only authorized users can access and export sensitive data. In this way, the solution of this embodiment not only ensures data security, but also provides an efficient and controllable data export service.

[0215] In an optional embodiment, for a flexible export processing mechanism, once the resource data is successfully acquired, the system immediately starts the export processing program, performs fine processing and conversion on the acquired data, and finally exports it to a predetermined location. This link can flexibly adopt a variety of export methods and processing algorithms according to the specific needs of the user, such as data compression, encryption protection and format conversion, so as to fully meet the personalized needs of the user.

[0216] For example, the cloud platform adopts a distributed microservice architecture, and each service is equipped with several replicas to achieve load balancing and ensure high availability and scalability of the system. When an export request arrives at the backend, the request is assigned to a backend service module for processing through a random algorithm.

[0217] Before processing a request, the backend service will try to actively acquire the Redis leader lock. The first service that successfully acquires the lock will be determined as the primary service, which is intended to prevent the same request from being triggered multiple times, thereby avoiding unnecessary waste of resources. The primary service will store its own service IP, request key, and port information in the Redis key so that other services can identify and work together. This design not only improves the processing efficiency of the system, but also effectively prevents the repeated processing of the same request.

[0218] If the export user's request is already in progress, the system will directly reject subsequent identical export requests. This strategy helps prevent the risk of system downtime caused by request attacks and ensures the stability and security of the system.

[0219] In addition, during the export process, the system will also perform corresponding processing according to the export strategy selected by the user. For example, if the user selects the file compression option, the system will compress the document after completing the data export. This not only helps save storage space, but also improves the efficiency of data transmission, providing users with a more convenient and efficient service experience.

[0220] In an optional embodiment, for parallel processing technology, resource data acquisition and export processing adopt a parallel execution strategy to achieve efficient connection of processes without interfering with each other. This design not only maximizes the use of system resources, but also significantly improves the concurrent processing capability and overall efficiency of export tasks.

[0221] In an optional embodiment, for the distributed asynchronous processing core, relying on the distributed microservice architecture, there are multiple copies of the backend service to ensure load balancing and high reliability. When an export request is received, the backend service will start the asynchronous processing thread to implement the first layer of asynchronous processing. If there are a large number of export requests, the system can assign independent threads to each request for parallel processing. Furthermore, the backend copy determines the main service through an intelligent master selection algorithm, which is responsible for querying the total amount of data, sharding calculations and task allocation. After completing the data export, the main service is also responsible for the merging and integration of files, and finally releases related resources to ensure the smoothness and efficiency of the entire processing flow.

[0222] For example, during asynchronous processing, when the main service starts an export operation, it will first initialize a thread as the main thread for this export, which marks the beginning of the first layer of asynchronous execution. This design is intended to improve the concurrent processing capabilities of the system so that the export operation will not block the execution of other tasks.

[0223] Next, the master service queries the database to obtain the total amount of exported resources and performs sharding processing. The sharding algorithm evenly distributes the data according to the number of replicas available in the system and determines the data range that each replica needs to process by calculating the offset of each replica. This sharding mechanism effectively disperses the pressure of data processing and improves the overall processing efficiency.

[0224] The interaction between the master service and the slave service is carried out through the HTTP protocol. Each service is assigned an internal IP address and port number when it is started, and this information is registered with the registry service. As the core component of the microservice architecture system, the registry saves the address information of all started services. By calling the interface of the registry, the master service can accurately obtain the total number of replicas in the current system.

[0225] For example, if there are three replicas in the system and the total amount of data to be exported is 30,000, the master service will be responsible for exporting the 1st to 10,000th data, the second slave service will be responsible for exporting the 10,001st to 20,000th data, and the third service will be responsible for exporting the 20,001st to 30,000th data. This clear division of labor ensures the orderliness and efficiency of data export.

[0226] In each replica, multiple threads can be started according to the configuration to execute data export tasks in batches. For example, when N=5 is configured, each replica will start 5 threads, each thread is responsible for encapsulating and writing 2000 data into the Excel file. This multi-threaded processing method further improves the speed of data export.

[0227] When all the master thread pools of all replicas have finished executing, the non-master replica will proactively notify the master replica that the file export has been saved and return the save path. The master replica will then reassemble all documents into a complete large document. Since the documents only need to be appended at this time, the processing speed is relatively fast. Finally, the system will delete all temporary files to save storage space and prevent resource waste.

[0228] In an optional embodiment, for a sound fault handling mechanism, in order to ensure the stability of the export process, this embodiment introduces a comprehensive fault handling mechanism. The system can intelligently capture and handle various abnormal situations that occur during the export process, and automatically perform operations such as error retry and data rollback, thereby effectively ensuring the reliability and stability of the export task.

[0229] For example, if an unknown exception occurs during execution, the fault handling mechanism can cover the following aspects:

[0230] First, exception capture. The system has the ability to fully capture all kinds of exceptions during the export process, which may involve file operations, network transmission, and database connections. The root causes of the exceptions may be various, such as missing files, lack of disk space, or network interruptions. Once these exceptions are captured, the system needs to accurately classify them and record detailed error information in the log for subsequent analysis and processing.

[0231] Secondly, for some exceptions that are recoverable, such as temporary network interruptions, the master service can automatically retry based on feedback from the slave service. This design aims to restore the normal operation process by re-executing the failed export task. The frequency and interval of retries should be flexibly configured according to the actual situation to balance efficiency and stability.

[0232] Furthermore, if the export task encounters an unrecoverable exception during execution, such as data corruption, the master service has the ability to re-initiate the request and backtrack to the previous security steps to ensure the integrity and accuracy of the data.

[0233] In addition, the user notification mechanism is also crucial. When an export task fails or an exception occurs, the system can quickly notify the user so that they can understand the task status in a timely manner and make corresponding adjustments. It also supports real-time notification through user operation logs to ensure transparent and timely information.

[0234] Finally, troubleshooting and repair are an indispensable part. Developers should be able to use the exception information recorded in the log to quickly locate and repair the fault that caused the export failure. Once the fault is handled, the developer can manually trigger the export request again to restore the normal service process.

[0235] In an optional embodiment, for real-time status monitoring and feedback, this embodiment provides real-time task execution status and progress monitoring functions, equipped with a user-friendly interface and instant feedback information. Users can query task execution status anytime and anywhere, and flexibly adjust and manage according to actual needs, greatly improving user experience and work efficiency.

[0236] For example, when the main thread is executing an export task, the system will track the progress of the program in real time and dynamically update the progress information to the Redis database. Redis is chosen as the storage medium because of its efficient data reading and writing capabilities and good support for asynchronous requests. Since export requests are all processed asynchronously, the front end needs to refresh regularly and call the back end interface to query the execution progress of the current task.

[0237] By storing progress information in Redis, the backend can quickly respond to the frontend's query requests and provide real-time progress feedback. This design not only improves the system's response speed, but also ensures the accuracy and consistency of progress information. In addition, the system uses prefabricated progress to update and display progress. In the key steps of the export task, the system will set the corresponding progress value so that the frontend can clearly display it to the user.

[0238] In an optional embodiment, when all documents have been processed, the primary copy may delete the request information stored in Redis to ensure that subsequent similar requests can be responded to normally.

[0239] In an optional embodiment, the asynchronous thread pool tool class can be implemented through Java and annotations. For example, in Java, the @Async annotation can be used to clearly identify a method or a method in a class as being executed asynchronously. Specifically, when the @Async annotation is added to a method, it indicates that the method will be executed asynchronously and will not block the main thread. Similarly, if the @Async annotation is added at the class level, it means that all methods in the class will run asynchronously. This annotation method provides a concise and efficient mechanism for implementing concurrent execution of methods.

[0240] For the @Async annotation to take effect, certain conditions must be met. First, the class using this annotation must be a class managed by the Spring framework, so that Spring can correctly handle the call and execution of asynchronous methods. Second, in the startup class or configuration class of the Spring application, you need to add the @Enable Async annotation to activate asynchronous support. This step is the key to ensure that the @Async annotation can work properly.

[0241] In addition, the @Async annotation provides flexibility in the choice of thread pool. If the name of the thread pool is not explicitly specified in the annotation, that is, there is no custom thread pool, then @Async will use the Simple AsyncTask Executor provided by Spring as the thread pool by default. This default thread pool can meet basic asynchronous execution requirements, but customizing the thread pool according to specific scenarios may bring better performance and resource utilization.

[0242] Through the embodiments provided in this application, a distributed cloud resource file export method based on a dual asynchronous mechanism is proposed, which is especially optimized for the export requirements of massive data (up to 100,000 levels), achieving a significant improvement in export effect. After rigorous testing, compared with the traditional synchronous file export method, the solution of this embodiment greatly reduces the export time from the original 20 minutes to only about 20 seconds when processing 100,000 data. This significant efficiency improvement undoubtedly brings a smoother experience to users. Specifically:

[0243] First, through the dual asynchronous mechanism, this embodiment significantly improves the efficiency and reliability of file export, which not only greatly reduces the waiting time of users, but also reduces the occupancy rate of system resources, so that the overall performance is significantly improved.

[0244] Secondly, this embodiment successfully realizes parallel processing and asynchronous transmission of data, which effectively reduces the response time of the system and reduces the load pressure, so that the system can still maintain efficient and stable operation when facing large-scale data processing.

[0245] Furthermore, this embodiment introduces an advanced fault handling mechanism and a comprehensive status monitoring function, which not only enhances the stability of the system and ensures rapid troubleshooting and recovery when problems occur, but also greatly improves the maintainability of the system and provides administrators with a more convenient management method.

[0246] Finally, this embodiment has wide applicability. No matter what type of cloud computing environment or scenario, it can show excellent performance and stability, fully demonstrating its high versatility and flexibility.

[0247] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0248] In this embodiment, a cloud file export device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0249] Figure 4 is a structural block diagram of a cloud file export device according to an embodiment of the present application, such as Figure 4 As shown, the device comprises:

[0250] A first acquisition unit 402 is used to acquire a file export request, wherein the file export request is used to request to export a cloud file to a target location;

[0251] The first execution unit 404 is used to respond to the file export request, execute the file acquisition task and the file export task, wherein the file acquisition task is a task of acquiring resource data of the cloud file, and the file export task is a task of exporting the resource data of the cloud file to the target location, and the file acquisition task and the file export task are in an asynchronous execution relationship.

[0252] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0253] As an optional solution, the first execution unit 404 includes:

[0254] The first execution module is used to utilize the distributed service architecture to respond to file export requests and execute file acquisition tasks and file export tasks, wherein the distributed service architecture is configured with multiple copies, and the copies are used to respond to file migration requests, and the file migration requests include file export requests.

[0255] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0256] As an optional solution, the first execution module includes:

[0257] An acquisition submodule is used to acquire a first thread of a primary service copy, wherein the primary service copy is a copy used to respond to a file export request among multiple copies, the multiple copies include the primary service copy, the first thread is a thread configured by the primary service copy and in an idle state, the primary service copy is configured with at least two service threads, the at least two service threads are used to respond to concurrent file migration requests, the idle state is a thread state to be responded to a file migration request, the service thread is the first thread or the second thread, the second thread is a thread in a working state, and the working state is a thread state that is responding to a file migration request;

[0258] The execution submodule is used to utilize the first thread to respond to the file export request and execute the file acquisition task and the file export task.

[0259] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0260] As an optional solution, get submodules, including:

[0261] The first determination sub-unit is used to determine a primary service copy from multiple copies as a copy among the multiple copies for responding to file export requests, wherein the primary service copy is the first copy among the multiple copies to obtain a dominant service lock, and each copy among the multiple copies is set to attempt to obtain the dominant service lock after receiving a file migration request.

[0262] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0263] As an optional solution, the device also includes:

[0264] A query subunit, for querying the data volume corresponding to the resource data of the cloud file after determining the primary service copy from the multiple copies as the copy for responding to the file export request among the multiple copies;

[0265] A second determination subunit is used to determine a primary service copy from the multiple copies as a copy for responding to the file export request among the multiple copies, and then, according to the data volume, divide the resource data of the cloud file into data slices to obtain multiple resource sub-data;

[0266] A calling sub-unit is used to call at least one copy other than the main service copy from the multiple copies after determining the main service copy from the multiple copies as the copy for responding to the file export request from the multiple copies, and indicate the offset corresponding to the resource sub-data in the multiple resource sub-data that each sub-copy in the at least one copy needs to process, wherein the offset is used to indicate each sub-copy to export the resource sub-data corresponding to each sub-copy;

[0267] The merging sub-unit is used to determine the main service copy from multiple copies as the copy for responding to the file export request, and then merge the resource sub-data exported by each sub-copy through the main service copy to obtain complete resource data, wherein the resource data of the cloud file includes the complete resource data.

[0268] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0269] As an optional solution, the second determining subunit includes:

[0270] An acquisition component, used to acquire the number of replicas currently allowed to be called among the multiple replicas, wherein the number of replicas is the number of replicas in at least one replica;

[0271] The combination component is used to combine the number of copies and the amount of data to shard the resource data of the cloud file to obtain multiple resource sub-data.

[0272] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0273] As an optional solution, the device also includes:

[0274] An acquisition sub-unit is used to acquire the resource sub-data corresponding to the exports of each sub-copy and transmitted to the main service copy before merging the resource sub-data corresponding to the exports of each sub-copy through the main service copy to obtain the complete resource data, wherein the resource transmission address of the main service copy is pre-stored in the service database, and the service database is used to provide the corresponding resource transmission address for each sub-copy.

[0275] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0276] As an optional solution, the device also includes:

[0277] The second execution unit is used to execute an exception handling task when export exception information is obtained during the process of responding to a file export request, executing a file acquisition task and executing a file export task, wherein the export exception information is used to indicate that an exception exists during the execution of the file acquisition task or the file export task, and the exception handling task is used to handle the exception.

[0278] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0279] As an optional solution, the second execution unit includes at least one of the following:

[0280] A second execution module, used for re-executing the task in an abnormal state or the file export task;

[0281] The third execution module is used to roll back the corresponding data of the task or file export task in the abnormal state to the time before the abnormal situation occurs.

[0282] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0283] As an optional solution, the device also includes:

[0284] A second acquisition unit is used to acquire the export exception information before executing the exception handling task;

[0285] A classification unit, used for classifying the derived exception information before executing the exception handling task to obtain the classified derived exception information, wherein different types of abnormal situations correspond to different types of derived exception information;

[0286] The second execution unit includes: a fourth execution module, which is used to execute an exception handling task according to the classified export exception information, wherein different types of export exception information correspond to different types of exception handling tasks.

[0287] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0288] As an optional solution, the device further includes at least one of the following:

[0289] A first display unit, used to display first task information corresponding to the file acquisition task in the process of responding to the file export request, executing the file acquisition task and executing the file export task, wherein the first task information is used to indicate the execution status of the file acquisition task;

[0290] A second display unit, used to display second task information corresponding to the file acquisition task in the process of responding to the file export request, executing the file acquisition task and executing the file export task, wherein the second task information is used to indicate the execution progress of the file acquisition task;

[0291] A third display unit, used to display third task information corresponding to the file export task in the process of responding to the file export request, executing the file acquisition task and executing the file export task, wherein the third task information is used to indicate the execution status of the file export task;

[0292] The fourth display unit is used to display fourth task information corresponding to the file export task in the process of responding to the file export request, executing the file acquisition task and executing the file export task, wherein the fourth task information is used to indicate the execution progress of the file export task.

[0293] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0294] As an optional solution, the device further includes: a first adjustment unit, configured to adjust or manage the execution state of the file acquisition task according to the first task information after displaying the first task information corresponding to the file acquisition task;

[0295] The device further includes: a second adjustment unit, configured to adjust or manage the execution progress of the file acquisition task according to the second task information after displaying the second task information corresponding to the file acquisition task;

[0296] The device further includes: a third adjustment unit, configured to adjust or manage the execution state of the file export task according to the third task information after displaying the third task information corresponding to the file export task;

[0297] The device also includes: a fourth adjustment unit, which is used to adjust or manage the execution progress of the file export task according to the fourth task information after displaying the fourth task information corresponding to the file export task.

[0298] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0299] As an optional solution, the device also includes:

[0300] A verification unit, used for performing permission verification on the file export request after obtaining the file export request;

[0301] A first setting unit is used to set the file export request as a prohibit response in the case where the permission check fails after obtaining the file export request;

[0302] The second setting unit is used to set the file export request as an allow response after obtaining the file export request and if the permission verification succeeds.

[0303] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0304] As an optional solution, the device also includes:

[0305] A third setting unit is used to, after obtaining the file export request, set the file export request to allow the response if the request identifier corresponding to the file export request is in the whitelist;

[0306] A fourth setting unit is used to, after obtaining the file export request, set the file export request as a prohibit response if the request identifier corresponding to the file export request is in the blacklist;

[0307] The device also includes: a fifth setting unit, which is used to place the request identifier corresponding to the file export request in a blacklist during the process of setting the file export request as a prohibited response when the permission check fails.

[0308] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0309] As an optional solution, the device also includes:

[0310] A third acquisition unit is used to, in the process of responding to the file export request, executing the file acquisition task and executing the file export task, in the case where other file migration requests other than the file export request are acquired, acquire the first response priority corresponding to the other file migration request;

[0311] A sixth setting unit, configured to, in the process of responding to a file export request, executing a file acquisition task and executing a file export task, set other file migration requests to be responded to after the file export request response if the first response priority is lower than the second response priority corresponding to the file export request;

[0312] The seventh setting unit is used to give priority to responding to other file migration requests when the first response priority is higher than the second response priority corresponding to the file export request in the process of responding to the file export request, executing the file acquisition task and executing the file export task.

[0313] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0314] As an optional solution, the device also includes:

[0315] The first deletion unit is used to delete the request information corresponding to the file export request after responding to the file export request, executing the file acquisition task and executing the file export task, and when the task completion information is obtained, wherein the task completion information is used to indicate that the file acquisition task and the file export task are completed.

[0316] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0317] As an optional solution, the device also includes:

[0318] The second deleting unit is used to delete temporary data in the process of executing the file acquisition task and the file export task after responding to the file export request and executing the file acquisition task and executing the file export task, when the task completion information is obtained, wherein the task completion information is used to indicate that the file acquisition task and the file export task are completed.

[0319] For a specific implementation example, please refer to the example shown in the above-mentioned method for exporting cloud files, which will not be described in detail in this example.

[0320] It should be noted that the above-mentioned virtual devices (modules, units, sub-modules, sub-units, components, etc.) can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to this: the above-mentioned virtual devices are all located in the same processor; or, the above-mentioned virtual devices are located in different processors in any combination.

[0321] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0322] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0323] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0324] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0325] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0326] Obviously, those skilled in the art should understand that the above-mentioned virtual devices or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0327] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for exporting cloud files, characterized in that: include: Obtain a file export request, wherein the file export request is used to request to export a cloud file to a target location; In response to the file export request, executing a file acquisition task and a file export task, wherein the file acquisition task is a task of acquiring resource data of a cloud file, and the file export task is a task of exporting the resource data of the cloud file to the target location; The responding to the file export request, executing the file acquisition task, and executing the file export task include: Using a distributed service architecture, responding to the file export request, executing the file acquisition task and the file export task, wherein the distributed service architecture is configured with multiple copies, and the copies are used to respond to the file migration request, and the file migration request includes the file export request; The utilizing of the distributed service architecture to respond to the file export request and execute the file acquisition task and the file export task includes: Obtaining a first thread of a primary service copy, wherein the primary service copy is a copy used to respond to the file export request among the multiple copies, the multiple services include the primary service copy, the first thread is a thread configured by the primary service copy and is in an idle state, the primary service copy is configured with at least two service threads, the at least two service threads are used to respond to concurrent file migration requests, the idle state is a thread state to be responded to the file migration request, the service thread is the first thread or the second thread, the second thread is a thread in a working state, and the working state is a thread state that is responding to the file migration request; The first thread of obtaining the primary service copy includes: A primary service replica, as a replica among the multiple replicas for responding to the file export request, wherein the primary service replica is the first replica among the multiple replicas to obtain a dominant service lock, and each replica among the multiple replicas is configured to attempt to obtain the dominant service lock after receiving the file migration request; After determining a primary service copy from the multiple copies as a copy among the multiple copies for responding to the file export request, the method further includes: Query the data volume corresponding to the resource data of the cloud file; According to the data volume, the resource data of the cloud file is segmented to obtain a plurality of resource sub-data; Calling at least one replica other than the primary service replica from the multiple replicas, and indicating the offset corresponding to the resource sub-data in the multiple resource sub-data that each sub-replica in the at least one replica needs to process, wherein the offset is used to indicate that each sub-replica exports the resource sub-data corresponding to each sub-replica; The resource sub-data corresponding to the exports of each of the sub-copies are merged through the master service copy to obtain complete resource data, wherein the resource data of the cloud file includes the complete resource data.

2. The method according to claim 1, characterized in that The utilizing of the distributed service architecture to respond to the file export request and execute the file acquisition task and the file export task includes: The first thread is utilized to respond to the file export request and execute the file acquisition task and the file export task.

3. The method according to claim 1, characterized in that According to the data volume, the resource data of the cloud file is segmented to obtain a plurality of resource sub-data, including: Obtaining the number of replicas currently allowed to be called among the multiple replicas, wherein the number of replicas is the number of replicas in the at least one replica; In combination with the number of copies and the amount of data, the resource data of the cloud file is segmented to obtain the multiple resource sub-data.

4. The method according to claim 1, characterized in that: Before merging the resource sub-data corresponding to the derived sub-replicas through the master service replica to obtain complete resource data, the method further includes: The resource sub-data corresponding to the export are obtained and transmitted to the master service copy, wherein the resource transmission address of the master service copy is pre-stored in a service database, and the service database is used to provide corresponding resource transmission addresses for each sub-copy.

5. The method according to claim 1, characterized in that In the process of responding to the file export request, executing the file acquisition task, and executing the file export task, the method further includes: When export exception information is obtained, an exception handling task is executed, wherein the export exception information is used to indicate that an abnormal situation exists during the execution of the file acquisition task or the file export task, and the exception handling task is used to handle the abnormal situation.

6. The method according to claim 5, characterized in that The performing of the exception handling task includes at least one of the following: Re-execute the task in the abnormal state or the file export task; Roll back the data corresponding to the task in the abnormal state or the file export task to the time before the abnormal situation occurs.

7. The method according to claim 5, characterized in that Before executing the exception handling task, the method further includes: Obtaining the export exception information; Classifying the derived exception information to obtain classified derived exception information, wherein different types of the abnormal situations correspond to different types of derived exception information; The executing the exception handling task includes: executing the exception handling task according to the classified derived exception information, wherein different types of the derived exception information correspond to different types of the exception handling tasks.

8. The method according to claim 1, characterized in that In the process of responding to the file export request, executing the file acquisition task, and executing the file export task, the method further includes at least one of the following: Displaying first task information corresponding to the file acquisition task, wherein the first task information is used to indicate the execution status of the file acquisition task; Displaying second task information corresponding to the file acquisition task, wherein the second task information is used to indicate the execution progress of the file acquisition task; Displaying third task information corresponding to the file export task, wherein the third task information is used to indicate the execution status of the file export task; Fourth task information corresponding to the file export task is displayed, wherein the fourth task information is used to indicate the execution progress of the file export task.

9. The method according to claim 8, characterized in that After displaying the first task information corresponding to the file acquisition task, the method further includes: adjusting or managing the execution state of the file acquisition task according to the first task information; After displaying the second task information corresponding to the file acquisition task, the method further includes: adjusting or managing the execution progress of the file acquisition task according to the second task information; After displaying the third task information corresponding to the file export task, the method further includes: adjusting or managing the execution state of the file export task according to the third task information; After displaying the fourth task information corresponding to the file export task, the method further includes: adjusting or managing the execution progress of the file export task according to the fourth task information.

10. The method according to claim 1, characterized in that After obtaining the file export request, the method further includes: Performing permission verification on the file export request; In the case where the permission check fails, setting the file export request to a prohibit response; In case the permission check is successful, the file export request is set to an allow response.

11. The method according to claim 10, characterized in that After obtaining the file export request, the method further includes: If the request identifier corresponding to the file export request is in the whitelist, setting the file export request to an allow response; When the request identifier corresponding to the file export request is in the blacklist, setting the file export request to a prohibit response; In the case where the permission check fails, in the process of setting the file export request as a prohibit response, the method further includes: placing a request identifier corresponding to the file export request in the blacklist.

12. The method according to any one of claims 1 to 11, characterized in that In the process of responding to the file export request, executing the file acquisition task, and executing the file export task, the method further includes: In the case where other file migration requests other than the file export request are obtained, obtaining a first response priority corresponding to the other file migration request; In a case where the first response priority is lower than a second response priority corresponding to the file export request, setting the other file migration request to be responded to after the file export request response; In the case where the first response priority is higher than the second response priority corresponding to the file export request, the other file migration requests are responded to first.

13. The method according to any one of claims 1 to 11, characterized in that After responding to the file export request, executing the file acquisition task, and executing the file export task, the method further includes: When task completion information is obtained, the request information corresponding to the file export request is deleted, wherein the task completion information is used to indicate that the file acquisition task and the file export task are completed.

14. The method according to any one of claims 1 to 11, characterized in that After responding to the file export request, executing the file acquisition task, and executing the file export task, the method further includes: When task completion information is obtained, temporary data in the process of executing the file acquisition task and the file export task is deleted, wherein the task completion information is used to indicate that the file acquisition task and the file export task are completed.

15. A cloud file export device, characterized in that: include: A first acquisition unit is used to acquire a file export request, wherein the file export request is used to request to export a cloud file to a target location; A first execution unit, configured to respond to the file export request, execute a file acquisition task, and execute a file export task, wherein the file acquisition task is a task of acquiring resource data of a cloud file, and the file export task is a task of exporting the resource data of the cloud file to the target location; A first execution module is used to respond to a file export request using a distributed service architecture, and execute a file acquisition task and a file export task, wherein the distributed service architecture is configured with multiple copies, and the copies are used to respond to a file migration request, and the file migration request includes a file export request; An acquisition submodule is used to acquire a first thread of a primary service copy, wherein the primary service copy is a copy used to respond to a file export request among multiple copies, multiple services include the primary service copy, the first thread is a thread configured by the primary service copy and in an idle state, the primary service copy is configured with at least two service threads, and the at least two service threads are used to respond to concurrent file migration requests, the idle state is a thread state of a file migration request to be responded to, the service thread is the first thread or the second thread, the second thread is a thread in a working state, and the working state is a thread state of a file migration request being responded to; A first determination subunit is used to determine a primary service replica from multiple replicas as a replica among the multiple replicas for responding to the file export request, wherein the primary service replica is the first replica among the multiple replicas to obtain a dominant service lock, and each replica among the multiple replicas is configured to attempt to obtain the dominant service lock after receiving the file migration request; A query subunit, for querying the data volume corresponding to the resource data of the cloud file after determining the primary service copy from the multiple copies as the copy for responding to the file export request among the multiple copies; A second determination subunit is used to determine a primary service copy from the multiple copies as a copy for responding to the file export request among the multiple copies, and then, according to the data volume, divide the resource data of the cloud file into data slices to obtain multiple resource sub-data; A calling sub-unit is used to call at least one copy other than the main service copy from the multiple copies after determining the main service copy from the multiple copies as the copy for responding to the file export request from the multiple copies, and indicate the offset corresponding to the resource sub-data in the multiple resource sub-data that each sub-copy in the at least one copy needs to process, wherein the offset is used to indicate each sub-copy to export the resource sub-data corresponding to each sub-copy; The merging sub-unit is used to determine the main service copy from multiple copies as the copy for responding to the file export request, and then merge the resource sub-data exported by each sub-copy through the main service copy to obtain complete resource data, wherein the resource data of the cloud file includes the complete resource data.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 14 when executed by a processor.

17. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 14 are implemented.

Images