Data asynchronous derivation method and device, equipment and storage medium thereof

Abstract

The embodiment of the application belongs to the technical field of data transmission, is applied to the field of data asynchronous export, and relates to a data asynchronous export method, device and equipment and a storage medium thereof, and comprises the steps that a target data export instruction is received, the export instruction is parsed, a parsing result of the instruction is obtained, wherein the parsing result of the instruction comprises a target interface address to be called; an export task is generated based on the target interface address and preset export parameters; the export task is executed to obtain target data; the target data is filled and written into a preset receiving file to generate an export file, and the export task is terminated. The method can export data with large data volume or complex computing logic, can update and check an export progress in real time, can support breakpoint export when a server is abnormal, can realize an asynchronous export function of mailing a business personnel after export is completed, can greatly improve user experience of a system, and can prevent server downtime or automatic restart due to too great server pressure caused by the export function.

Description

Technical Field

[0001] This application relates to the field of data transmission technology, and in particular to a method, apparatus, device and storage medium for asynchronous data export. Background Technology

[0002] Data export is a fundamental function of a business system and an important way for business personnel to verify and review data. However, when the data volume is large or the data calculation logic is complex, export timeouts often occur. In such cases, only a portion of the data can be exported, or the export may fail without any follow-up prompts due to high server load or a recent restart.

[0003] Therefore, there is an urgent need for an asynchronous data export solution to achieve the goal of successfully exporting large amounts of data. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, device and storage medium for asynchronous data export, so as to solve the problem that export timeout or export failure often occurs in the prior art when the data volume is large or the data calculation logic is complex.

[0005] To address the aforementioned technical problems, this application provides an asynchronous data export method, employing the following technical solution:

[0006] An asynchronous data export method includes the following steps:

[0007] Step S1: Receive the target data export instruction, parse the export instruction, and obtain the instruction parsing result, wherein the instruction parsing result includes the address of the target interface to be called;

[0008] Step S2: Generate an export task based on the target interface address and preset export parameters;

[0009] Step S3: Execute the export task to obtain the target data;

[0010] Step S4: Fill the target data into the preset receiving file, generate the export file, and terminate the export task.

[0011] Furthermore, before executing the step of generating an export task based on the target interface address and preset export parameters, the method further includes:

[0012] Based on the preset lock detection method, determine whether an export task already exists between the target user and the target data;

[0013] If it exists, a prompt is returned to the target user;

[0014] If it does not exist, an export task between the target user and the target data is established through the target interface, and a Redis distributed lock is added to the export task.

[0015] Furthermore, after performing the step of generating an export task based on the target interface address and preset export parameters, the method further includes:

[0016] Based on the configuration level of the export task, determine whether it is necessary to isolate the execution of the export task.

[0017] If it is necessary to isolate the execution of the export task, the export task is sent to the preset isolation thread pool according to the preset isolation thread pool identifier.

[0018] If isolation processing of the export task is not required, the export task is sent to the preset ordinary thread pool according to the preset ordinary thread pool identifier.

[0019] Furthermore, during the step of executing the export task and obtaining the target data, the method further includes:

[0020] Based on the preset percentage algorithm formula, calculate the percentage of the data that has been exported to the total amount of data that the export task needs to export;

[0021] A dynamic progress bar is constructed based on the percentage and a preset export progress bar, and the dynamic progress bar is displayed in real time on the preset export monitoring interface.

[0022] Furthermore, after performing the step of displaying the dynamic progress bar in real time on the preset export monitoring interface, the method further includes:

[0023] A scheduled task method is used to periodically monitor the percentage corresponding to the dynamic progress bar;

[0024] If the percentage remains unchanged for N consecutive timed task intervals, then the export task is terminated and steps S2 to S3 are re-executed, where N is a positive integer greater than 1.

[0025] If M consecutive export tasks fail, a manual intervention prompt is sent to the export monitoring interface, where M is a positive integer greater than 1.

[0026] Furthermore, after performing the step of terminating the export task, the method further includes:

[0027] Obtain the lock identifier information of the Redis distributed lock corresponding to the export task;

[0028] The Redis distributed lock corresponding to the export task is unlocked based on the lock identifier information.

[0029] Furthermore, the step of filling the target data into a preset receiving file and generating an export file specifically includes:

[0030] The acquired target data is inserted into the designated receiving position reserved in the receiving file, and the received file with the insertion completed is obtained;

[0031] According to the preset algorithm formula: The received file after insertion is paginated, where p represents the target page number, L represents the total number of data entries in the target data, and l represents the number of data entries that can be recorded per page of the exported file;

[0032] The received file after pagination processing is obtained as the exported file.

[0033] Furthermore, after performing the step of filling the target data into a preset receiving file and generating an export file, the method further includes:

[0034] A heartbeat reporting mechanism is used to report the export status of the target data in real time.

[0035] If the export status is "export completed", an email notification indicating that the data export is complete will be sent to the target user according to a preset email trigger task. The email notification includes the filename and download link of the exported file.

[0036] To address the aforementioned technical problems, this application also provides an asynchronous data export device, which employs the following technical solution:

[0037] An asynchronous data export device, comprising:

[0038] The instruction processing module is used to receive a target data export instruction, parse the export instruction, and obtain the instruction parsing result, wherein the instruction parsing result includes the address of the target interface to be called;

[0039] The task generation module is used to generate an export task based on the target interface address and preset export parameters;

[0040] The task execution module is used to execute the export task and obtain the target data;

[0041] The export processing module is used to fill the target data into a preset receiving file, generate an export file, and terminate the export task.

[0042] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0043] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the asynchronous data export method described above.

[0044] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0045] A computer-readable storage medium storing computer-readable instructions that, when executed by a processor, implement the steps of the asynchronous data export method described above.

[0046] Compared with the prior art, the embodiments of this application have the following main advantages:

[0047] The asynchronous data export method described in this application involves receiving a target data export instruction, parsing the instruction, obtaining the instruction parsing result (including the address of the target interface to be called), generating an export task based on the target interface address and preset export parameters, executing the export task to obtain the target data, filling the target data into a preset receiving file to generate an export file, and terminating the export task. This method can export large amounts of data or data with complex computational logic, provides real-time updates and viewing of the export progress, supports breakpoint export in case of server anomalies, and provides asynchronous export functionality with email notifications to business personnel upon completion of the export. This significantly improves the user experience and avoids excessive server load due to the export function, preventing server crashes or automatic restarts. Attached Figure Description

[0048] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0050] Figure 2 A flowchart of an embodiment of the asynchronous data export method according to this application;

[0051] Figure 3 yes Figure 2A flowchart of a specific embodiment of step S4 shown;

[0052] Figure 4 A schematic diagram of a structural embodiment of the asynchronous data export apparatus according to this application;

[0053] Figure 5 A schematic diagram of the structure of an embodiment of the computer device according to this application. Detailed Implementation

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0055] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0056] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0057] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0058] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0059] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0060] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0061] It should be noted that the asynchronous data export method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the asynchronous data export device is generally set in the server / terminal device.

[0062] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0063] Continue to refer to Figure 2 A flowchart illustrating an embodiment of the asynchronous data export method according to this application is shown. The asynchronous data export method includes the following steps:

[0064] Step S1: Receive the target data export instruction, parse the export instruction, and obtain the instruction parsing result, wherein the instruction parsing result includes the target interface address to be called.

[0065] In this embodiment, the export command is issued by the target user through human-computer interaction. Specifically, an export command trigger button can be set in a preset operation interface. When the target user needs to obtain target data and export it, he / she can directly issue the target data export command by clicking the trigger button.

[0066] Step S2: Generate an export task based on the target interface address and preset export parameters.

[0067] In this embodiment, the preset export parameters include, but are not limited to, the `param` parameter passed from the front-end page, the Excel template file, custom table headers, table name, exported file name, the method name for querying the total number of data rows, and the method name for querying the data list. Correspondingly, the export parameters can be pre-assembled, and the assembly result can be cached in a Redis cache. When the target interface address is obtained, the assembly result is directly retrieved from the Redis cache, and an export task is generated based on the target interface address and the assembly result.

[0068] In this embodiment, before executing the step of generating an export task based on the target interface address and preset export parameters, the method further includes: determining whether an export task already exists between the target user and the target data according to a preset lock detection method; if it exists, returning a prompt to the target user; if it does not exist, establishing an export task between the target user and the target data through the target interface, and adding a Redis distributed lock to the export task.

[0069] By using Redis distributed locks, we can prevent the same data from being exported multiple times simultaneously, thus avoiding wasting server resources.

[0070] In this embodiment, after executing the step of generating an export task based on the target interface address and preset export parameters, the method further includes: determining whether the execution of the export task needs to be isolated according to the configuration level of the export task; if the execution of the export task needs to be isolated, then the export task is sent to a preset isolated thread pool according to a preset isolation thread pool identifier; if the execution of the export task does not need to be isolated, then the export task is sent to a preset ordinary thread pool according to a preset ordinary thread pool identifier.

[0071] By setting up an isolated thread pool specifically for exporting specific data, we can prevent the data export from consuming thread resources and affecting the normal use of the system.

[0072] Step S3: Execute the export task to obtain the target data.

[0073] In this embodiment, during the step of executing the export task and obtaining the target data, the method further includes: calculating the percentage of the amount of data that has been exported to the total amount of data that needs to be exported in the export task according to a preset percentage algorithm formula; constructing a dynamic progress bar based on the percentage and a preset export progress bar, and displaying the dynamic progress bar in real time on a preset export monitoring interface.

[0074] By using a real-time dynamic progress bar, target users can intuitively understand the export status of the target data.

[0075] In this embodiment, after performing the step of displaying the dynamic progress bar in real time on the preset export monitoring interface, the method further includes: using a timed task method to periodically monitor the percentage corresponding to the dynamic progress bar; if the percentage remains unchanged for N consecutive timed task intervals, then the export task is terminated and steps S2 to S3 are re-executed, where N is a positive integer greater than 1; if M consecutive export tasks fail, then a manual intervention prompt is sent to the export monitoring interface, where M is a positive integer greater than 1.

[0076] By monitoring scheduled tasks, delayed or stopped export tasks can be identified in a timely manner, making it easier for target users to regenerate export tasks. At the same time, by tracking the number of failed export tasks, manual intervention can be promptly alerted to adjust the export tasks accordingly.

[0077] In this embodiment, after performing the step of terminating the execution of the export task, the method further includes: obtaining the lock identifier information of the Redis distributed lock corresponding to the export task; and unlocking the Redis distributed lock corresponding to the export task according to the lock identifier information.

[0078] The unlocking process allows the target user to export the current data again later.

[0079] Step S4: Fill the target data into the preset receiving file, generate the export file, and terminate the export task.

[0080] Continue to refer to Figure 3 , Figure 3 yes Figure 2 A flowchart of a specific embodiment of step S4 shown includes:

[0081] Step 301: Insert the acquired target data into the designated receiving position reserved in the receiving file, and obtain the received file after insertion;

[0082] Step 302, according to the preset algorithm formula: The received file after insertion is paginated, where p represents the target page number, L represents the total number of data entries in the target data, and l represents the number of data entries that can be recorded per page of the exported file;

[0083] Step 303: Obtain the received file after pagination processing as the exported file.

[0084] Pagination improves the readability of exported files.

[0085] In this embodiment, after performing the step of filling the target data into a preset receiving file and generating an exported file, the method further includes: using a heartbeat reporting mechanism to report the export status of the target data in real time; if the export status is "export completed", then sending an email notification to the target user indicating that the data export is complete according to a preset email trigger task, wherein the email notification includes the filename and download link of the exported file.

[0086] In this embodiment, after performing the step of terminating the execution of the export task, the method further includes: obtaining the lock identifier information of the Redis distributed lock corresponding to the export task; and unlocking the Redis distributed lock corresponding to the export task according to the lock identifier information.

[0087] This application involves receiving a target data export instruction, parsing the instruction to obtain an instruction parsing result, wherein the instruction parsing result includes the address of the target interface to be called; generating an export task based on the target interface address and preset export parameters; executing the export task to obtain the target data; filling the target data into a preset receiving file to generate an export file; and terminating the execution of the export task. This method utilizes distributed locking mechanisms, isolated thread pools, and scheduled tasks to export large amounts of data or data with complex computational logic. It allows for real-time updates and viewing of the export progress, supports breakpoint export in case of server anomalies, and provides asynchronous export functionality with email notifications to business personnel upon completion of the export. This significantly improves the user experience while preventing excessive server load due to the export function, thus preventing server crashes or automatic restarts.

[0088] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0089] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0090] In this embodiment, based on distributed storage and big data processing technologies in artificial intelligence, a target data export instruction is received, the export instruction is parsed, and an instruction parsing result is obtained. The instruction parsing result includes the address of the target interface to be called. An export task is generated based on the target interface address and preset export parameters. The export task is executed to obtain the target data. The target data is filled into a preset receiving file to generate an export file. This system can export large amounts of data or data with complex computational logic, and provides real-time updates and viewing of the export progress. It supports breakpoint export in case of server abnormalities and asynchronous export functionality, including email notification to business personnel upon completion of the export. This significantly improves the user experience while preventing excessive server load due to the export function, thus preventing server crashes or automatic restarts.

[0091] Further reference Figure 4 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of an asynchronous data export device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0092] like Figure 4 As shown, the asynchronous data export device 400 described in this embodiment includes: an instruction processing module 401, a task generation module 402, a task execution module 403, and an export processing module 404. Wherein:

[0093] The instruction processing module 401 is used to receive a target data export instruction, parse the export instruction, and obtain an instruction parsing result, wherein the instruction parsing result includes the address of the target interface to be called;

[0094] The task generation module 402 is used to generate an export task based on the target interface address and preset export parameters;

[0095] Task execution module 403 is used to execute the export task and obtain the target data;

[0096] The export processing module 404 is used to fill the target data into a preset receiving file, generate an export file, and terminate the execution of the export task.

[0097] In some specific embodiments of this application, the asynchronous data export device 400 further includes a lock identification module, which is used to determine whether an export task already exists between the target user and the target data according to a preset lock detection method.

[0098] In some specific embodiments of this application, the asynchronous data export device 400 further includes a locking processing module, which is used to establish an export task between the target user and the target data through the target interface, and add a Redis distributed lock to the export task.

[0099] In some specific embodiments of this application, the asynchronous data export device 400 further includes an isolation identification module, which is used to determine whether the execution of the export task needs to be isolated based on the configuration level of the export task.

[0100] In some specific embodiments of this application, the asynchronous data export device 400 further includes an isolation processing module, which is used to send the export task to a preset isolation thread pool according to a preset isolation thread pool identifier.

[0101] In some specific embodiments of this application, the asynchronous data export device 400 further includes a general processing module, which is used to send the export task to a preset general thread pool according to a preset general thread pool identifier.

[0102] In some specific embodiments of this application, the asynchronous data export device 400 further includes an export percentage calculation module, which is used to calculate the percentage of the amount of data that has been exported to the total amount of data that the export task needs to export, according to a preset percentage algorithm formula.

[0103] In some specific embodiments of this application, the asynchronous data export device 400 further includes an export progress monitoring module, which is used to construct a dynamic progress bar based on the percentage and a preset export progress bar, and to display the dynamic progress bar in real time on a preset export monitoring interface.

[0104] In some specific embodiments of this application, the asynchronous data export device 400 further includes a timed monitoring module, which is used to periodically monitor the percentage corresponding to the dynamic progress bar using a timed task method.

[0105] In some specific embodiments of this application, the asynchronous data export device 400 further includes an export exception handling module. The export exception handling module is used to terminate the execution of the export task and re-execute steps S2 to S3 if the percentage remains unchanged for N consecutive timed task intervals, where N is a positive integer greater than 1; it is also used to send a manual intervention prompt to the export monitoring interface if M consecutive export tasks fail, where M is a positive integer greater than 1.

[0106] In some specific embodiments of this application, the asynchronous data export device 400 further includes a paging processing module, which is used to process data according to a preset algorithm formula: The received file after insertion is paginated.

[0107] In some specific embodiments of this application, the asynchronous data export device 400 further includes an unlocking processing module, which is used to obtain the lock identifier information of the Redis distributed lock corresponding to the export task; and is also used to unlock the Redis distributed lock corresponding to the export task according to the lock identifier information.

[0108] In some specific embodiments of this application, the asynchronous data export device 400 further includes a real-time reporting module, which is used to report the export status of the target data in real time using a heartbeat reporting mechanism.

[0109] In some specific embodiments of this application, the asynchronous data export device 400 further includes an email notification module, which is used to send an email notification to the target user indicating that the data export is complete according to a preset email trigger task. The email notification includes the filename and download link of the exported file.

[0110] This application involves receiving a target data export instruction, parsing the instruction to obtain an instruction parsing result, wherein the instruction parsing result includes the address of the target interface to be called; generating an export task based on the target interface address and preset export parameters; executing the export task to obtain the target data; filling the target data into a preset receiving file to generate an export file; and terminating the execution of the export task. This method utilizes distributed locking mechanisms, isolated thread pools, and scheduled tasks to export large amounts of data or data with complex computational logic. It allows for real-time updates and viewing of the export progress, supports breakpoint export in case of server anomalies, and provides asynchronous export functionality with email notifications to business personnel upon completion of the export. This significantly improves the user experience while preventing excessive server load due to the export function, thus preventing server crashes or automatic restarts.

[0111] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0112] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0113] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.

[0114] The computer device 5 includes a memory 5a, a processor 5b, and a network interface 5c that are interconnected via a system bus. It should be noted that only the computer device 5 with components 5a-5c is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0115] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0116] The memory 5a includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 5a may be an internal storage unit of the computer device 5, such as the hard disk or memory of the computer device 5. In other embodiments, the memory 5a may also be an external storage device of the computer device 5, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 5. Of course, the memory 5a may include both the internal storage unit and its external storage device of the computer device 5. In this embodiment, the memory 5a is typically used to store the operating system and various application software installed on the computer device 5, such as computer-readable instructions for asynchronous data export methods. In addition, the memory 5a can also be used to temporarily store various types of data that have been output or will be output.

[0117] In some embodiments, the processor 5b may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 5b is typically used to control the overall operation of the computer device 5. In this embodiment, the processor 5b is used to execute computer-readable instructions stored in the memory 5a or to process data, for example, to execute computer-readable instructions for the asynchronous data export method.

[0118] The network interface 5c may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 5 and other electronic devices.

[0119] The computer device proposed in this embodiment belongs to the field of data transmission technology. This application receives a target data export instruction, parses the instruction, and obtains the instruction parsing result, wherein the instruction parsing result includes the address of the target interface to be called; based on the target interface address and preset export parameters, an export task is generated; the export task is executed to obtain the target data; the target data is filled into a preset receiving file to generate an export file, and the execution of the export task is terminated. This method uses distributed lock mechanisms, isolated thread pools, and scheduled tasks to export large amounts of data or data with complex computational logic, updates and displays the export progress in real time, supports breakpoint export in case of server abnormalities, and provides an asynchronous export function that notifies business personnel via email upon completion of the export. This greatly improves the user experience of the system while avoiding excessive server pressure due to the export function, preventing server crashes or automatic restarts.

[0120] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by a processor to cause the processor to perform the steps of the asynchronous data export method described above.

[0121] The computer-readable storage medium proposed in this embodiment belongs to the field of data transmission technology. This application receives a target data export instruction, parses the instruction, and obtains the instruction parsing result, wherein the instruction parsing result includes the address of the target interface to be called; based on the target interface address and preset export parameters, an export task is generated; the export task is executed to obtain the target data; the target data is filled into a preset receiving file to generate an export file, and the execution of the export task is terminated. This method uses distributed lock mechanisms, isolated thread pools, and scheduled tasks to export large amounts of data or data with complex computational logic, updates and displays the export progress in real time, supports breakpoint export in case of server abnormalities, and provides an asynchronous export function that notifies business personnel via email upon completion of the export. This greatly improves the user experience of the system while avoiding excessive server pressure due to the export function, preventing server crashes or automatic restarts.

[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0123] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A method for asynchronous data export, characterized in that, Includes the following steps: Step S1: Receive the target data export instruction, parse the export instruction, and obtain the instruction parsing result, wherein the instruction parsing result includes the address of the target interface to be called; Step S2: Based on the target interface address and preset export parameters, generate an export task. The preset export parameters include, but are not limited to, the `param` parameter passed from the front-end page, an Excel template file, custom table headers, table name, exported file name, the method name for querying the total number of data rows, and the method name for querying the data list. Specifically, the export parameters are pre-assembled, and the assembly result is cached in a preset Redis cache. When the target interface address is obtained, the assembly result is retrieved from the Redis cache. An export task is generated based on the target interface address and the assembly result. After generating the export task, the method further includes: adding a Redis distributed lock to the export task; Step S3, execute the export task to obtain the target data, wherein, during the process of executing the export task and obtaining the target data, the following steps are also included: Based on the preset percentage algorithm formula, calculate the percentage of the data that has been exported to the total amount of data that the export task needs to export; A dynamic progress bar is constructed based on the percentage and a preset export progress bar, and the dynamic progress bar is displayed in real time on the preset export monitoring interface. A scheduled task method is used to periodically monitor the percentage corresponding to the dynamic progress bar; If the percentage remains unchanged for N consecutive timed task intervals, the export task is terminated, and steps S2 to S3 are re-executed, where N is a positive integer greater than 1. After terminating the export task, the method further includes: obtaining the lock identifier information of the Redis distributed lock corresponding to the export task; and unlocking the Redis distributed lock corresponding to the export task based on the lock identifier information. If M consecutive export tasks fail, a manual intervention prompt is sent to the export monitoring interface, where M is a positive integer greater than 1. Step S4: Fill the target data into a preset receiving file to generate an export file, and terminate the export task. After performing the step of terminating the export task, the method further includes: obtaining the lock identifier information of the Redis distributed lock corresponding to the export task; and unlocking the Redis distributed lock corresponding to the export task according to the lock identifier information.

2. The asynchronous data export method according to claim 1, characterized in that, Before executing the step of generating an export task based on the target interface address and preset export parameters, the method further includes: Based on the preset lock detection method, determine whether an export task already exists between the target user and the target data; If it exists, a prompt is returned to the target user; If it does not exist, then an export task between the target user and the target data is established through the target interface.

3. The asynchronous data export method according to claim 1, characterized in that, After performing the step of generating an export task based on the target interface address and preset export parameters, the method further includes: Based on the configuration level of the export task, determine whether it is necessary to isolate the execution of the export task. If it is necessary to isolate the execution of the export task, the export task is sent to the preset isolation thread pool according to the preset isolation thread pool identifier. If isolation processing of the export task is not required, the export task is sent to the preset ordinary thread pool according to the preset ordinary thread pool identifier.

4. The asynchronous data export method according to claim 1, characterized in that, The step of filling the target data into a preset receiving file and generating an export file specifically includes: The acquired target data is inserted into the designated receiving position reserved in the receiving file, and the received file with the insertion completed is obtained; According to the preset algorithm formula: The received file after insertion is paginated, wherein... Indicates the target page number. This indicates the total number of data entries in the target data. This indicates the number of data entries that can be recorded per page of the exported file; The received file after pagination processing is obtained as the exported file.

5. The asynchronous data export method according to claim 1, characterized in that, After performing the step of filling the target data into a preset receiving file and generating an export file, the method further includes: A heartbeat reporting mechanism is used to report the export status of the target data in real time. If the export status is "export completed", an email notification indicating that the data export is complete will be sent to the target user according to a preset email trigger task. The email notification includes the filename and download link of the exported file.

6. A data asynchronous export device, characterized in that, The asynchronous data export device implements the steps of the asynchronous data export method as described in any one of claims 1 to 5, and the asynchronous data export device comprises: The instruction processing module is used to receive a target data export instruction, parse the export instruction, and obtain the instruction parsing result, wherein the instruction parsing result includes the address of the target interface to be called; The task generation module is used to generate an export task based on the target interface address and preset export parameters; The task execution module is used to execute the export task and obtain the target data; The export processing module is used to fill the target data into a preset receiving file, generate an export file, and terminate the export task.

7. A computer device comprising a memory and a processor, the memory storing computer-readable instructions, wherein the processor, when executing the computer-readable instructions, implements the steps of the asynchronous data export method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the asynchronous data export method as described in any one of claims 1 to 5.

Images