Data export method, data export device, electronic device and medium

Abstract

The present disclosure provides a data export method applied to a first server. The method includes: receiving a call request for executing a first data export task; obtaining the amount of exported data of the first data export task according to the information of the call request; determining a data export method based on the amount of exported data of the first data export task, which includes: when the amount of exported data of the first data export task is greater than or equal to a first preset threshold, using multi-threaded concurrent data export; when the amount of exported data of the first data export task is less than the first preset threshold, using a single thread to export data; determining at least one data export file to export data to the at least one data export file. The present disclosure also provides a data export method, device, electronic device and storage medium applied to a second server. The data export method, device, electronic device and storage medium provided by the present disclosure can be used in the financial field or other fields.

Description

Technical Field

[0001] The present disclosure relates to the financial field, and more specifically, to a data export method, a data export device, an electronic device and a storage medium. Background Art

[0002] With the development of computer technology, huge amounts of data are generated, which needs to be collected, stored, exported and analyzed. With the existing data export function, users can obtain the required target data according to the selected data type and filtering conditions. In the case of small data volume, simple business and small number of concurrent users, the existing data export function can meet most data export needs.

[0003] In the process of implementing the concept of the present disclosure, the inventors found that there are at least the following problems in the prior art: when processing the export of large amounts of data, a large amount of system resources will be occupied, resulting in a decrease in the performance of the data export function and a reduced user experience. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide a data export method, device, electronic device and storage medium that can select different export modes according to the amount of exported data.

[0005] One aspect of an embodiment of the present disclosure provides a data export method applied to a first server. The data export method includes: receiving a call request for executing a first data export task; obtaining the amount of exported data of the first data export task according to information of the call request; determining a data export method based on the amount of exported data of the first data export task, which includes: when the amount of exported data of the first data export task is greater than or equal to a first preset threshold, using multi-threaded concurrent data export; when the amount of exported data of the first data export task is less than the first preset threshold, using a single thread to export data; determining at least one data export file to export data to the at least one data export file.

[0006] According to an embodiment of the present disclosure, the use of multi-threaded concurrent data export includes: splitting the first data export task into N second data export tasks, the amount of exported data of the second data export task is less than the first preset threshold; using multi-threaded concurrent execution of N second data export tasks, where N is an integer greater than or equal to 2.

[0007] According to an embodiment of the present disclosure, decomposing the first data export task into N second data export tasks includes: setting a second preset threshold, the second preset threshold is smaller than the first preset threshold; starting from the starting position of the first data export task, allocating a part of the first data export task to a second data export task every second preset threshold, to obtain N second data export tasks.

[0008] According to an embodiment of the present disclosure, the concurrent execution of N second data export tasks using multiple threads includes: determining a first data export file; obtaining the starting position of the export data of the first data export task in the first data export file; obtaining an export position in the first data export file every second preset threshold from the starting position until N-1 export positions are obtained; and writing the export data of N second data export tasks into the first data export file based on the starting position and the N-1 export positions.

[0009] According to an embodiment of the present disclosure, the concurrent execution of N second data export tasks using multiple threads includes looping the following operations until all N second data export tasks are exported: determining the second data export file exported by the current first data export task; if the available space of the second data export file is greater than or equal to the amount of exported data of the second data export task to be exported next, writing the second data export task to the second data export file; or if the available space of the second data export file is less than the amount of exported data of the second data export task to be exported next, creating a third data export file and writing the second data export task to the third data export file.

[0010] According to an embodiment of the present disclosure, determining at least one data export file includes: if the first data export task exports data for the first time, creating a fourth data export file to export the data to the fourth data export file; or if the first data export task does not export data for the first time, exporting the data to a fifth data export file, wherein the fifth data export file is the file to which the first data export task exported data the last time.

[0011] Another aspect of the disclosed embodiment provides a data export method applied to a second server. The data export method includes: obtaining a third data export task; determining a target server corresponding to the third data export task from at least one first server, including: querying the target server corresponding to the third data export task according to the pre-stored correspondence between the at least one first server and the data service type; issuing a call request so that the target server executes the third data export task.

[0012] According to an embodiment of the present disclosure, the third data export task is pre-added to a message queue, and obtaining the third data export task includes: obtaining the third data export task from the message queue for asynchronous processing.

[0013] According to an embodiment of the present disclosure, the issuing of a call request includes: querying a data service configuration table according to the data service type of the third data export task to obtain interface information of the target server, wherein the data service configuration table includes the interface information of the at least one first server and the pre-stored correspondence; issuing a call request based on the interface information of the target server.

[0014] According to an embodiment of the present disclosure, before querying the data service configuration table, the method further includes: setting interface information of the at least one first server in the data service configuration table; and setting a correspondence between the at least one first server in the data service configuration table and a data service type.

[0015] Another aspect of the disclosed embodiment provides a data export device applied to a first server. The data export device includes a request receiving module, a data query module, a first determination module, and a file determination module. The task acquisition module is used to acquire a third data export task; the second determination module is used to determine a target server corresponding to the third data export task from at least one first server, including: querying the target server corresponding to the third data export task according to the pre-stored correspondence between the at least one first server and the data service type; the request sending module is used to issue a call request so that the target server executes the third data export task.

[0016] Another aspect of the disclosed embodiment provides a data export device applied to a second server. The data export device includes a task acquisition module, a second determination module, and a request sending module. The task acquisition module is used to acquire a third data export task; the second determination module is used to determine a target server corresponding to the third data export task from at least one first server, which includes: querying the target server corresponding to the third data export task according to the pre-stored correspondence between the at least one first server and the data service type; the request sending module is used to issue a call request so that the target server executes the third data export task.

[0017] Another aspect of the present disclosure provides an electronic device. The electronic device includes one or more memories and one or more processors. The memories store computer executable instructions. The processor executes the instructions to implement the above method.

[0018] Another aspect of an embodiment of the present disclosure provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method as described above.

[0019] According to another aspect of the embodiments of the present disclosure, a computer program product is provided, including a computer program / instruction, which implements the above-mentioned method when the computer program / instruction is executed by a processor.

[0020] One or more of the above embodiments have the following advantages or beneficial effects: it can at least partially solve the problem of decreased data export performance when exporting large amounts of data, and adopt a corresponding data export method to export data according to the amount of exported data, effectively utilize system resources, and improve data export performance. It can also cope with high-concurrency scenarios where multiple users establish data export tasks, and asynchronously process data export tasks, thereby improving server stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0022] Figure 1 The system architecture of data export according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 2 A flow chart schematically illustrates a data export method applied to a second server according to an embodiment of the present disclosure;

[0024] Figure 3 A flowchart of setting a data service configuration table according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 4 The flowchart of the second server issuing a call request according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 5 A flowchart of a data export method applied to a first server according to an embodiment of the present disclosure is schematically shown;

[0027] Figure 6 Schematically shows a flow chart of determining a data export file according to an embodiment of the present disclosure;

[0028] Figure 7A flowchart of exporting data using multiple threads concurrently according to an embodiment of the present disclosure is schematically shown;

[0029] Figure 8 A flowchart of decomposing a first data export task into N second data export tasks according to an embodiment of the present disclosure is schematically shown;

[0030] Fig. 9 A flowchart of concurrently executing N second data export tasks using multiple threads according to an embodiment of the present disclosure is schematically shown;

[0031] Fig.10 A flowchart schematically shows a method of concurrently executing N second data export tasks using multiple threads according to another embodiment of the present disclosure;

[0032] Fig.11 A block diagram schematically shows a data export device applied to a second server according to an embodiment of the present disclosure;

[0033] Fig.12 The application scenario diagram of the task acquisition module according to the embodiment of the present disclosure is schematically shown;

[0034] Fig.13 The schematic diagram shows the process of storing messages in a message queue at a fixed time according to an embodiment of the present disclosure;

[0035] Fig.14 A block diagram schematically shows a data export device applied to a first server according to an embodiment of the present disclosure;

[0036] Fig.15 The flowchart schematically shows the process of periodically taking out task messages from a message queue and executing them according to an embodiment of the present disclosure.

[0037] Fig.16 The block diagram of a computer system suitable for implementing the data export method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0038] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0039] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0040] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0041] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0042] The embodiment of the present disclosure provides a data export method applied to a first server. The data export method includes: receiving a call request for executing a first data export task; obtaining the amount of exported data of the first data export task according to the information of the call request; determining a data export method based on the amount of exported data of the first data export task, which includes: when the amount of exported data of the first data export task is greater than or equal to a first preset threshold, using multi-threaded concurrent data export; when the amount of exported data of the first data export task is less than the first preset threshold, using a single thread to export data; determining at least one data export file to export data to the at least one data export file.

[0043] The embodiment of the present disclosure also provides a data export method applied to a second server. The data export method includes: obtaining a third data export task; determining a target server corresponding to the third data export task from at least one first server, including: querying the target server corresponding to the third data export task according to the pre-stored correspondence between the at least one first server and the data service type; issuing a call request so that the target server executes the third data export task. The embodiment of the present disclosure also provides a data export device, an electronic device, and a medium.

[0044] Figure 1 The system architecture of the data export method according to the embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 The examples shown are only examples of system architectures to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios. For example, an embodiment of the present disclosure can be applied to the field of financial technology to export data generated in financial services in order to analyze and process the data.

[0045] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, 104, networks 105, 106, a task management server 107, and data service servers 108, 109, 110. The network 105 is used to provide a medium for communication links between the terminal devices 101, 102, 103, 104 and the task management server 107, and the network 106 is used to provide a medium for communication links between the task management server 107 and the data service servers 108, 109, 110. The networks 105 and 106 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0046] The user can use the terminal devices 101, 102, 103, 104 to interact with the task management server 107 through the network 105 to send and receive messages for processing data export tasks. The task management server 107 sends a call request to the data service servers 108, 109, 110 through the network 106. Various communication client applications can be installed on the terminal devices 101, 102, 103 and 104, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0047] The terminal devices 101 , 102 , 103 and 104 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers and the like.

[0048] The task management server 107 may be a server that provides various services, such as a background management server that provides support for websites browsed, service systems logged in, applications, etc. by users using the terminal devices 101, 102, 103, and 104 (only as an example). The background management server may analyze and process the received data such as the user data export request, and feed back the processing results (such as web pages, information, or data obtained or generated according to the user request) to the terminal device.

[0049] The data service servers 108, 109, 110 may deploy specific data service systems, wherein the data service systems are maintained by dedicated service personnel. The data service servers 108, 109, 110 may be servers for providing functions such as storage, export, acquisition or management of data, and execute the call request issued by the task management server 107.

[0050] It should be noted that the data export method applied to the second server provided in the embodiment of the present disclosure can generally be executed by the task management server 107. Accordingly, the data export device applied to the second server provided in the embodiment of the present disclosure can generally be set in the task management server 107. The data export method applied to the second server provided in the embodiment of the present disclosure can also be executed by a server or server cluster different from the task management server 107. Accordingly, the data export device applied to the second server provided in the embodiment of the present disclosure can also be set in a server or server cluster different from the task management server 107.

[0051] It should be noted that the data export method applied to the first server provided in the embodiment of the present disclosure can generally be executed by at least one of the data service servers 108, 109, and 110. Accordingly, the data export device applied to the first server provided in the embodiment of the present disclosure can generally be set in at least one of the data service servers 108, 109, and 110. The data export method applied to the first server provided in the embodiment of the present disclosure can also be executed by a server or server cluster different from the data service servers 108, 109, and 110. Accordingly, the data export device applied to the first server provided in the embodiment of the present disclosure can also be set in a server or server cluster different from the data service servers 108, 109, and 110.

[0052] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the actual application, there may be any number of terminal devices, networks and servers.

[0053] The following combination Figure 1The system architecture 100 is used to exemplarily illustrate the technical solution of the embodiment of the present disclosure. Among them, the task management server 107 can be used to process the user's data export task. The data that the user wants to export can be managed by any one of the data service servers 108, 109, and 110, wherein the corresponding data can be stored in the data service servers 108, 109, and 110, and can also be obtained by the data service servers 108, 109, and 110 from other databases.

[0054] Figure 2 The flowchart of the data export method applied to the second server according to the embodiment of the present disclosure is schematically shown. The second server may be the task management server 107 .

[0055] In operation S210, a third data export task is obtained;

[0056] According to the embodiments of the present disclosure, referring to Figure 1 , the user can create a data export task through the terminal device 101. For example, if the user wants to download the consumption flow details in the bank account, first filter the conditions such as account, date, amount range, income or expenditure, and then click the export button on the bank webpage to create a new download task, and finally the task management server 107 obtains the data export task of the bank account flow details created by the user.

[0057] According to an embodiment of the present disclosure, the third data export task is pre-added to a message queue, and the obtaining of the third data export task includes: obtaining the third data export task from the message queue for asynchronous processing.

[0058] Reference Figure 1 First, for example, after a user creates a data export task through the terminal device 101, the task management server 107 receives a record of the data export task, which includes information such as the screening conditions and types of data services that the user wants to download. It should be known that the task management server 107 can process multiple tasks created by one user or multiple tasks created by multiple users, for example, a task list including multiple data export tasks can be formed.

[0059] Then, for example, the task management server 107 is instructed to periodically traverse and query the tasks in the task list that are in an uncompleted state, and store the uncompleted export tasks in the message queue.

[0060] Finally, the task management server 107 may be configured to take out the data export task from the message queue at regular intervals and process the data export task asynchronously to achieve the purpose of obtaining data.

[0061] By using the data export method of the embodiment of the present disclosure, after a user creates a data export task, the task will not be processed immediately to avoid occupying current resources. Instead, the data export task is placed in a message queue, and the data export task is obtained from the message queue at another time and processed asynchronously. This allows the creation of a data export task to be separated from the specific execution of the data export. The user can perform other operations while waiting for the file download to be completed, which improves the user experience.

[0062] In operation S220, a target server corresponding to the third data export task is determined from at least one first server, which includes: querying the target server corresponding to the third data export task based on a pre-stored correspondence between the at least one first server and the data service type.

[0063] The following combination Figure 3 , describing in detail the process of querying the target server corresponding to the third data export task in operation S220.

[0064] Figure 3 A flowchart for setting a data service configuration table according to an embodiment of the present disclosure is schematically shown.

[0065] like Figure 3 As shown, the data export method according to the embodiment of the present disclosure further includes setting a data service configuration table, which may specifically include operations S310 to S320.

[0066] In operation S310, interface information of the at least one first server in the data service configuration table is set.

[0067] In operation S320, a correspondence between the at least one first server and a data service type in the data service configuration table is set.

[0068] According to an embodiment of the present disclosure, referring to Figure 1 For example, the data service server 108 may be configured with a data service system for managing bank account flows, the data service server 109 may be configured with a data service system for managing regular financial management, and the data service server 110 may be configured with a data service system for managing precious metal transactions. In addition, a data service configuration table may be provided in the task management server 107.

[0069] First, since the data service servers 108, 109 and 110 are respectively deployed with data service systems of different categories, the corresponding relationship between each data service server 108, 109 and 110 and the data service category can be set in the data service configuration table.

[0070] Secondly, the business personnel can set the corresponding interface information in the data service configuration table according to the different interface regulations of the data service servers 108, 109 and 110. According to the interface information, the task management server 107 can generalize and call the data service servers 108, 109 and 110 to perform the data export function, so as to realize the method of obtaining data according to the specific data service.

[0071] According to an embodiment of the present disclosure, operations S310 to S320 may be performed before operation S210 to set a data service configuration table. After operation S210, the interface information and corresponding relationship in the pre-stored data configuration table may be used to query the target server corresponding to the third data export task, and then operation S220 may be performed.

[0072] In operation S230, a call request is issued to cause the target server to execute the third data export task.

[0073] The following combination Figure 4 , describing in detail the process of issuing a call request in operation S230.

[0074] Figure 4 The flowchart of the second server issuing a call request according to an embodiment of the present disclosure is schematically shown.

[0075] like Figure 4 As shown, the second server sending the calling request may also include operations S410 to S420.

[0076] In operation S410, a data service configuration table is queried according to the data service type of the third data export task to obtain interface information of the target server, wherein the data service configuration table includes the interface information of the at least one first server and the pre-stored correspondence.

[0077] In operation S420, a call request is issued based on the interface information of the target server.

[0078] According to the embodiments of the present disclosure, referring to Figure 1 For example, after the task management server 107 takes out the data export task created by the user using the terminal device 101 from the message queue, it first reads that the data type of the data export task is the bank account transaction details. Then, the task management server 107 queries the data service configuration table and obtains that the bank account transaction details are managed by the data service system deployed in the data service server 108 according to the pre-configured correspondence. Then, according to the interface information in the data service configuration table, the task management server 107 uses the generalized call method to instruct the data service server 108 to execute the data export task.

[0079] By using the data export method of the disclosed embodiment, it is possible to realize that multiple data service systems share a set of data export source codes. For example, in the prior art, since each data service server has different interface specifications and thus different data export business logics, it is necessary to develop for each business logic to meet the user's data export needs. The data export method of the disclosed embodiment can unify and integrate the data export tasks created by different users through the task management server 107 and process them asynchronously, so that data service systems with different business logics share a set of data export services, effectively reducing development costs.

[0080] Figure 5 A flowchart of a data export method applied to a first server according to an embodiment of the present disclosure is schematically shown. The first server may be Figure 1 Any one of the data service servers 108, 109 and 110.

[0081] like Figure 5 As shown, the data export method applied to the first server may include operations S510 to S540.

[0082] In operation S510, a call request for executing a first data export task is received.

[0083] In operation S520, the amount of exported data of the first data export task is obtained according to the information of the call request.

[0084] According to an embodiment of the present disclosure, for example, the task management server 107 receives a data export task about bank account transaction details, and uses a generalized call method to send a call request to the data service server 108. The call request information includes parameters such as account name, date, amount range, income, etc. Then, the data service server 108 queries the data that the user needs to export through the call request information.

[0085] In operation S530, a data export method is determined based on the amount of exported data of the first data export task, which may include operations S531 to S532.

[0086] In operation S531, when the amount of exported data of the first data export task is greater than or equal to a first preset threshold, multi-threaded concurrent data export is adopted.

[0087] In operation S532, when the amount of exported data of the first data export task is less than a first preset threshold, a single thread is used to export data.

[0088] According to the disclosed embodiment, the amount of exported data is compared with a preset download threshold, and the corresponding data export method is used to export data based on the comparison result. For example, the preset threshold can be the maximum data download amount set by the bank back-end staff according to the performance of the server.

[0089] For example, when the amount of data in the user's export task is greater than or equal to the preset maximum data download amount, the data export task can be executed in a concurrent manner and in multiple threads. When the amount of data in the user's export task is less than the preset maximum data download amount, the data can be exported in a single thread.

[0090] In operation S540, at least one data export file is determined to export the data into the at least one data export file.

[0091] Combine the following Figure 6 , describing in detail the process of exporting data to a file in operation S540.

[0092] Figure 6 The flowchart of determining a data export file according to an embodiment of the present disclosure is schematically shown.

[0093] like Figure 6 As shown, determining the data export file may include operations S610 to S620.

[0094] In operation S610, if the first data export task exports data for the first time, a fourth data export file is created to export the data into the fourth data export file.

[0095] In operation S620, if the first data export task is not the first data export, the data is exported to a fifth data export file, wherein the fifth data export file is the file to which the first data export task last exported data.

[0096] Using the embodiments of the present disclosure, the data volume of the user's export task is compared with a preset threshold. If it is less than the set value, the data can be exported in a single-threaded manner, which will not take up too many resources due to the small amount of data. If it is greater than the set value, the data can be exported in a multi-threaded manner. This can effectively solve the page freeze problem caused by users creating large data volume export tasks in the prior art, improve data export performance, and provide users with a better experience.

[0097] Figure 7 The flowchart of exporting data using multiple threads concurrently according to an embodiment of the present disclosure is schematically shown.

[0098] like Figure 7As shown, according to an embodiment of the present disclosure, exporting data using multiple threads concurrently in operation S531 may include operations S710 to S720.

[0099] In operation S710, the first data export task is decomposed into N second data export tasks, and the amount of exported data of the second data export tasks is less than a preset threshold.

[0100] In operation S720, N second data export tasks are executed concurrently by using multiple threads, where N is an integer greater than or equal to 2.

[0101] According to an embodiment of the present disclosure, when a data export task created by a user corresponds to a large amount of data, for example, the user wants to download the bank account transaction details, and after setting the account, date, amount range, income or expenditure and other conditions, there are 100,000 transaction details. For example, if the preset threshold is 10,000, the target transaction details to be exported are greater than the preset threshold. Then the task including 100,000 data is disassembled into multiple export tasks with data amounts less than the preset threshold, and then multiple tasks with small data amounts are executed concurrently in a multi-threaded manner.

[0102] By utilizing the data export method of the disclosed embodiment, a large data volume download task is broken down into multiple small data volume export tasks, and then exported in batches, which can effectively improve data export performance and solve the problem of exporting large data files.

[0103] Figure 8 The flowchart of decomposing a first data export task into N second data export tasks according to an embodiment of the present disclosure is schematically shown.

[0104] like Figure 8 As shown, according to an embodiment of the present disclosure, decomposing a first data export task into N second data export tasks may include operations S810 to S820.

[0105] In operation S810, a second preset threshold is set, and the second preset threshold is smaller than the first preset threshold.

[0106] In operation S820, starting from the starting position of the first data export task, a part of the first data export task is allocated to a second data export task every second preset threshold value to obtain N second data export tasks.

[0107] According to an embodiment of the present disclosure, for example, there are 100,000 bank account transaction details exported by a user, and the first preset threshold is 10,000, and the second preset threshold can be set to 5,000. Starting from the starting position of the above 100,000 data, a part of the 100,000 data is allocated to a small data volume export task (such as 0-5000, 5001-10000...) every 5,000 data, until the 100,000 data are disassembled, and 20 small data volume export tasks are obtained.

[0108] It should be noted that the first preset threshold and the second preset threshold can be flexibly set according to factors such as server processing performance or current data export function usage. For example, the second preset threshold is 7,000. When the 100,000 pieces of data are disassembled from the starting position to the end position, the last task is not 7,000. Then the 100,000th piece of data can be set as the disassembly end condition, that is, the last task includes 98,001 to 100,000 pieces of data. This disclosure does not impose specific restrictions on it.

[0109] Fig. 9 The flowchart of concurrently executing N second data export tasks using multiple threads according to an embodiment of the present disclosure is schematically shown.

[0110] like Fig. 9 As shown, according to an embodiment of the present disclosure, concurrently executing N second data export tasks using multiple threads may include operations S910 to S940.

[0111] In operation S910, a first data export file is determined. The first data export file is used to receive data written in N second data export tasks.

[0112] According to an embodiment of the present disclosure, for example, for 100,000 pieces of data exported by a user, a data export task with a small data volume is disassembled every 5,000 pieces of data. After obtaining 20 tasks with a small data volume, these 20 tasks are executed concurrently using multiple threads. First, determine whether the 100,000 pieces of data are exported for the first time, such as querying the export log, querying the user export task record, or scanning whether the export file already exists. Secondly, for example, operations S610 to S620 can be performed. If the 100,000 pieces of data are exported for the first time, a data export file is created. If the 100,000 pieces of data are not exported for the first time, the source export file created previously is found, and the source export file is used as the first data export file. Finally, the first data export file is determined based on the above judgment results.

[0113] In operation S920, a starting position for exporting data into the first data export file is obtained.

[0114] According to an embodiment of the present disclosure, if a user exports 100,000 data for the first time and creates a new export file, the starting position where the 100,000 data are written in the export file is first obtained. For example, if the export file format is Excel, the starting position is the first row of the first table in Excel.

[0115] According to another embodiment of the present disclosure, if the user is not exporting 100,000 pieces of data for the first time, the 100,000 pieces of data are appended to the existing source export file. In this case, the starting position of the appended writing in the source file needs to be obtained.

[0116] In operation S930, an export position in the first data export file is obtained every second preset threshold from the starting position until N-1 export positions are obtained.

[0117] In operation S940, the export data of the N second data export tasks are written into the first data export file based on the start position and the N-1 export positions.

[0118] For example, if a user needs to export 100,000 data items, and a data export task with a small amount of data is decomposed every 5,000 data items, and 20 tasks with a small amount of data are obtained, after determining the starting position for writing data, first obtain an export position every 5,000 data items from the starting position until 19 export positions are obtained. Among them, the starting position and the 19 export positions correspond to the 20 decomposed tasks one by one (for example, the positions corresponding to the 1st, 5001st, 10001st data items in the file...). Finally, the 20 data export tasks are written into the file in parallel according to the above starting position and 19 export positions.

[0119] According to an embodiment of the present disclosure, if the export file already exists and the source file is an empty file, the data is directly written to the file after obtaining the starting position and N-1 export positions. If some data already exists in the source file, after obtaining the starting position and N-1 export positions in the source file, the existing data can be overwritten, or the existing partial data can be compared with the exported data, and the existing data will no longer be exported. The present disclosure does not impose any restrictions on this, and it can be set according to actual conditions, and the final exported file includes all the data required by the user and does not contain duplicate or redundant data.

[0120] According to an embodiment of the present disclosure, after first determining the locations where the N second export tasks are written in the first data export task, the N second export tasks may be batch exported in parallel.

[0121] Fig.10The flowchart of concurrently executing N second data export tasks using multiple threads according to another embodiment of the present disclosure is schematically shown.

[0122] like Fig.10 As shown, according to another embodiment of the present disclosure, concurrently executing N second data export tasks using multiple threads may also include looping operations S1010 to S1030 until all the N second data export tasks are exported.

[0123] In operation S1010, a second data export file exported by the current first data export task is determined.

[0124] In operation S1020, if the available space of the second data export file is greater than or equal to the amount of exported data of the second data export task to be exported next, the second data export task is written into the second data export file.

[0125] In operation S1030, if the available space of the second data export file is smaller than the amount of export data of the second data export task to be exported next, a third data export file is created, and the second data export task is written into the third data export file.

[0126] According to an embodiment of the present disclosure, for example, 100,000 pieces of data are decomposed into 20 tasks with small data volumes, and the 20 tasks correspond to 1 to 5,000, 5,001 to 10,000, ... 95,001 to 100,000 pieces of data, respectively.

[0127] First, determine the export file for each task to be exported next. Specifically, for example, when executing the first task corresponding to 1 to 5,000 data, scan whether the export file already exists locally to determine whether the 100,000 data are exported for the first time. If there is no export file, it means it is the first export, then create an export file (hereinafter referred to as file A) and write the first task. When executing the second task corresponding to 5,001 to 10,000 data, scan whether the export file already exists locally. At this time, if file A created by the first task is scanned, file A will be used as the second data export file exported by the current first data export task. In addition, for example, when determining the export file for the last task corresponding to 95,001 to 100,000 data, if multiple export files are scanned, the last created export file can be determined as the second data export file based on the creation time of the export file.

[0128] Next, after the second data export file is determined, the amount of exported data of the next task to be exported is compared with the available space of the second data export file. For example, if the created export file A can accommodate 12,000 data, then the task containing 1 to 5,000 data is compared with the available space of file A. If it is found that file A can allow 5,000 data to be written, then the task containing 1 to 5,000 data is written into file A.

[0129] Then, file A can still accommodate 10,000 data items. When executing the task containing 5,001 to 10,000 data items, the amount of 5,000 data items is compared with the available space of file A. If it is determined that file A can allow 5,000 data items to be written, the data containing 5,001 to 10,000 data items will be written into file A.

[0130] Then, file A can still accommodate 2000 data items. When executing a task containing 10001 to 15000 data items, file A is scanned and used as the export file to be written next. The amount of 5000 data items is compared with the available space of file A. If it is determined that the available space of file A is less than the amount of 5000 export data items, a new export file B (i.e., the third data export file) is created, and the data containing 10001 to 15000 data items is written into the newly created export file B, wherein export file B can accommodate 12000 data items.

[0131] Then, the export file B can still accommodate 7,000 data items. When executing the task containing 15,001 to 20,000 data items, the files A and B are scanned, and the later created file B is determined as the next export file to be written based on the order of creation time. The amount of data containing 5,000 items is compared with the available space of file B, and then written to file B.

[0132] Finally, referring to the above steps, operations S1010 to S1030 are continuously executed in a loop for the remaining 16 tasks until all 20 data export tasks are completed.

[0133] It should be noted that the above-mentioned use of the number of data items as the quantification standard of the exported data size is only an embodiment of the present disclosure, and the present disclosure does not make specific restrictions on the quantification unit of the data volume and the format of the exported file.

[0134] According to another embodiment of the present disclosure, using multithreading to concurrently execute N second data export tasks may also include: writing each second data export task into a separate data export file, thereby obtaining N data export files. After the N second data export tasks are executed in parallel, the generated N data files are merged to obtain a file containing all the exported data.

[0135] Fig.11 A block diagram of a data export device applied to a second server according to an embodiment of the present disclosure is schematically shown.

[0136] like Fig.11 As shown, the data export device 1100 includes a task acquisition module 1110, a second determination module 1120 and a request sending module 1130. The data export device 1100 can be used to implement reference Figure 2 to Figure 4 The described data export method.

[0137] Specifically, the task acquisition module 1110 may, for example, execute operation S210 to acquire the third data export task.

[0138] The second determination module 1120 can, for example, perform operation S220, which is used to determine the target server corresponding to the third data export task from at least one first server, which includes: querying the target server corresponding to the third data export task based on the pre-stored correspondence between the at least one first server and the data service type.

[0139] The request sending module 1130 may, for example, perform operation S230 for issuing a call request so that the target server executes the third data export task.

[0140] According to an embodiment of the present disclosure, the data export device 1100 may further include a configuration table module, wherein the configuration table module may further perform operations S310 to S320, which will not be described in detail herein.

[0141] According to an embodiment of the present disclosure, the second determining module 1120 may, for example, further perform operation S410 and transmit the query result to the sending request module 1130 , and then the sending request module 1130 may perform operation S420 .

[0142] Fig.12 The application scenario diagram of the task acquisition module 1110 according to an embodiment of the present disclosure is schematically shown.

[0143] like Fig.12 As shown, the application scenario of the task acquisition module 1110 according to the embodiment of the present disclosure may include operations S1210 to S1250.

[0144] In operation S1210, after the user creates an export task, an asynchronous data export task is obtained.

[0145] In operation S1220, the parameters (such as data service type and data screening conditions, etc.) carried in the data export task created by the user are packaged and stored in the message queue.

[0146] In operation S1230 , the message queue includes a plurality of messages, each of which represents a data export task placed by the task acquisition module 1110 .

[0147] In operation S1240, a message is taken from the message queue.

[0148] In operation S1250, the message taken out from the message queue is processed, that is, the data export task. According to the data service configuration table, the export service method configured by the service personnel is obtained to realize data export.

[0149] According to an embodiment of the present disclosure, storing messages in or retrieving messages from a message queue may be performed by polling, timing, or random time methods, separating user creation of data export tasks and execution of data export to achieve asynchronous processing of data export tasks.

[0150] Fig.13 The flowchart of storing messages in a message queue at a fixed time according to an embodiment of the present disclosure is schematically shown.

[0151] like Fig.13 As shown, storing messages in the message queue at a fixed time may include operations S1310 to S1370, which may be executed by the task management server 107.

[0152] In operation S1310, referring to Figure 1 For example, the task management server 107 may be used to perform a timing operation and pre-set parameters of the timing operation, such as performing a timing operation every 5 seconds.

[0153] In operation S1320, each data export task in the task list is traversed periodically.

[0154] In operation S1330, it is determined whether the execution status of each data export task is in execution, if so, operation S1340 is executed, if not, operation S1360 is executed.

[0155] In operation S1340, the execution time of the data export task being executed is obtained to determine whether the execution has timed out. If so, operation S1350 is executed; if not, operation S1370 is executed.

[0156] In operation S1350, if the data export task execution times out, the execution status of the data export task is updated to failure. For a failed data export task, for example, the result can be fed back to the user who created the data export task, and the user can create it again or contact the business personnel.

[0157] In operation S1360, if the data export task has not been executed, the data export task is a task to be executed and is added to a message queue.

[0158] In operation S1370, the timing operation is ended. In addition, for example, it can be set to feedback the execution result after the timing operation is ended.

[0159] Fig.14 A block diagram of a data export device applied to a first server according to an embodiment of the present disclosure is schematically shown.

[0160] like Fig.14 As shown, the data export device 1400 includes a task acquisition module 1410, a data query module 1420, a first determination module 1430 and a file determination module 1440. The data export device 1400 can be used to implement reference Figures 5 to 10 The described data export method.

[0161] Specifically, the request receiving module 1410 may, for example, perform operation S510 to receive a call request for executing the first data export task.

[0162] The data query module 1420 may, for example, execute operation S520 to obtain the amount of exported data of the first data export task according to the information of the call request.

[0163] The first determining module 1430 may, for example, perform operation S530, for determining a data export method based on the amount of exported data of the first data export task, including, when the amount of exported data of the first data export task is greater than or equal to a first preset threshold, exporting data in a multi-threaded concurrent manner; when the amount of exported data of the first data export task is less than the first preset threshold, exporting data in a single thread;

[0164] The file determination module 1440 may, for example, perform operation S540 to determine at least one data export file to export the data into the at least one data export file.

[0165] Fig.15 The flowchart schematically shows the process of periodically taking out task messages from a message queue and executing them according to an embodiment of the present disclosure.

[0166] like Fig.15 As shown, storing messages in the message queue regularly may include operations S1501 to S1515. Operations S1501 to S1506 and S1515 may be performed by the task management server 107, and operations S1507 to S1514 may be performed by any one of the data service servers 108, 109 or 110.

[0167] In operation S1501 , for example, the task management server 107 is instructed to take out one or more data export tasks from the message queue every 5 seconds.

[0168] In operation S1502, a null value check is performed on each task message to determine whether the read task message contains information required for execution. If the message is a null value without any information, the task cannot be executed and ends. If it is not a null value, operation S1503 is executed.

[0169] In operation S1503, for example, multiple task messages are taken out from the message queue each time, and the multiple task messages are assigned to multiple threads to asynchronously execute the data export task.

[0170] In operation S1504, for the data export task started to be executed in operation S1503, the execution status is modified to being executed in the task list, and the execution time is updated.

[0171] In operation S1505, the interface information of the target server is obtained according to the retrieved data export task, which may refer to operation S410 and will not be described in detail here.

[0172] In operation S1506, a call request is issued according to the obtained interface information using a generalized call method, which may refer to operation S420.

[0173] According to the embodiments of the present disclosure, the specific business class information configured by the business personnel according to the data business server interface specification can be read, and then the data export task method can be generalized and called according to the specific business class information.

[0174] According to the embodiments of the present disclosure, for example, a business development module can be configured in the task management server, and all data business servers share the business development module. In the business development module, business developers who maintain data business servers can adapt to corresponding sub-services, and can create a sub-class containing a specific method for obtaining business data (such as interface information of the data business server) in the sub-service, so as to obtain data according to the specific business acquisition method.

[0175] In operation S1507, the data export task is executed, for example, the target data is searched according to the filtering conditions set by the user in the data export task.

[0176] In operation S1508, it is determined whether to execute the tasks concurrently according to the size of the exported data found and the preset export threshold, and the file of the exported data is generated according to the file generation method and the query result. If the tasks are executed concurrently, operation S1509 is executed, and if the tasks are not executed concurrently, operation S1511 is executed.

[0177] According to the embodiment of the present disclosure, if the query result is empty, an empty file is generated. For example, if the data that the user needs to export does not exist, after connecting to the data source, the data cannot be queried, and the query result is returned as empty, an empty file is generated for the user, and the data export is terminated, which is conducive to continuing the execution according to the pre-designed logic.

[0178] In operation S1509, the data export task is decomposed into a plurality of data export tasks with small data volumes.

[0179] In operation S1510, according to the disassembled data export tasks with a small data volume, the exported data is concurrently acquired from the starting position of the exported data.

[0180] In operation S1511, if it is not executed concurrently, the data export task is executed by a single thread, the target data is obtained from the data source in pages, and is written into the export file.

[0181] In operation S1512, since each file has a maximum download quantity limit, when the available space of the file is insufficient to support a data export task with a small data volume to completely export data, multiple files should be created.

[0182] According to an embodiment of the present disclosure, before each data export, it should be determined whether it is the first time to export data, so as to determine the export file. If it is the first time to export, a new file is created and the data is exported. If it is not the first time to export data, the data is directly appended to the source file. In addition, if the appended data exceeds the available space of the file, a new file is created.

[0183] In operation S1513, after all the data export files are successfully created, all the files are compressed and uploaded to a designated download location, and the download address is sent to the user so that the user can obtain the exported data.

[0184] In operation S1514, after the file is uploaded successfully, the local compressed file and the source file are deleted.

[0185] According to an embodiment of the present disclosure, for example, a data export file can be created locally, and after the data export is completed, the file is compressed and uploaded to the cloud server, so that the user can obtain data from the cloud server. After the file is successfully uploaded, the compressed file and the source file are deleted. And the task processing completion information is returned to the task management server 107.

[0186] In operation S1515, the task information is updated. For example, the successfully exported task can be deleted from the task list, or the execution status can be changed to success to avoid repeated processing.

[0187] By using the data export device of the embodiment of the present disclosure, unfinished and newly created tasks are added to the message queue asynchronously through timed scanning, and then the tasks in the queue are taken out through timed scanning, and the corresponding tasks are executed to realize the file download function. All data export services share a set of codes, which is convenient for centralized maintenance. At the same time, when facing the problem of exporting large data files, the data volume of the downloaded file is first predicted. If it exceeds the maximum number of single downloads, the large data download task can be disassembled into multiple small data export tasks according to the index position, and then multiple data files are exported in a concurrent manner to improve data export performance.

[0188] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0189] For example, the modules in the data export device 1100 or the data export device 1400 can be combined in one module, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the modules in the data export device 1100 or the data export device 1400 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the modules in the data export device 1100 or the data export device 1400 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0190] Fig.16 The block diagram of a computer system suitable for implementing the data export method according to an embodiment of the present disclosure is schematically shown. Fig.16 The computer system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0191] like Fig.16 As shown, the computer system 1600 according to the embodiment of the present disclosure includes a processor 1601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1602 or the program loaded from the storage part 1608 to the random access memory (RAM) 1603. The processor 1601 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1601 may also include an onboard memory for caching purposes. The processor 1601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0192] In RAM 1603, various programs and data required for the operation of system 1600 are stored. Processor 1601, ROM 1602, and RAM 1603 are connected to each other via bus 1604. Processor 1601 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1602 and / or RAM 1603. It should be noted that the program can also be stored in one or more memories other than ROM 1602 and RAM 1603. Processor 1601 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0193] According to an embodiment of the present disclosure, the system 1600 may further include an input / output (I / O) interface 1605, which is also connected to the bus 1604. The system 1600 may further include one or more of the following components connected to the I / O interface 1605: an input section 1606 including a keyboard, a mouse, etc.; an output section 1607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1608 including a hard disk, etc.; and a communication section 1609 including a network interface card such as a LAN card, a modem, etc. The communication section 1609 performs communication processing via a network such as the Internet. A drive 1610 is also connected to the I / O interface 1605 as needed. A removable medium 1611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1610 as needed, so that a computer program read therefrom is installed into the storage section 1608 as needed.

[0194] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains a program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the data export method provided by the embodiment of the present disclosure.

[0195] When the computer program is executed by the processor 1601, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module. In one embodiment, the computer program can rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program can also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 1609, and / or installed from a removable medium 1611. The program code contained in the computer program can be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0196] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0197] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0198] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus, or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1602 and / or RAM 1603 described above and / or one or more memories other than ROM 1602 and RAM 1603.

[0199] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0200] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0201] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A data export method, applied to a first server, wherein: include: receiving a call request for executing the first data export task from the second server; Obtaining the amount of exported data for the first data export task according to the information of the call request; Determining a data export method based on the amount of exported data of the first data export task, including: when the amount of exported data of the first data export task is greater than or equal to a first preset threshold, exporting data by concurrently using multiple threads; when the amount of exported data of the first data export task is less than the first preset threshold, exporting data by a single thread; determining at least one data export file to export the data into the at least one data export file; The second server is configured to perform the following operations: Obtain a third data export task; Determining a target server corresponding to the third data export task from the plurality of first servers, wherein: According to the pre-stored correspondence between the plurality of first servers and data service types, querying a target server corresponding to the third data export task, and querying interface information of the target server, wherein each first server has a different interface specification; issuing the call request based on the interface information of the target server so that the target server executes the third data export task, wherein the first data export task is the third data export task executed by the target server; The third data export task is pre-added to a message queue by the second server, and the second server is used to integrate and asynchronously process data export tasks created by different users. The obtaining of the third data export task includes: The third data export task is obtained from the message queue for asynchronous processing.

2. The data export method according to claim 1, wherein: The method of exporting data by multi-thread concurrently includes: Splitting the first data export task into N second data export tasks, wherein the amount of exported data of the second data export tasks is less than the first preset threshold; Multithreading is used to concurrently execute N of the second data export tasks, where N is an integer greater than or equal to 2.

3. The data export method according to claim 2, wherein: The step of decomposing the first data export task into N second data export tasks comprises: Setting a second preset threshold, wherein the second preset threshold is smaller than the first preset threshold; Starting from the starting position of the first data export task, a part of the first data export task is allocated to a second data export task every second preset threshold value to obtain N second data export tasks.

4. The data export method according to claim 2, wherein: The concurrently executing the N second data export tasks using multiple threads includes: determining a first data export file, wherein the at least one data export file includes the first data export file; Obtaining a starting position for exporting data into the first data export file; Obtaining an export position in the first data export file every second preset threshold from the starting position until N-1 export positions are obtained, and the second preset threshold is less than the first preset threshold; The export data of N second data export tasks are written into the first data export file based on the starting position and N-1 export positions.

5. The data export method according to claim 2, wherein: The concurrently executing the N second data export tasks by using multiple threads includes looping the following operations until all the N second data export tasks are exported: Determine a second data export file currently exported by the second data export task, wherein the at least one data export file includes the second data export file; If the available space of the second data export file is greater than or equal to the amount of export data of the second data export task to be exported next, write the second data export task into the second data export file; or If the available space of the second data export file is smaller than the amount of export data of the second data export task to be exported next, a third data export file is created and the second data export task is written into the third data export file, wherein the at least one data export file includes the third data export file.

6. The data export method according to claim 1, wherein determining at least one data export file comprises: If the first data export task exports data for the first time, creating a fourth data export file to export the data into the fourth data export file; or If the first data export task is not the first data export, the data is exported to a fifth data export file, wherein the fifth data export file is the file to which the first data export task last exported data.

7. A data export method, applied to a second server, comprising: Obtain a third data export task; Determining a target server corresponding to the third data export task from the plurality of first servers, wherein: According to the pre-stored correspondence between the plurality of first servers and data service types, querying a target server corresponding to the third data export task, and querying interface information of the target server, wherein each first server has a different interface specification; issuing a call request based on the interface information of the target server, so that the target server executes the third data export task; The third data export task is pre-added to a message queue by the second server, and the second server is used to integrate and asynchronously process data export tasks created by different users. The obtaining of the third data export task includes: The third data export task is obtained from the message queue for asynchronous processing.

8. The data export method according to claim 7, wherein the issuing of the call request comprises: querying a data service configuration table according to the data service type of the third data export task to obtain interface information of the target server, wherein the data service configuration table includes the interface information of the plurality of first servers and the pre-stored corresponding relationship; A call request is issued based on the interface information of the target server.

9. The data export method according to claim 8, further comprising before querying the data service configuration table: Setting interface information of the plurality of first servers in the data service configuration table; as well as The corresponding relationship between the plurality of first servers and data service types in the data service configuration table is set.

10. A data export device, applied to a first server, wherein: include: A request receiving module, used for receiving a call request for executing the first data export task from the second server; A data query module, used for obtaining the amount of exported data of the first data export task according to the information of the call request; A first determination module is used to determine a data export method based on the amount of exported data of the first data export task, which includes: when the amount of exported data of the first data export task is greater than or equal to a first preset threshold, using multi-threaded concurrent data export; when the amount of exported data of the first data export task is less than the first preset threshold, using a single thread to export data; a file determination module, configured to determine at least one data export file, so as to export data into the at least one data export file; The second server is configured to perform the following operations: Obtain a third data export task; Determining a target server corresponding to the third data export task from the plurality of first servers, wherein: According to the pre-stored correspondence between the plurality of first servers and data service types, querying a target server corresponding to the third data export task, and querying interface information of the target server, wherein each first server has a different interface specification; issuing the call request based on the interface information of the target server so that the target server executes the third data export task, wherein the first data export task is the third data export task executed by the target server; The third data export task is pre-added to a message queue by the second server, and the second server is used to integrate and asynchronously process data export tasks created by different users. The obtaining of the third data export task includes: The third data export task is obtained from the message queue for asynchronous processing.

11. A data export device, applied to a second server, comprising: An acquisition task module, used for acquiring a third data export task; The second determining module is used to determine a target server corresponding to the third data export task from the plurality of first servers, including: According to the pre-stored correspondence between the plurality of first servers and data service types, querying a target server corresponding to the third data export task, and querying interface information of the target server, wherein each first server has a different interface specification; A request sending module, used for sending a call request based on the interface information of the target server, so that the target server executes the third data export task; The third data export task is pre-added to a message queue by the second server, and the second server is used to integrate and asynchronously process data export tasks created by different users. The obtaining of the third data export task includes: The third data export task is obtained from the message queue for asynchronous processing.

12. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors execute the method according to any one of claims 1 to 6 or any one of claims 7 to 9.

13. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the method according to any one of claims 1 to 6 or any one of claims 7 to 9.

Images