File export method, device, electronic device and storage medium

Abstract

The present invention discloses a file export method, device, electronic device and storage medium. The method comprises: determining at least one data screening condition group in response to a condition entry operation; in response to a data export operation, in the case where at least one data screening condition group contains at least one execution screening condition group, determining the target export data corresponding to the execution screening condition group for each execution screening condition group; for at least one execution screening condition group, sequentially performing the operation of streaming the target export data into an initial table file based on the condition priority to obtain a streaming export file; determining the target data file corresponding to at least one data screening condition group based on the streaming export file. The technical solution of the present invention can avoid the problems of the processor being unable to load and the server temporarily overflowing with memory caused by the simultaneous export of a large amount of data (millions of data or tens of millions of data), and ensure the fluency of data export and file generation.

Description

Technical Field

[0001] The present invention relates to the field of computer application technology, and in particular to a file exporting method, device, electronic equipment and storage medium. Background Art

[0002] Current enterprise systems usually export large quantities of data and generate data files to expand related businesses such as data analysis, data sharing, data backup, and data migration.

[0003] Conventional methods of first reading a large amount of pre-exported data and then writing the read data into a file usually cause lags. The larger the amount of pre-exported data, the more obvious the lags are, and even the processor may be overloaded or the server may temporarily overflow memory, causing the file export task request to time out and file export to fail. Summary of the invention

[0004] The present invention provides a file export method, device, electronic device and storage medium to solve the technical problem that when exporting large amounts of data, a processor overload or a temporary server memory overflow usually occurs, resulting in a request timeout for the file export task and a file export failure.

[0005] According to one aspect of the present invention, a file export method is provided, the method comprising:

[0006] In response to a condition entry operation for a data screening item, determining at least one data screening condition group;

[0007] In response to a data export operation for at least one of the data filtering condition groups, in the case where at least one of the data filtering condition groups includes at least one execution filtering condition group, for each of the execution filtering condition groups, determining target export data corresponding to the execution filtering condition group, and generating an initial table file corresponding to the execution filtering condition group;

[0008] For at least one of the execution screening condition groups, sequentially performing the operation of streaming the target export data into the initial table file based on the condition priority, so as to obtain a streaming export file corresponding to each of the execution screening condition groups;

[0009] A target data file corresponding to at least one of the data screening condition groups is determined based on the streaming export file.

[0010] According to another aspect of the present invention, there is provided a file exporting device, the device comprising:

[0011] A condition entry module, configured to determine at least one data screening condition group in response to a condition entry operation for a data screening item;

[0012] a condition processing module, for responding to a data export operation for at least one of the data screening condition groups, and in the case that at least one of the data screening condition groups includes at least one execution screening condition group, for each of the execution screening condition groups, determining target export data corresponding to the execution screening condition group, and generating an initial table file corresponding to the execution screening condition group;

[0013] A data streaming export module, configured to sequentially perform the operation of streaming the target export data into the initial table file based on the condition priority for at least one of the execution screening condition groups, so as to obtain a streaming export file corresponding to each of the execution screening condition groups;

[0014] The file determination module is used to determine a target data file corresponding to at least one of the data screening condition groups based on the streaming export file.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the file export method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the file export method described in any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention is to determine at least one data screening condition group in response to the condition entry operation for the data screening entry; in response to the data export operation for at least one of the data screening condition groups, in the case where at least one of the data screening condition groups contains at least one execution screening condition group, for each of the execution screening condition groups, determine the target export data corresponding to the execution screening condition group, and generate the initial table file corresponding to the execution screening condition group; for at least one of the execution screening condition groups, sequentially perform the operation of streaming the target export data into the initial table file based on the condition priority to obtain the streaming export file corresponding to each of the execution screening condition groups; and determine the target data file corresponding to at least one of the data screening condition groups based on the streaming export file. The present invention realizes the data file export task of multiple data screening condition groups initiated by one key, that is, when a large amount of data (millions of data or tens of millions of data) export tasks are initiated at the same time, the sequential data export of multiple data screening condition groups and the data streaming export of each data screening condition group are combined at the same time, effectively avoiding the problems of processor overload and server temporary memory overflow that may occur when a large amount of data is exported at the same time, and ensuring the smoothness of task execution (that is, data export and file generation).

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flowchart of a file export method provided according to Embodiment 1 of the present invention;

[0024] Figure 2 is a flowchart of a file export method provided according to Embodiment 2 of the present invention;

[0025] Figure 3 is a timing diagram of a file export method provided according to an embodiment of the present invention;

[0026] Figure 4 is a structural schematic diagram of a file exporting device provided according to Embodiment 3 of the present invention;

[0027] Figure 5 It is a structural schematic diagram of an electronic device for implementing the file export method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment 1

[0031] Figure 1 A flowchart of a file export method is provided for the first embodiment of the present invention. This embodiment is applicable to the case where a large number of data export tasks are initiated at the same time. The method can be executed by a file export device. The file export device can be implemented in the form of hardware and / or software. The file export device can be configured in a computer. Figure 1 As shown, the method includes:

[0032] S110 . In response to a condition entry operation for a data screening item, determine at least one data screening condition group.

[0033] In the embodiment of the present invention, the data to be exported in the form of a file may be data in a target system. The target system may be preset according to scenario requirements and is not specifically limited here. Optionally, the target system may be a data management system of an enterprise.

[0034] The data filtering item can be understood as an item for filtering data in the target system, or can be understood as an item for retrieving data in the target system. In an embodiment of the present invention, the data filtering item can be preset according to the scenario requirements, and is not specifically limited here. Optionally, the data filtering item can include a time item, a data category item, and a data source item.

[0035] In an embodiment of the present invention, the data screening entry may include at least one data screening option. Exemplarily, the time entry may include a data screening option for year A, a data screening option for year B, and a data screening option for year C, etc. The data category entry may include a data screening option for an expenditure data category and a data screening option for an income data category. The data source entry may include a data screening option for cooperative enterprise A, a data screening option for cooperative enterprise B, and a data screening option for cooperative enterprise C, etc.

[0036] The condition entry operation can be understood as an operation of entering data screening conditions. Optionally, the condition entry operation can be a trigger operation for a data screening option in the data screening entry, or an operation of entering text related to data screening in the data screening entry.

[0037] The data screening condition can be understood as a condition for screening system data. In an embodiment of the present invention, a data screening condition group may include at least one data screening condition. Exemplarily, the data screening condition group may include a year screening condition A, an input data category condition, and a cooperative enterprise A screening condition. Alternatively, the data screening condition includes a year screening condition B.

[0038] S120. In response to a data export operation for at least one of the data filtering condition groups, in a case where at least one of the data filtering condition groups includes at least one execution filtering condition group, for each of the execution filtering condition groups, determining target export data corresponding to the execution filtering condition group, and generating an initial table file corresponding to the execution filtering condition group.

[0039] The execution filter condition group can be understood as a filter condition group for the data streaming export operation to be performed in at least one of the data filter condition groups. Correspondingly, the at least one data filter condition group can also include a filter condition group (i.e., the storage filter condition group) that does not need to perform the data streaming export operation. That is to say, in an embodiment of the present invention, at least one of the data filter condition groups can include at least one execution filter condition group and / or at least one storage filter condition group.

[0040] The data export operation may be an operation that triggers a data export control.

[0041] The target export data can be understood as data in the target system that meets the screening conditions of the execution screening condition group, and can also be understood as data to be stream-exported. In an embodiment of the present invention, the target export data can be millions or tens of millions of data.

[0042] The initial table file can be understood as a table file initialized for the execution screening condition group. Exemplarily, the initial table file can be an Excel file.

[0043] S130. For at least one of the execution filter condition groups, sequentially perform the operation of streaming the target export data into the initial table file based on the condition priority, so as to obtain a streaming export file corresponding to each of the execution filter condition groups.

[0044] Among them, the condition priority can be understood as the priority of the execution filtering condition group. In an embodiment of the present invention, the determination of the condition priority can be preset according to the scenario requirements and is not specifically limited here. Optionally, the condition priority is determined based on the condition data contained in the execution filtering condition group, or the condition priority is determined based on the number of data items of the target export data corresponding to the execution filtering condition group, etc.

[0045] In the implementation of the present invention, the data streaming export for the plurality of execution filter condition groups is executed sequentially, i.e., serially. That is, after the data streaming export for one execution filter condition group is completed and the streaming export file is obtained, the data streaming export for another execution filter condition group is performed. This is to avoid the situation where the processor cannot be loaded due to the simultaneous export of a large amount of data.

[0046] Optionally, the step of streaming the target export data into the initial table file comprises:

[0047] Traversing the target export data, extracting target field data of target field length in the target export data, and writing the target field data into a temporary field list;

[0048] When the target field data in the temporary field list reaches a preset field length threshold, the target field data in the temporary field list is written into the initial table file until the data traversal of the target export data is completed.

[0049] Optionally, after writing the target field data in the temporary field list into the initial table file, the method further includes:

[0050] The target field data in the temporary field list is deleted.

[0051] The target field length can be understood as the field length of the field data extracted each time. In the embodiment of the present invention, the target field length can be preset according to the scenario requirements and is not specifically limited here. Correspondingly, the target field data can be understood as the field data of the target field length, that is, the field data extracted from the target export data each time.

[0052] The temporary field list can be understood as an intermediate list for writing field data into a table file. The temporary field list can temporarily cache field data of a field length threshold.

[0053] The field length threshold may be understood as a field length threshold. In the embodiment of the present invention, the field length threshold may be preset according to scenario requirements and is not specifically limited here. Optionally, the field length threshold is greater than or equal to the target field length.

[0054] Based on the above implementation scheme, streaming export of a large amount of data in the system is achieved, avoiding problems such as processor overload and temporary server memory overflow that may occur when a large amount of data is exported at the same time, and ensuring the smoothness of task execution when a large number of data export tasks are initiated at the same time.

[0055] Optionally, after writing the target field data in the temporary field list into the initial table file, the method further includes:

[0056] Save real-time table files in case of network interruption;

[0057] After saving the real-time table file, the method further includes:

[0058] When the network is restored, the target export data is continued to be traversed, and the target field data in the temporary field list is written into the real-time table file.

[0059] The real-time table file may be understood as a table file saved in real time.

[0060] Based on the above-mentioned embodiment scheme, the breakpoint-resume function of data streaming export is realized, avoiding the need to re-traverse data and stream export after network interruption.

[0061] S140. Determine a target data file corresponding to at least one of the data screening condition groups based on the streaming export file.

[0062] The technical solution of the embodiment of the present invention is to determine at least one data screening condition group in response to the condition entry operation for the data screening entry; in response to the data export operation for at least one of the data screening condition groups, in the case where at least one of the data screening condition groups contains at least one execution screening condition group, for each of the execution screening condition groups, determine the target export data corresponding to the execution screening condition group, and generate the initial table file corresponding to the execution screening condition group; for at least one of the execution screening condition groups, sequentially perform the operation of streaming the target export data into the initial table file based on the condition priority to obtain the streaming export file corresponding to each of the execution screening condition groups; and determine the target data file corresponding to at least one of the data screening condition groups based on the streaming export file. The present invention realizes the data file export task of multiple data screening condition groups initiated by one key, that is, when a large amount of data (millions of data or tens of millions of data) export tasks are initiated at the same time, the sequential data export of multiple data screening condition groups and the data streaming export of each data screening condition group are combined at the same time, effectively avoiding the problems of processor overload and server temporary memory overflow that may occur when a large amount of data is exported at the same time, and ensuring the smoothness of task execution (that is, data export and file generation).

[0063] Embodiment 2

[0064] Figure 2 This is a flowchart of a file export method provided in Embodiment 2 of the present invention. This embodiment is directed to the above embodiment, in which the data export operation in response to at least one set of data screening conditions is added. Figure 2 As shown, the method includes:

[0065] S210: In response to a condition entry operation for a data screening item, determine at least one data screening condition group.

[0066] S220: In response to a data export operation for at least one of the data screening condition groups, determine a condition path record library.

[0067] The condition path record library may include a plurality of record screening condition groups and a file storage path corresponding to each screening condition record group.

[0068] The record filtering condition group may be understood as a filtering condition group that has been historically executed for data streaming export.

[0069] The record screening condition group and the file storage path may correspond one to one.

[0070] The file storage path can be understood as the storage path of the historical export file. In the implementation of the present invention, one file storage path can correspond to one historical export file.

[0071] The condition record library includes a plurality of record screening condition groups and a file storage path corresponding to each screening condition record group.

[0072] S230: For each of the data screening condition groups, compare the data screening condition group with the record screening condition group to obtain a comparison result.

[0073] Specifically, it is determined whether the filtering conditions in the data filtering condition group and the record filtering condition group are consistent, and the determination result is used as the comparison result.

[0074] S240: Determine an execution filtering condition group and / or a storage filtering condition group included in the data filtering condition group according to the comparison result.

[0075] Specifically, for each of the data filtering condition groups, if there is a record filtering condition group consistent with the data filtering condition group in the condition path record library, the data filtering condition group is used as the storage filtering condition group; otherwise, it is used as the execution filtering condition group.

[0076] Optionally, after the data export operation in response to at least one of the data screening condition groups, the method further includes:

[0077] In the case where at least one of the data screening condition groups includes at least one of the storage screening condition groups, for each of the storage screening condition groups, determining a target storage path corresponding to the storage screening condition group through the condition path record library;

[0078] For at least one of the storage screening condition groups, a data storage file corresponding to the storage screening condition group is determined according to the target storage path.

[0079] The target storage path may be understood as a file storage path corresponding to the screening condition record group that is consistent with the storage screening condition group in the condition path record library.

[0080] Specifically, determining the data storage file corresponding to the storage screening condition group according to the target storage path includes:

[0081] Download the storage file under the target storage path to obtain the data storage file.

[0082] Based on the above embodiment scheme, the reuse of the export files of the consistent data screening condition group is achieved.

[0083] Optionally, after obtaining the streaming export file corresponding to each of the execution screening condition groups, the method further includes:

[0084] For each of the execution screening condition groups, storing the streaming export file corresponding to the execution screening condition group, and determining a file storage path of the streaming export file;

[0085] The execution screening condition group and the file storage path corresponding to the execution screening condition group are stored in the condition path record library.

[0086] The file storage path may be understood as the storage path of the streaming export file.

[0087] Based on the above embodiment scheme, the storage path of the streaming export file is recorded to further realize the multiplexing of the stored streaming export file.

[0088] S250. In the case where at least one of the data filtering condition groups includes at least one execution filtering condition group, for each of the execution filtering condition groups, determining the target export data corresponding to the execution filtering condition group, and generating an initial table file corresponding to the execution filtering condition group.

[0089] S260. For at least one of the execution filter condition groups, sequentially perform the operation of streaming the target export data into the initial table file based on the condition priority, so as to obtain a streaming export file corresponding to each of the execution filter condition groups.

[0090] S270. Determine a target data file corresponding to at least one of the data screening condition groups based on the streaming export file.

[0091] Optionally, determining, based on the streaming export file, a target data file corresponding to at least one of the data screening condition groups includes:

[0092] Determine a target data file corresponding to each of the data screening condition groups according to the streaming export file and the data storage file;

[0093] After determining the target data file corresponding to each data screening condition group according to the streaming export file and the data storage file, the method further includes:

[0094] The target data file is compressed to obtain a target compressed file corresponding to the data screening condition group.

[0095] The target data file can be understood as an exported target file, and the target compressed file can be understood as an exported compressed file.

[0096] Based on the above-mentioned embodiment scheme, the compressed export of files is realized, which avoids the occurrence of network request timeout when exporting large files, improves the file export efficiency and reduces the space occupied.

[0097] The technical solution of the embodiment of the present invention is to determine a condition path record library, wherein the condition record library includes multiple record filter condition groups and the file storage path corresponding to each filter condition record group; for each data filter condition group, the data filter condition group and the record filter condition group are compared to obtain a comparison result; and the execution filter condition group and / or storage filter condition group contained in the data filter condition group are determined according to the comparison result. The present invention realizes the classification of the filter condition group, so as to determine the data export file corresponding to the filter condition group in a targeted manner for different filter condition groups.

[0098] Figure 3 FIG. 1 is a timing diagram of a file export method provided according to an embodiment of the present invention. Figure 3 As shown, the file export method is further explained as follows:

[0099] 1. File compression. The script developed by the present invention provides file compression function to improve data transmission efficiency and storage efficiency and can ensure the security of the compression process and the integrity of the data. Common compression formats are supported, such as zip and gzip.

[0100] Specifically, a compression option is provided when exporting data, and compression processing of batch files is supported.

[0101] 2. Streaming export: The present invention exports data in the form of streams, which can reduce memory consumption.

[0102] Specifically, data is read and written step by step. It supports breakpoint resume function to ensure the stability of the export process.

[0103] 3. Export cache reuse: In the scenario where the same data is exported multiple times, the present invention reduces data processing time by exporting cache reuse.

[0104] Specifically, a cache file is generated during the first export; the cache file is directly read during subsequent exports, skipping repeated processing steps. The present invention can also provide a cache invalidation strategy to regularly clean up the cache file to ensure the timeliness of the stored files.

[0105] The technical solution of the present invention can realize file export of large amounts of data with low latency, high real-time performance, dynamic data export, low memory usage and efficient bandwidth utilization.

[0106] Embodiment 3

[0107] Figure 4This is a schematic diagram of the structure of a file export device provided in Embodiment 3 of the present invention. Figure 4 As shown, the device includes: a condition entry module 310, a condition processing module 320, a data streaming export module 330 and a file determination module 340.

[0108] Among them, the condition entry module 310 is used to determine at least one data filtering condition group in response to the condition entry operation for the data filtering entry; the condition processing module 320 is used to respond to the data export operation for at least one of the data filtering condition groups, and when at least one of the data filtering condition groups contains at least one execution filtering condition group, for each of the execution filtering condition groups, determine the target export data corresponding to the execution filtering condition group, and generate an initial table file corresponding to the execution filtering condition group; the data streaming export module 330 is used to perform the operation of streaming the target export data into the initial table file in sequence based on the condition priority for at least one of the execution filtering condition groups, so as to obtain the streaming export file corresponding to each of the execution filtering condition groups; the file determination module 340 is used to determine the target data file corresponding to at least one of the data filtering condition groups based on the streaming export file.

[0109] The technical solution of the embodiment of the present invention is to determine at least one data screening condition group in response to the condition entry operation for the data screening entry; in response to the data export operation for at least one of the data screening condition groups, in the case where at least one of the data screening condition groups contains at least one execution screening condition group, for each of the execution screening condition groups, determine the target export data corresponding to the execution screening condition group, and generate the initial table file corresponding to the execution screening condition group; for at least one of the execution screening condition groups, sequentially perform the operation of streaming the target export data into the initial table file based on the condition priority to obtain the streaming export file corresponding to each of the execution screening condition groups; and determine the target data file corresponding to at least one of the data screening condition groups based on the streaming export file. The present invention realizes the data file export task of multiple data screening condition groups initiated by one key, that is, when a large amount of data (millions of data or tens of millions of data) export tasks are initiated at the same time, the sequential data export of multiple data screening condition groups and the data streaming export of each data screening condition group are combined at the same time, effectively avoiding the problems of processor overload and server temporary memory overflow that may occur when a large amount of data is exported at the same time, and ensuring the smoothness of task execution (that is, data export and file generation).

[0110] Optionally, the condition processing module 320 further includes: a record library determination unit, a condition comparison unit, and a condition classification unit;

[0111] The record library determination unit is used to determine a condition path record library after the data export operation in response to at least one of the data screening condition groups, wherein the condition record library includes a plurality of record screening condition groups and a file storage path corresponding to each of the screening condition record groups;

[0112] The condition comparison unit is used to compare the data screening condition group with the record screening condition group for each data screening condition group to obtain a comparison result;

[0113] The condition classification unit is used to determine the execution filtering condition group and / or the storage filtering condition group included in the data filtering condition group according to the comparison result.

[0114] Optionally, the condition processing module 320 further includes: a path retrieval unit and a file retrieval unit;

[0115] The path retrieval unit is configured to determine, after the data export operation in response to at least one of the data filtering condition groups, a target storage path corresponding to the storage filtering condition group through the condition path record library for each of the storage filtering condition groups, when at least one of the data filtering condition groups includes at least one of the storage filtering condition groups;

[0116] The file retrieval unit is used to determine, for at least one of the storage screening condition groups, a data storage file corresponding to the storage screening condition group according to the target storage path.

[0117] Optionally, the file export device further includes: a file storage module and a path storage module;

[0118] The file storage module is used for storing the streaming export file corresponding to each execution screening condition group for each execution screening condition group after obtaining the streaming export file corresponding to each execution screening condition group, and determining the file storage path of the streaming export file;

[0119] The path storage module is used to store the execution screening condition group and the file storage path corresponding to the execution screening condition group in the condition path record library.

[0120] Optionally, the file determination module 340 is specifically configured to:

[0121] Determine a target data file corresponding to each of the data screening condition groups according to the streaming export file and the data storage file;

[0122] The file exporting device further includes:

[0123] The file compression module is used to compress the target data file after determining the target data file corresponding to each data screening condition group according to the streaming export file and the data storage file, so as to obtain the target compressed file corresponding to the data screening condition group.

[0124] Optionally, the data streaming export module 330 includes: a field extraction unit and a data writing unit;

[0125] Wherein, the field extraction unit is used to traverse the target export data, extract the target field data of the target field length in the target export data, and write the target field data into a temporary field list;

[0126] The data writing unit is used to write the target field data in the temporary field list into the initial table file when the target field data in the temporary field list reaches a preset field length threshold until the data traversal of the target export data is completed.

[0127] Optionally, the data writing unit includes:

[0128] The network interruption subunit is used for saving the real-time table file in case of network interruption after writing the target field data in the temporary field list into the initial table file.

[0129] The data writing unit further includes:

[0130] The network recovery subunit is used for, after the real-time table file is saved, continuing to traverse the target export data in the case of network recovery, and writing the target field data in the temporary field list into the real-time table file.

[0131] The file exporting device provided in the embodiment of the present invention can execute the file exporting method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0132] Embodiment 4

[0133] Figure 5A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0134] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0135] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0136] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a file export method.

[0137] In some embodiments, the file export method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the file export method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the file export method in any other appropriate manner (e.g., by means of firmware).

[0138] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0140] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0142] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0143] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0144] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0145] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A file export method, characterized in that: include: In response to a condition entry operation for a data screening item, determining at least one data screening condition group; In response to a data export operation for at least one of the data filtering condition groups, in the case where at least one of the data filtering condition groups includes at least one execution filtering condition group, for each of the execution filtering condition groups, determining target export data corresponding to the execution filtering condition group, and generating an initial table file corresponding to the execution filtering condition group; For at least one of the execution screening condition groups, sequentially performing the operation of streaming the target export data into the initial table file based on the condition priority, so as to obtain a streaming export file corresponding to each of the execution screening condition groups; A target data file corresponding to at least one of the data screening condition groups is determined based on the streaming export file.

2. The method according to claim 1, characterized in that After the data export operation in response to at least one of the data screening condition groups, the method further includes: Determine a condition path record library, wherein the condition record library includes a plurality of record screening condition groups and a file storage path corresponding to each screening condition record group; For each of the data screening condition groups, the data screening condition group and the record screening condition group are compared to obtain a comparison result; An execution filtering condition group and / or a storage filtering condition group included in the data filtering condition group is determined according to the comparison result.

3. The method according to claim 2, characterized in that After the data export operation in response to at least one of the data screening condition groups, the method further includes: In the case where at least one of the data screening condition groups includes at least one of the storage screening condition groups, for each of the storage screening condition groups, determining a target storage path corresponding to the storage screening condition group through the condition path record library; For at least one of the storage screening condition groups, a data storage file corresponding to the storage screening condition group is determined according to the target storage path.

4. The method according to claim 3, characterized in that After obtaining the streaming export file corresponding to each of the execution screening condition groups, the method further includes: For each of the execution screening condition groups, storing the streaming export file corresponding to the execution screening condition group, and determining a file storage path of the streaming export file; The execution screening condition group and the file storage path corresponding to the execution screening condition group are stored in the condition path record library.

5. The method according to claim 3, characterized in that: Determining a target data file corresponding to at least one of the data screening condition groups based on the streaming export file includes: Determine a target data file corresponding to each of the data screening condition groups according to the streaming export file and the data storage file; After determining the target data file corresponding to each data screening condition group according to the streaming export file and the data storage file, the method further includes: The target data file is compressed to obtain a target compressed file corresponding to the data screening condition group.

6. The method according to claim 1, characterized in that The step of streaming the target export data into the initial table file comprises: Traversing the target export data, extracting target field data of target field length in the target export data, and writing the target field data into a temporary field list; When the target field data in the temporary field list reaches a preset field length threshold, the target field data in the temporary field list is written into the initial table file until the data traversal of the target export data is completed.

7. The method according to claim 6, characterized in that After writing the target field data in the temporary field list into the initial table file, the method further includes: Save real-time table files in case of network interruption; After saving the real-time table file, the method further includes: When the network is restored, the target export data is continued to be traversed, and the target field data in the temporary field list is written into the real-time table file.

8. A file export device, characterized in that: include: A condition entry module, configured to determine at least one data screening condition group in response to a condition entry operation for a data screening item; a condition processing module, for responding to a data export operation for at least one of the data screening condition groups, and in the case that at least one of the data screening condition groups includes at least one execution screening condition group, for each of the execution screening condition groups, determining target export data corresponding to the execution screening condition group, and generating an initial table file corresponding to the execution screening condition group; A data streaming export module, configured to sequentially perform the operation of streaming the target export data into the initial table file based on the condition priority for at least one of the execution screening condition groups, so as to obtain a streaming export file corresponding to each of the execution screening condition groups; The file determination module is used to determine a target data file corresponding to at least one of the data screening condition groups based on the streaming export file.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the file export method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the file export method according to any one of claims 1 to 7 when executed.

Images