File exporting method and device, computer device and storage medium
By registering data query statements and field information to the task table, reasonably setting the processing volume of data query tasks, and using a thread pool to execute tasks, the stability and efficiency issues of the server in a high-concurrency and large-scale file export environment are resolved, and stable and effective file output of the server is achieved.
Patent Information
- Application Number
- CN202310534444.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2025-12-23
- Estimated Expiration
- 2043-05-12
AI Technical Summary
In environments with high concurrency and large numbers of file exports, the server is under tremendous pressure, causing file exports to slow down or crash.
By registering data query statements and selected field information into a task table, the server processes a preset number of data query tasks each time. The processing volume of data query tasks is reasonably set by using a task table, and a thread pool is used to execute data query tasks.
Under high concurrency and big data conditions, the server's stability and file output efficiency were guaranteed.
Smart Images

Figure CN116644096B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of big data processing, and in particular to a file export method and device, computer equipment, a storage medium and a computer program product. BACKGROUND
[0002] In various application scenarios, users want to export files from the Internet, and due to different user needs, the content and format of the exported files are different. In a multi-concurrent and large-quantity file export environment, great pressure is caused to the server, resulting in slow file export and even server memory overflow and collapse. SUMMARY
[0003] Therefore, it is necessary to provide a file export method and device, computer equipment, a storage medium and a computer program product to solve the above technical problems.
[0004] In a first aspect, the present application provides a file export method applied to a client, comprising:
[0005] In response to an export operation on a current data query page, an information selection page matching the current data query page is displayed;
[0006] In response to a field information selection operation on the information selection page, selected field information and a data query condition corresponding to the data query page are obtained;
[0007] The data query condition is parsed to obtain a data query statement;
[0008] The data query statement and the field information are registered to a task table; wherein the task table is used to instruct a server to obtain a preset number of data query statements from the task table, execute corresponding data query tasks, and generate a file according to the query result of the data query task and the field information, and return the file to the client.
[0009] In a possible implementation, before the response to the export operation on the data query page, the information selection page matching the data query page is displayed, the method further comprises:
[0010] A preset script file is loaded and installed; wherein the script file has a script program for obtaining a data query condition of a data query page and generating field information corresponding to an information selection page according to field information of the data query page.
[0011] In a possible implementation, the current data query page has identification information, and the parsing of the data query condition to obtain a data query statement comprises:
[0012] obtain a plurality of data query statements matching the identification information;
[0013] select a data query statement matching the data query condition from the plurality of data query statements.
[0014] In a possible implementation, the data query statement includes an initial data query statement with a placeholder and a query parameter with a parameter position and a parameter type; and the obtaining of the plurality of data query statements matching the identification information includes:
[0015] obtain a plurality of objects matching the identification information;
[0016] The selecting of the data query statement matching the data query condition from the plurality of data query statements includes:
[0017] select a target object matching the data query condition from the plurality of objects;
[0018] In a case where the target object is of a first preset type, obtain a preset class function of the target object;
[0019] determine a query parameter corresponding to the target object based on the class function;
[0020] input the target object into an extraction function of an initial data query statement, and output the initial data query statement through the extraction function.
[0021] In a possible implementation, the selecting of the target object matching the data query condition from the plurality of objects further includes
[0022] In a case where the target object is of a second preset type, generate a data query statement according to the target object in a generation manner matching the second preset type.
[0023] In a possible implementation, the data query statement includes an initial data query statement with a placeholder and a query parameter with a parameter position and a parameter type, the server is preset with a thread pool of data query tasks, the server obtains a preset number of data query statements from the task table at a timing, and executes a corresponding data query task, including:
[0024] In a case where an export event is listened to, the thread pool is configured to obtain a preset number of data pairs from the task table, the data pair including an initial data query statement with a placeholder and a query parameter with a parameter position and a parameter type;
[0025] The thread pool is configured to obtain a function object corresponding to the initial data query statement with the placeholder.
[0026] The thread pool is further configured to obtain a data query statement according to the function object and the query parameter with the parameter bit and the parameter type, and execute a corresponding data query task.
[0027] In a possible implementation, the file includes a CSV file, and the server executes the corresponding data query task by performing the following operations:
[0028] The server connects a database through a JDBC driver and reads data in batches, and takes the read data as a query result corresponding to the data query task.
[0029] In a second aspect, the present application provides a file export apparatus, which is characterized in that the apparatus includes:
[0030] A display module is configured to display an information selection page matched with a current data query page in response to an export operation of the current data query page.
[0031] An acquisition module is configured to acquire selected field information and a data query condition corresponding to the data query page in response to a field information selection operation of the information selection page.
[0032] A parsing module is configured to parse the data query condition to obtain a data query statement.
[0033] A generation module is configured to register the data query statement and the field information to a task table, wherein the task table is configured to instruct a server to acquire a preset number of data query statements from the task table, execute corresponding data query tasks, generate a file according to a query result of the data query task and the field information, and return the file to the client.
[0034] In a possible implementation, the apparatus further includes:
[0035] A loading module is configured to load and install a preset script file, wherein the script file is pre-written with a script program for acquiring a data query condition of a data query page and generating field information of an information selection page corresponding to the data query page according to field information of the data query page.
[0036] In a possible implementation, the current data query page corresponds to identification information, and the parsing module includes:
[0037] A first acquisition submodule is configured to acquire a plurality of data query statements matched with the identification information.
[0038] A selection submodule is configured to select a data query statement matched with the data query condition from the plurality of data query statements.
[0039] In a possible implementation, the data query statement includes an initial data query statement with a placeholder and query parameters with parameter positions and parameter types, and the obtaining submodule includes:
[0040] The first obtaining unit is configured to obtain a plurality of objects matched with the identification information.
[0041] The selection submodule includes:
[0042] The selection unit is configured to select a target object matched with the data query condition from the plurality of objects.
[0043] The second obtaining unit is configured to obtain a preset class function of the target object in a case where the target object is of a first preset type.
[0044] The determination unit is configured to determine a query parameter corresponding to the target object based on the class function.
[0045] The output unit is configured to input the target object into an extraction function of the initial data query statement, and output an initial data query statement obtained through the extraction function.
[0046] In a possible implementation, the selection submodule further includes:
[0047] The third obtaining unit is configured to generate a data query statement according to the target object in a case where the target object is of a second preset type and in a generation manner matched with the second preset type.
[0048] In a possible implementation, the generation module includes:
[0049] The second obtaining submodule is configured to, in a case where an export event is listened to, obtain a preset number of data pairs from the task table by the thread pool, the data pair including an initial data query statement with a placeholder and query parameters with parameter positions and parameter types.
[0050] The third obtaining submodule is configured to obtain a function object corresponding to the initial data query statement with the placeholder by the thread pool.
[0051] The execution submodule is configured to obtain a data query statement according to the function object and the query parameters with the parameter positions and the parameter types by the thread pool, and execute a corresponding data query task.
[0052] In a possible implementation, the file includes a CSV file, and the generation module includes:
[0053] The reading submodule is configured to connect a database through a JDBC driver and read data in batches, and the read data is taken as a query result corresponding to the data query task.
[0054] In a third aspect, the present application provides a computer device. The computer device comprises a memory and a processor. The memory stores a computer program. The processor implements the steps of the file export method according to any one of the embodiments of the present application when executing the computer program.
[0055] In a fourth aspect, the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the file export method according to any one of the embodiments of the present application.
[0056] In a fifth aspect, the present application provides a computer program product. The computer program product comprises a computer program. The computer program is executed by a processor to implement the steps of the file export method according to any one of the embodiments of the present application.
[0057] The file export method, the file export device, the computer device, the storage medium and the computer program product have the advantages that the data query statement and the selected field information are registered in the task table, the server processes a preset number of data query tasks each time, compared with the traditional method of directly sending the data query statement to the server for processing, the present application reasonably sets the processing amount of the data query task through the task table, and in the case of high concurrency and large data, the stable and effective file output of the server is ensured, and the efficiency of the file data is improved. BRIEF DESCRIPTION OF DRAWINGS
[0058] Figure 1 FIG. 1 is a diagram of an application environment of the file export method according to an embodiment;
[0059] Figure 2 FIG. 2 is a first flowchart of the file export method according to an embodiment;
[0060] Figure 3 FIG. 3 is a diagram of an information selection page according to an embodiment;
[0061] Figure 4 FIG. 4 is a second flowchart of the file export method according to another embodiment;
[0062] Figure 5 FIG. 5 is an architecture diagram of a server end of the file export method according to an embodiment;
[0063] Figure 6 FIG. 6 is a structural block diagram of the file export device according to an embodiment;
[0064] Figure 7Fig. 1 is a diagram of an internal structure of a computer device in one embodiment.
[0065] Figure 8 Fig. 1 is a diagram of an internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0066] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.
[0067] The file export method provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 . The terminal 102 communicates with the server 104 through a network. The data storage system can store data required to be processed by the server 104. The data storage system can be integrated on the server 104, or placed on a cloud or other network server. The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.
[0068] In one embodiment, as shown in Figure 2 , a file export method is provided. Taking the terminal in Figure 1 as an example, the method includes the following steps:
[0069] Step S201, in response to an export operation on a current data query page, an information selection page matching the current data query page is displayed.
[0070] Specifically, different business scenarios correspond to different data query pages, such as a product information query page, an activity information query page, etc. In an example embodiment, the data query page can set a query condition, such as products with a price from interval A to interval B, and again, such as users with a user level from three stars to five stars. In an example embodiment, the data query page and the information selection page can be set to have a preset corresponding relationship, and the information selection page can be uniquely determined by the displayed data query page. In another example embodiment, the information selection page can also be generated by the data query page, that is, the current data query page is subjected to information extraction processing to obtain the information selection page.
[0071] In the embodiments of the present disclosure, some query controls can be displayed on the current data query page, and after the query control is triggered, an information selection page matched with the current data query page is displayed. In an example embodiment, as shown in FIG. 2, the information selection page can include selection of export fields, application permissions, subsidiary permissions, and application reasons, etc. The content information of the information selection page is matched with the data query page. It should be noted that the export fields of the above information selection page can include other custom fields, and those skilled in the art can also make other changes under the inspiration of the technical essence of the present application, as long as the functions and effects achieved are the same or similar to the present application, which should be covered within the protection scope of the present application. Figure 3
[0072] In step S203, in response to the field information selection operation on the information selection page, the selected field information and the data query condition corresponding to the data query page are obtained.
[0073] Specifically, the specific content of the field information can be set according to the specific application scenario, which is related to the data in the data query page. The field information is used to distinguish the type of data in the exported file. In an example embodiment, the data query condition in the data query page can be automatically captured. In the embodiments of the present disclosure, other information such as application permissions, application reasons, data use expiration date, etc. can also be included in the information selection page, which is used for subsequent permission verification and data date filtering. The present disclosure does not limit the specific content of other information.
[0074] In step S205, the data query condition is parsed to obtain a data query statement.
[0075] Specifically, the data query statement can include different types of data query statements according to the type of database, such as SQL (Structured Query Language) query statement, PL / SQL query statement, T-SQL query statement, MongoDB query statement, Redis query statement, Cassandra query statement, etc. The data query statement obtained by parsing the data query condition can include a complete data query statement, which can be directly used for database access; or it can include part of the data query statement, and when it only contains part of the data query statement, the entire data query statement can be restored through subsequent processing.
[0076] In step S207, the data query statement and the field information are registered to a task table. The task table is used to indicate the server to obtain a preset number of data query statements from the task table, execute the corresponding data query task, generate a file according to the query result of the data query task and the field information, and return the file to the client.
[0077] Specifically, the data query statement and the field information are registered in the task table, and registration means recording. In an example embodiment, in order to distinguish each data query task, identification information of the data query task can also be set. The identification information can be generated when registration, or can be set according to the function identification corresponding to the data query page. In an example embodiment, each data query task in the task table can include a processing state of the task, such as unprocessed, processing, processed, etc. The newly registered data query task can be marked as an unprocessed state.
[0078] In an example embodiment, the server side can set a timing task, detect the data query task in the unprocessed state in the task table in a timing manner, and read a preset number of data query statements to execute the corresponding data query task. In another example embodiment, a task discovery thread can also be set. The task discovery thread is used to retrieve the to-be-processed task in the task table and export a task publishing event. In order to support distributed deployment of services, a redis can be introduced to realize a distributed lock, and a task unique ID is used as a key. In the embodiment of the present disclosure, the preset number can be set according to the processing capability of the server and the type of the data query task. In an example embodiment, the preset number can be selected from multiple setting values in a trial operation manner. In another example embodiment, when the data query task is configured through a thread pool, the task concurrency can be set to realize the effect of peak shaving and valley filling.
[0079] In the above file export method, by registering the data query statement and the selected field information in the task table, the server processes a preset number of data query tasks each time. Compared with the traditional method of directly sending the data query statement to the server for processing, in the embodiment of the present disclosure, the processing amount of the data query task is reasonably set in an offline manner through the task table. In the case of high concurrency and large data, the stable and effective file output of the server is ensured, and the efficiency of the file data is improved.
[0080] Reference Figure 4 As shown in the figure, in a possible implementation manner, before the response to the export operation of the data query page, the information selection page matched with the data query page is displayed, the method further includes:
[0081] In step S401, a preset script file is loaded and installed. The script file prewrites a script program for obtaining a data query condition of a data query page and generating field information corresponding to an information selection page according to field information of the data query page.
[0082] Specifically, the script file can include an executable file written according to a preset format using a preset descriptive language, such as a JS file. In the embodiment of the present disclosure, the script file can include a software program for obtaining a data query condition of a data query page and generating field information corresponding to an information selection page according to field information of the data query page.
[0083] In an example embodiment, the script file can be downloaded and installed before a user opens the data query page, and after installation, the preset instructions in the script file can be executed. Specifically, the script file can include various definitions of data query page identification information, field information collection functions, and the like. After installing the script file in any data query page that needs to add an export file function, the script file can be called to realize the processing of exporting the query condition of the data query page, the query result field, and the identification information of the data query page when the export control is clicked. Specifically, the script file can automatically extract the query result field in the data query page and display it in the information selection page.
[0084] In the embodiment of the present disclosure, the front-end processing of the data query page file export is realized by loading and installing the script file in the original system, the system and the file export are decoupled, the framework code of the existing system does not need to be changed, and the file export implementation has better portability.
[0085] In a possible implementation, the current data query page corresponds to identification information, and the parsing of the data query condition to obtain a data query statement includes:
[0086] Obtaining a plurality of data query statements matched with the identification information;
[0087] Selecting a data query statement matched with the data query condition from the plurality of data query statements.
[0088] Specifically, each data query page is used to implement a certain function, and therefore, the data query pages can be classified. In an example embodiment, identification information is used to distinguish each type of data query page. In an example embodiment, a plurality of data query statements corresponding to the identification information are set in advance, and a data query statement matched with the data query condition is obtained from the plurality of data query statements.
[0089] In the embodiment of the present disclosure, a plurality of data query statements matched with the identification information are set in advance, and a corresponding data query statement is selected from the plurality of data query statements according to the data query condition, thereby realizing the conversion of the visual data query condition to the machine-recognized query statement, which is reasonable in design and strong in implementability.
[0090] In a possible implementation, the data query statement includes an initial data query statement with a placeholder and a query parameter with a parameter position and a parameter type; the obtaining of the multiple data query statements matching the identification information includes:
[0091] obtaining multiple objects matching the identification information;
[0092] the selecting of the data query statement matching the data query condition from the multiple data query statements includes:
[0093] selecting a target object matching the data query condition from the multiple objects;
[0094] in a case where the target object is of a first preset type, obtaining a preset class function of the target object;
[0095] determining a query parameter corresponding to the target object based on the class function;
[0096] inputting the target object into an extraction function of the initial data query statement, and obtaining an initial data query statement through the extraction function.
[0097] Specifically, the multiple objects matching the identification information are, for example, MappedStatement objects. Each object corresponds to a query parameter with a parameter position and a parameter type, for example: [{“0”:[“java.long.String”,”filename”],“1”:[“xxx.xxx.xxx”,”filename2”]}]. Wherein, 0 and 1 represent the parameter position, java.long.String represents the parameter type, and filename represents the field name.
[0098] In an example embodiment, the first preset type can include the above-mentioned string and field name type. The above-mentioned query parameter with a parameter position and a parameter type cannot constitute a query parameter of a data query statement, and needs to be further processed. Specifically, a preset class function corresponding to the target object is obtained. The preset class function can be generated in the following manner: when the bytecode is loaded into the memory, the bytecode is dissected to create a corresponding class function, the information of the query parameter can be stored in the object of the class function, and by obtaining the class function, the query parameter corresponding to the target object is obtained.
[0099] In an example embodiment, the initial data query statement can be extracted through an extraction function of the initial data query statement. The initial data query statement has a placeholder, for example, “?” and “%”. The placeholder is used to indicate the insertion position of the above-mentioned query parameter in the data query statement, so as to constitute a complete data query statement.
[0100] The embodiment of the present disclosure provides a data query statement parsing method, which has the beneficial effects of convenient operation and strong implementability.
[0101] In a possible implementation, after the target object matching the data query condition is selected from the plurality of objects, the method further includes
[0102] In a case where the target object is of a second preset type, a data query statement is generated according to the target object in a generation mode matching the second preset type.
[0103] Specifically, the target object can be provided with an attribute, and when the attribute is of the second preset type, the data query statement is generated in the generation mode matching the second preset type. In an exemplary embodiment, when the second preset type includes a table, an array or other special types, the data query statement can be generated through a self-defined generation mode, for example, a self-defined program code.
[0104] The embodiment of the present disclosure can obtain the data query statement through the self-defined coding mode when facing more complex or rare target object types, and ensures that the file export method can be used in more application scenarios.
[0105] In a possible implementation, the data query statement includes an initial data query statement with a placeholder and a query parameter with a parameter bit and a parameter type, the server is preset with a thread pool of data query tasks, the server obtains a preset number of data query statements from the task table at a timing, and executes a corresponding data query task, including:
[0106] In a case where an export event is listened to, the thread pool is configured to obtain a preset number of data pairs from the task table, and the data pair includes an initial data query statement with a placeholder and a query parameter with a parameter bit and a parameter type;
[0107] The thread pool is configured to obtain a function object corresponding to the initial data query statement with the placeholder;
[0108] The thread pool is further configured to obtain a data query statement according to the function object and the query parameter with the parameter bit and the parameter type, and execute a corresponding data query task.
[0109] Specifically, the thread pool includes a multi-thread processing form, and the tasks are added to a queue in a processing process, and then the tasks are automatically started after the threads are created. The export event can include an event in which the data query statement and the field information are registered in the task table. After the export event is listened to, a preset number of data pairs are obtained from the task table.
[0110] In the embodiment of the present disclosure, the initial data query statement can include an initial data query statement with placeholders. In an exemplary embodiment, a function object can be set for the initial data query statement, the function object containing a correspondence between a query parameter and a parameter value. According to the query parameter with the parameter type and the function object, a parameter value corresponding to the query parameter is determined, the parameter value is inserted into a position of a placeholder of the initial query statement, and the placeholder is deleted, so as to obtain a data query statement.
[0111] The embodiment of the present disclosure provides a data query statement analysis method, which has the beneficial effects of convenient operation and strong implementability.
[0112] In a possible implementation, the file includes a CSV file, and the server performs a corresponding data query task, including:
[0113] The server connects the database through a JDBC driver and reads data in batches, and takes the read data as a query result corresponding to the data query task.
[0114] Specifically, the file can include a CSV file, which is relatively small and can store a large amount of data, and is suitable for large batch data export. In an exemplary embodiment, field information can be taken as a table header of the file, and read data of each batch can be written into a position corresponding to the table header. In an exemplary embodiment, the database is connected through a JDBC driver to read data in batches, and a file cache mode is used for efficient writing into the CSV file. Figure 4 The data query statement and the field information corresponding to the task are registered in the task table, the task publishing unit searches whether there is a task registered in the task table, if yes, an event is generated, and an output listener is created. Each thread in the thread pool processes the corresponding data query task, and when the processing is completed, the output listener unit listens to the result of the data query, generates a message, and returns the data query result to the client through the task table.
[0115] It should be understood that, although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise stated herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.
[0116] Based on the same inventive concept, the embodiments of the present application also provide a file exporting device for implementing the above-mentioned file exporting method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above-mentioned method, and therefore the specific limitations in one or more file exporting device embodiments provided below can refer to the limitations of the file exporting method described above, which will not be repeated here.
[0117] In one embodiment, as shown in Figure 6 a file exporting device is provided, comprising:
[0118] a display module 601 configured to display an information selection page matched with a current data query page in response to an exporting operation of the current data query page;
[0119] an acquisition module 603 configured to acquire selected field information and a data query condition corresponding to the data query page in response to a field information selection operation of the information selection page;
[0120] a parsing module 605 configured to parse the data query condition to obtain a data query statement;
[0121] a generation module 607 configured to register the data query statement and the field information to a task table; wherein the task table is used to instruct a server to acquire a preset number of data query statements from the task table, execute corresponding data query tasks, and generate a file according to a query result of the data query task and the field information, and return the file to the client.
[0122] In a possible implementation, the device further comprises:
[0123] a loading module configured to load and install a preset script file; wherein the script file has a script program pre-written to acquire a data query condition of a data query page and generate field information of an information selection page corresponding to the data query page according to field information of the data query page.
[0124] In a possible implementation, the current data query page has identification information, and the parsing module comprises:
[0125] a first acquisition sub-module configured to acquire a plurality of data query statements matched with the identification information;
[0126] a selection sub-module configured to select a data query statement matched with the data query condition from the plurality of data query statements.
[0127] In a possible implementation, the data query statement includes an initial data query statement with a placeholder and query parameters with parameter positions and parameter types, and the obtaining submodule includes:
[0128] The first obtaining unit is configured to obtain a plurality of objects matched with the identification information.
[0129] The selection submodule includes:
[0130] The selection unit is configured to select a target object matched with the data query condition from the plurality of objects.
[0131] The second obtaining unit is configured to, in a case where the target object is of a first preset type, obtain a preset class function of the target object.
[0132] The determination unit is configured to determine a query parameter corresponding to the target object based on the class function.
[0133] The output unit is configured to input the target object into an extraction function of an initial data query statement, and output an initial data query statement obtained through the extraction function.
[0134] In a possible implementation, the selection submodule further includes:
[0135] The third obtaining unit is configured to, in a case where the target object is of a second preset type, generate a data query statement according to the target object in a generation mode matched with the second preset type.
[0136] In a possible implementation, the generation module includes:
[0137] The second obtaining submodule is configured to, in a case where an export event is listened to, the thread pool is configured to obtain a preset number of data pairs from the task table, and the data pair includes an initial data query statement with a placeholder and query parameters with parameter positions and parameter types.
[0138] The third obtaining submodule is configured to, the thread pool is configured to obtain a function object corresponding to the initial data query statement with the placeholder.
[0139] The execution submodule is configured to, the thread pool is further configured to obtain a data query statement according to the function object and the query parameters with the parameter positions and the parameter types, and execute a corresponding data query task.
[0140] In a possible implementation, the file includes a CSV file, and the generation module includes:
[0141] The reading submodule is used by the server to connect to the database via JDBC driver and read data in batches, and use the read data as the query result corresponding to the data query task.
[0142] Each module in the aforementioned file export device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0143] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores file export data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a file export method.
[0144] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a file export method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.
[0145] Those skilled in the art will understand that Figure 8It should be noted that the structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0146] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.
[0147] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limited to, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.
[0148] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, as long as the combinations of technical features do not have contradictions, they shall be considered within the scope of the present disclosure.
[0149] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it shall not be understood as a limitation on the patent scope of the present application. It shall be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these shall be within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims
1. A file export method characterized by, The application is applied to a client, comprising: in response to an export operation on a current data query page, displaying an information selection page matched with the current data query page; in response to a field information selection operation on the information selection page, obtaining selected field information and a data query condition corresponding to the data query page; parsing the data query condition to obtain a data query statement; registering the data query statement and the field information to a task table; wherein the task table is used to instruct a server to obtain a preset number of data query statements from the task table, execute corresponding data query tasks, and generate a file according to a query result of the data query task and the field information, and return the file to the client.
2. The method of claim 1, wherein, Before the response to the export operation on the data query page, the information selection page matched with the data query page is displayed, further comprising: loading and installing a preset script file; wherein the script file has a script program for obtaining a data query condition of a data query page and generating field information of an information selection page according to field information of the data query page.
3. The method of claim 1, wherein, The current data query page corresponds to identification information, and the parsing of the data query condition to obtain a data query statement comprises: obtaining a plurality of data query statements matched with the identification information; selecting a data query statement matched with the data query condition from the plurality of data query statements.
4. The method of claim 3, wherein, The data query statement comprises an initial data query statement with a placeholder and a query parameter with a parameter position and a parameter type; the obtaining of the plurality of data query statements matched with the identification information comprises: obtaining a plurality of objects matched with the identification information; the selecting of the data query statement matched with the data query condition from the plurality of data query statements comprises: selecting a target object matched with the data query condition from the plurality of objects; in the case that the target object is of a first preset type, obtaining a preset class function of the target object; determining a query parameter corresponding to the target object based on the class function; inputting the target object into an extraction function of the initial data query statement, and outputting the initial data query statement through the extraction function.
5. The method of claim 4, wherein, After the selecting of the target object matched with the data query condition from the plurality of objects, further comprising in the case that the target object is of a second preset type, generating a data query statement according to the target object in a generation mode matched with the second preset type.
6. The method of claim 1, wherein, The data query statement comprises an initial data query statement with a placeholder and a query parameter with a parameter position and a parameter type, the server is preset with a thread pool of data query tasks, and the server obtains a preset number of data query statements from the task table to execute corresponding data query tasks in a timely manner, comprising: in the case that an export event is listened to, the thread pool is used to obtain a preset number of data pairs from the task table, the data pair comprises an initial data query statement with a placeholder and a query parameter with a parameter position and a parameter type; The thread pool is configured to obtain a function object corresponding to the initial data query statement with placeholders; The thread pool is further configured to obtain a data query statement according to the function object and the query parameter with the parameter bit and the parameter type, and execute a corresponding data query task.
7. The method according to any one of claims 1 to 6, characterized in that, The file includes a CSV file, and the server executes a corresponding data query task, including: The server connects a database through a JDBC driver and reads data in batches, and takes the read data as a query result corresponding to the data query task.
8. A file export apparatus characterized by comprising: The apparatus is applied to a client, and the apparatus includes: A display module configured to display an information selection page matched with a current data query page in response to an export operation of the current data query page; An acquisition module configured to acquire selected field information and a data query condition corresponding to the data query page in response to a field information selection operation of the information selection page; An analysis module configured to analyze the data query condition to obtain a data query statement; A generation module configured to register the data query statement and the field information to a task table, wherein the task table is configured to instruct a server to obtain a preset number of data query statements from the task table, execute corresponding data query tasks, generate a file according to a query result of the data query task and the field information, and return the file to the client. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the file export method in any one of claims 1 to 7.
10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the file export method in any one of claims 1 to 7.
Citation Information
Patent Citations
Data export system and working method thereof
CN106776829A
Method for customizing and exporting data in service management system
CN110750553A