Data export method, device, system, electronic device and storage medium
By using long connection channels and asynchronous notifications during the data export process, the problems of long waiting time for users and complexity of developers are solved, and efficient data export and simplified development process are achieved.
Patent Information
- Application Number
- CN202210455785.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-24
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-04-24
AI Technical Summary
During the data export process, users need to wait for a long time, which affects work efficiency. Developers need to develop multiple back-end interactive interfaces and data export templates for different business functions, which increases development complexity and inefficiency.
Through the long-connected communication channel between the server and the client, the target data file is generated by asynchronous notification method, and the file download message is received through the long-connected channel to realize asynchronous data export. Users do not need to wait for the data preparation to be completed, developers only need to Just configure the business data source and export metrics.
It improves user's work efficiency, reduces waiting time, simplifies developers' data export needs in different business functions, and improves development efficiency.
Smart Images

Figure CN114925135B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a data export method, device, system, electronic device and storage medium. Background Art
[0002] With the popularization of office automation, the demand for data export involves more and more scenarios, especially in big data analysis and collaborative office applications, the demand for data export is more prominent.
[0003] In the prior art, when the amount of exported data is large and the export time is long, the user needs to wait for a long time in a function interface for the data to be exported, which greatly affects the user's work efficiency. Summary of the invention
[0004] The purpose of the present invention is to provide a data export method, device, system, electronic device and storage medium. When exporting data, users do not need to wait for a long time in a function interface, which improves the work efficiency of users. At the same time, developers do not need to develop corresponding back-end interactive interfaces and configure data export templates for data export requirements of different business functions. They only need to configure the corresponding business data source and all exportable indicators when implementing specific business data export functions, which can greatly improve the development efficiency of developers.
[0005] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:
[0006] In a first aspect, an embodiment of the present invention provides a data export method, which is applied to a server, wherein the server is communicatively connected to a client, and a long connection communication channel exists between the server and the client. The method comprises: receiving a data export message sent by the client by monitoring the long connection communication channel, wherein the data export message comprises an export rule and a data source; obtaining target data from the data source according to the export rule, and generating a target data file comprising the target data; and sending a file download message to the client through the long connection communication channel, so that the client receives the file download message and downloads the target data file based on the file download message by monitoring the long connection communication channel.
[0007] Further, the export rule includes export parameters and export types, and the step of acquiring target data from the data source and generating a target data file including the target data according to the export rule includes:
[0008] Acquire the target data from the data source according to the export parameter;
[0009] The target data file is generated according to the export type and the target data.
[0010] Further, the export parameters include query parameters and conversion parameters, and the step of acquiring the target data from the data source according to the export parameters includes:
[0011] Acquire initial data from the data source according to the query parameter;
[0012] The initial data is converted into the target data according to the conversion parameters.
[0013] Furthermore, the data source includes a target address and a type of the target address, and the step of acquiring initial data from the data source according to the query parameter includes:
[0014] If the target address type is a database type, querying the initial data from the target database represented by the target address according to the query parameter;
[0015] If the target address type is an interface type, the target interface represented by the target address is called according to the query parameter to obtain the initial data returned by the target interface.
[0016] Furthermore, the export rule also includes an export configuration corresponding to the export type. When the export type is a data type, the export configuration is used to characterize the target file format of the export. When the export type is a data chart type, the export configuration is used to characterize the target chart type. The step of generating the target data file according to the export type and the target data includes:
[0017] If the export type is a data type, generating the target data file consistent with the target file format according to the target data;
[0018] If the export type is a data chart type, a target chart file is generated according to the target data and the target chart type, and the target chart file is used as the target data file.
[0019] Furthermore, the data export request includes a user identifier for logging into the client, and the method further includes:
[0020] generating an export record according to the user identifier and the target data file;
[0021] generating a record identifier of the export record;
[0022] The export record and the corresponding record identifier are stored.
[0023] Furthermore, the server is in communication connection with at least one client, the server stores at least one export record, each of the export records includes a user identifier of the at least one client and a corresponding data file that has not been downloaded, and the method further includes:
[0024] By monitoring the persistent connection communication channel, an initialization message sent by a target client among the at least one client is received, wherein the initialization message includes a target user identifier for logging into the target client;
[0025] If there is a target export record corresponding to the target user identifier in the export record, a file download message is sent to the target client, so that the target client receives the file download message by monitoring the long connection communication channel and downloads the data file corresponding to the target export record based on the file download message.
[0026] In a second aspect, an embodiment of the present invention provides a data export method, which is applied to a client, wherein the client is connected to a server, and a long connection communication channel exists between the server and the client. The method comprises: sending a data export request message to the server through the long connection communication channel, wherein the data export request message comprises an export rule and a data source, so as to instruct the server to obtain target data from the data source according to the export rule and generate a target data file comprising the target data; and instructing the server to send a file download message to the client through the long connection communication channel; receiving the file download message by monitoring the long connection communication channel and downloading the target data file based on the file download message.
[0027] Further, the file download message includes a target record identifier, and the step of downloading the target data file based on the file download message includes:
[0028] Parsing the file download message to obtain the target record identifier;
[0029] An HTTP request is sent to the server, wherein the HTTP request includes the target record identifier, so that the server determines the target data file according to the target record identifier and sends the target data file to the client.
[0030] In a third aspect, an embodiment of the present invention provides a data export method, which is applied to a data export system, wherein the data export system includes a server and a client, and a long connection communication channel exists between the server and the client, and the method includes: the client sends a data export request message to the server, wherein the data export request message includes an export rule and a data source; the server receives the data export request message by monitoring the long connection communication channel; the server obtains target data from the data source according to the export rule, and generates a target data file including the target data; the server sends a file download message to the client through the long connection communication channel; the client receives the file download message by monitoring the long connection communication channel and downloads the target data file based on the file download message.
[0031] In a fourth aspect, an embodiment of the present invention provides a data export device, which is applied to a server, and the server is communicatively connected to a client. A long connection communication channel exists between the server and the client, and the device comprises: a first receiving module, for receiving a data export message sent by the client by monitoring the long connection communication channel, wherein the data export message comprises an export rule and a data source; a generating module, for obtaining target data from the data source according to the export rule, and generating a target data file comprising the target data; a first sending module, for sending a file download message to the client through the long connection communication channel, so that the client receives the file download message by monitoring the long connection communication channel and downloads the target data file based on the file download message.
[0032] In a fifth aspect, an embodiment of the present invention provides a data export device, which is applied to a client, and the client is connected to a server. A long connection communication channel exists between the server and the client, and the device comprises: a second sending module, which is used to send a data export request message to the server through the long connection communication channel, and the data export request message comprises an export rule and a data source, so as to instruct the server to obtain target data from the data source according to the export rule and generate a target data file comprising the target data; and to instruct the server to send a file download message to the client through the long connection communication channel; a second receiving module, which is used to receive the file download message by monitoring the long connection communication channel and download the target data file based on the file download message.
[0033] In a sixth aspect, an embodiment of the present invention provides a data export system, the system comprising a server and a client, the server being communicatively connected to the client, and a long connection communication channel existing between the server and the client; the server executing the data export method of the first aspect, and the client executing the data export method of the second aspect.
[0034] In the seventh aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory; the memory is used to store programs; the processor is used to implement the data export method of the first aspect above, or implement the data export method of the second aspect above when executing the program.
[0035] In an eighth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data export method of the first aspect described above, or implements the data export method of the second aspect described above.
[0036] Compared with the prior art, the embodiments of the present invention provide a data export method, device, system, electronic device and storage medium. Through the long connection communication channel between the server and the client, after the server generates the target data file required by the client, it uses asynchronous notification to realize asynchronous data export. Therefore, when exporting data, the user does not need to wait for a long time in a functional interface, which improves the user's work efficiency. At the same time, developers do not need to develop corresponding back-end interactive interfaces and configure data export templates for data export requirements of different business functions. They only need to configure the corresponding business data source and all exportable indicators when implementing specific business data export functions, which can greatly improve the development efficiency of developers. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0038] Figure 1 This is an example diagram of a data export system provided in an embodiment of the present invention.
[0039] Figure 2 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 1 .
[0040] Figure 3 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 2 .
[0041] Figure 4 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 3 .
[0042] Figure 5 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 4 .
[0043] Figure 6 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 5 .
[0044] Figure 7 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 6 .
[0045] Figure 8 A flow chart of a data export method applied to a server in a data export system provided in an embodiment of the present invention.
[0046] Fig. 9 A schematic flow chart of a data export method applied to a client in a data export system provided by an embodiment of the present invention.
[0047] Fig.10 A block diagram of a first data export device applied to a server in a data export system provided by an embodiment of the present invention.
[0048] Fig.11 A block diagram of a second data export device applied to a client in a data export system provided by an embodiment of the present invention.
[0049] Fig.12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.
[0050] Icons: 10 - server; 20 - client; 30 - electronic device; 31 - processor; 32 - memory; 33 - bus; 100 - first data export device; 110 - first receiving module; 120 - generating module; 130 - first sending module; 140 - storage module; 200 - second data export device; 210 - second sending module; 220 - second receiving module. DETAILED DESCRIPTION
[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0052] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0053] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0054] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0055] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.
[0056] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.
[0057] Please refer to Figure 1 , Figure 1 An example diagram of a data export system provided by an embodiment of the present invention, Figure 1In the embodiment, the data export system includes a server 10 and a client 20. The server 10 and the client 20 are connected in communication, and there is a long connection communication channel between the two. A long connection means that multiple data packets can be sent continuously on a connection. During the connection, if no data packet is sent, both parties need to send a link detection packet. The long connection can be implemented in the form of WebSocket. In WebSocket, the server and the client only need to perform a handshake action through the Hyper Text Transfer Protocol HTTP (HTTP) protocol, and then establish a separate communication channel of the Transmission Control Protocol TCP (TCP) to transmit data. WebSocket is also an application layer protocol like HTTP, but it is a two-way communication protocol built on TCP. The specific process of establishing a long connection can be: first, a Transmission Control Protocol / Internet Protocol TCP / IP (Transmission Control Protocol / Internet Protocol, TCP / IP) communication link between the client 20 and the server 10 is established, and the client 20 can actively initiate TCP / IP communication to the server 10; secondly, based on TCP / IP communication, when the user operates the client 20 to enter the interface containing the data export function, or when the client 20 triggers the corresponding data export function, the client 20 will actively initiate an HTTP upgrade request to the server 10; thirdly, after receiving the upgrade request initiated by the client 20, the server 10 verifies the legitimacy of the identity of the client 20; if the request of the client 20 is illegal, the connection of the client 20 is rejected; if the upgrade request initiated by the client 20 is legal, the HTTP request is upgraded to a Web Socket connection, the client information is bound to the corresponding Web Socket connection, and stored in a preset connection pool.
[0058] After establishing the persistent connection communication channel, the client 20 sends a data export message through the persistent connection communication channel. The data export message includes an export rule and a data source. The server 10 listens to the persistent connection communication channel to receive the data export message. The server 10 obtains the target data from the data source according to the export rule and generates a target data file including the target data. The server 10 sends a file download message to the client 20 through the persistent connection communication channel. The client 20 receives the file download message by listening to the persistent connection communication channel and downloads the target data file based on the file download message.
[0059] In this embodiment, the server 10 can be a physical computer device, or a virtual machine that can implement the same functions as a physical computer device, or a cloud device, etc. The client 20 can be a desktop computer, a laptop computer, a mobile terminal, a mobile phone, etc.
[0060] It should be noted that the number of the client 20 communicating with the server 10 may be one or more.
[0061] based on Figure 1 The data export system shown in the figure, the embodiment of the present invention also provides an application Figure 1 For the method of exporting data from the system, please refer to Figure 2 , Figure 2 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 1 , the process includes the following steps:
[0062] Step S10: The client sends a data export request message to the server, wherein the data export request message includes export rules and data sources.
[0063] In this embodiment, the client sends a data export request message to the server by the user operating the client 20 and entering the interface displayed by the client 20 that includes the data export function. It can also be triggered by the client 20 actively initiating an HTTP connection request to the server 10 when the user triggers a data export-related function on the client 20.
[0064] In this embodiment, the export rules are used to determine, including but not limited to the exported target data, the file format of the target data file including the target data, and the presentation form of the target data in the target data file, etc. For example, the export rules may include query conditions and / or conversion rules. The target data is the data in the data source that meets the query conditions, or the target data is the data converted according to the conversion rules for the data in the data source that meets the query conditions. The conversion rules are not limited to counting, summing, averaging, minimum value, maximum value, merging, data type conversion, and the user may also customize the preset conversion rules. The file format of the target data file is not limited to Excel, text, CSV, JSON, etc. The presentation form of the target data in the target data file may be data or a data chart. The chart includes but is not limited to line charts, column charts, bar charts, pie charts, donut charts, bubble charts, radar charts, rectangular tree charts, curve area charts, distribution charts, funnel charts, and scatter charts.
[0065] In this embodiment, the data source is used to determine the source of the target data. The target data may come from a database, or may be obtained through a remote procedure call (RPC) interface.
[0066] In the embodiment of the present invention, users can configure export rules as needed, thereby avoiding the problem of heavy reliance on data export templates and inability to customize data export rules when exporting data in the prior art, effectively improving user experience and meeting users' more personalized export needs. At the same time, developers do not need to develop corresponding back-end interactive interfaces and configure data export templates for data export requirements of different business functions. They only need to configure the corresponding business data source and all exportable indicators when implementing specific business data export functions, which can greatly improve the development efficiency of developers.
[0067] Step S11: The server receives a data export request message by monitoring the long connection communication channel.
[0068] In this embodiment, once a persistent connection communication channel is established between the client 20 and the server 10, both the server 10 and the client 20 can receive messages sent by the other party by monitoring the persistent connection communication channel.
[0069] Step S12: the server obtains the target data from the data source according to the export rule, and generates a target data file including the target data.
[0070] In this embodiment, the target data may be data obtained according to the export rule, or may be data obtained by first obtaining data from a data source and then performing data conversion on the obtained data.
[0071] In this embodiment, when the target data volume is not large, the target data file may be a single file. When the target data volume is large, the target data file may be a compressed package file obtained by compressing multiple files.
[0072] Step S13: The server sends a file download message to the client through the persistent connection communication channel.
[0073] In this embodiment, after preparing the target data file, the server 10 sends a file download message in an asynchronous manner to notify the client to download the target data file. Therefore, the client 20 does not need to stay on the designated interface to wait while the server is preparing data, but can close the interface or switch to other interfaces to handle other matters during the waiting process. When the server 10 generates the target data file, it will automatically send a file download message to the client 20 to remind the client 20 to download the target data file.
[0074] It should be noted that if the user has not logged out from the client, in order to promptly notify the user to download the target data file, the user can receive the file download message automatically sent by the server 10 in any functional interface. If the user has logged out from the client 20, in order to ensure that the exported target data file will not be downloaded by other users, the server 10 will no longer send the file download message to the client 20.
[0075] Step S14: The client receives a file download message by monitoring the long connection communication channel and downloads the target data file based on the file download message.
[0076] In this embodiment, after receiving the download message, the client 20 may send an HTTP request to the server 10 to download the target data file.
[0077] The above method provided by the embodiment of the present invention uses a long connection communication channel between the server and the client. After the server generates the target data file required by the client, it uses asynchronous notification to realize asynchronous data export. Therefore, when exporting data, the user does not need to wait for a long time in a functional interface, thereby improving the user's work efficiency.
[0078] exist Figure 2 Based on the embodiment of the present invention, a specific implementation method for the server 10 to obtain target data and generate a target data file is provided. Please refer to Figure 3 , Figure 3 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 2 The implementation subject of this implementation is the server 10 in the data export system, and the process includes the following steps:
[0079] Step S12-10, acquiring target data from a data source according to the export parameters.
[0080] In this embodiment, the export rules include export parameters and export types. The export parameters are used to determine the query conditions and conversion rules satisfied by the target data. The export type is used to determine the presentation form of the target data in the target data file and / or the format of the target data file.
[0081] As a specific implementation form, the export parameters include query parameters and conversion parameters. The query parameters are used to determine the query conditions satisfied by the target data, and the conversion parameters are used to determine the conversion rules. The process of obtaining the target data is as follows:
[0082] First, get the initial data from the data source according to the query parameters.
[0083] In this embodiment, the data source generally includes a target address and a target address type. The source of the initial data can be obtained according to the target address, and the method for obtaining the initial data can be determined according to the target address type. Different target address types have different methods for obtaining the initial data. The target address types include at least the following two types: (1) Database type: For the database type, the method for obtaining the initial data is: according to the query parameters, the initial data is queried from the target database represented by the target address. The specific implementation may be to create a communication connection with the target database; convert the query parameters into a query statement of the target database and submit it to the target database; execute the query statement to obtain the initial data from the target database. The database here can be a relational database or a non-relational database. (2) Interface type: For the interface type, the method for obtaining the initial data is: according to the query parameters, the target interface represented by the target address is called to obtain the initial data returned by the target interface. The target interface here can be an RPC interface. The specific implementation may be: according to the RPC protocol, the query parameters are submitted to the RPC interface to obtain the initial data returned by the RPC interface.
[0084] Secondly, the initial data is converted into target data according to the conversion parameters.
[0085] In this embodiment, when the user does not need to convert the initial data, the conversion parameters do not need to be set. In this case, the initial data is the target data. When the user needs to convert the initial data, the conversion parameters can be set in advance. The server 10 converts the target data according to the conversion parameters and uses the converted data as the target data. For example, the initial data is the height of all first-grade students in xx Primary School, and its data type is floating point. The floating point height is converted into text type according to the conversion parameters, and the converted text type height data is the target data.
[0086] It should be noted that the user can also specify the target data to include both the data before conversion and the data after conversion by setting.
[0087] Sub-step S12-11, generating a target data file according to the export type and target data.
[0088] In this embodiment, the export rule also includes an export configuration corresponding to the export type. When the export type is a data type, the export configuration is used to characterize the target file format of the export. For example, the export configuration indicates that the target file format is TXT format. When the export type is a data chart type, the export configuration is used to characterize the target chart type. For example, the target chart type is a bar chart. The specific method is: if the export type is a data type, a target data file consistent with the target file format is generated according to the target data; if the export type is a data chart type, a target chart file is generated according to the target data and the target chart type, and the target chart file is used as the target data file.
[0089] In this embodiment, during the process of the client 20 downloading the target data file from the server 10, the communication connection between the two may be interrupted, or an abnormality may occur in the client 20, resulting in the abnormal closure of the interface of the current data export function and the abnormal interruption of the download process. In order not to affect the download of the target data file after the communication connection between the two is restored to normal, or when the user logs in from one client 20 to another client 20, the embodiment of the present invention also provides a specific implementation method for storing the target data file, please refer to Figure 4 , Figure 4 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 3 The execution subject of the method is the server 10 in the data export system, and the method further includes the following steps:
[0090] Step S15, generating an export record according to the user identifier and the target data file.
[0091] In this embodiment, the user identification can be represented by a character string that uniquely represents the user. The same user can log in to only one client 20 at the same time, or can log in to multiple clients 20 at the same time. For the former, the corresponding target data file can be determined based on the user identification. For example, the same user previously exited the system after clicking export data on client A, and then logged in on client B. The server 10 can send a file download message to the client B where the user is currently logged in, achieving the purpose of not restricting the user to only be able to continue to complete the data export on the same client. For the latter, the user identification and the client identification are required to jointly determine the corresponding target data file. The client identification is used to uniquely represent the client 20. The client identification can be represented by the client's media access control MAC (Media Access Control, MAC) address, and can also be represented by a character string that can uniquely represent the client.
[0092] In this embodiment, as a specific implementation method, the export record includes a user identifier, a function number, a client identifier, a target data file, a record identifier, and an exported or not identifier. Based on the user identifier, the client identifier, and the exported or not identifier, the client 20 corresponding to the export record that has not been downloaded and the user identifier that logs into the client 20 can be obtained, and then the identifier of the export record that has not been downloaded can be determined. Based on the record identifier of the export record, the corresponding target data file can be obtained. Therefore, the same user can still correctly download the target data file when logging into different clients 20 at the same time.
[0093] Step S16, generating a record identifier for the export record.
[0094] In this embodiment, the record identifier is used to uniquely identify the export record. After preparing the target data file, the server 10 can send the record identifier to the client 20, and the client 20 can obtain the corresponding target data file from the server 10 according to the record identifier.
[0095] Step S17, storing the exported records and corresponding record identifiers.
[0096] In this embodiment, in order to facilitate the identification of the download status of the export record, a flag can be added to the export record. For example, the flag is initialized to 0. When the client 20 has not downloaded or has not completed the download, the flag will not be updated. Only when the client 20 has completed the download, the flag will be updated to 1. In this way, the client 20 can be notified to download when there are only export records that have not been downloaded, thereby avoiding unnecessary data communication between the two.
[0097] After the target data file is stored, in order to enable the client 20 to continue downloading the target data file after abnormal recovery, or the client 20 re-enters the interface to download the target data file in time, or the user can continue to download the target data file after logging in to a different client 20, the embodiment of the present invention also provides a specific implementation method for enabling the client 20 to download the target data file normally in this scenario, please refer to Figure 5 , Figure 5 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 4 , the method further comprises the following steps:
[0098] Step S18: The target client sends an initialization message through the persistent connection communication channel, where the initialization message includes a target user identifier for logging into the target client.
[0099] Step S19: The server receives an initialization message by monitoring the long connection communication channel.
[0100] In this embodiment, there can be multiple clients 20. When any client 20 recovers abnormally or re-enters the interface, the client 20 will send an initialization message to the server 10 through the long connection communication channel. The initial message includes the target user identifier for logging in itself.
[0101] Step S20: If there is a target export record corresponding to the target user identifier in the export record, a file download message is sent to the target client.
[0102] Step S21, the target client receives a file download message by monitoring the long connection communication channel and downloads the data file corresponding to the target export record based on the file download message.
[0103] It should be noted that, in addition to the initialization message and the data export request message, the client 20 also sends a heartbeat message to the server 10 through the long connection communication channel. The heartbeat message is used to ensure that the communication connection between the server 10 and the client 20 is normal. After receiving the heartbeat message, the server 10 returns a heartbeat confirmation message to the client 20. Through the heartbeat message and the heartbeat confirmation message, both the client 20 and the server 10 can know whether the communication link between the two parties is normal in a timely manner.
[0104] exist Figure 5 Based on the embodiment of the present invention, a specific implementation method for the client 20 to download the target data file is also provided. Please refer to Figure 6 , Figure 6 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 5 , the method comprises the following sub-steps:
[0105] Sub-step S21 - 10 , the target client parses the file download message and obtains the target record identifier.
[0106] In this embodiment, as described above, when the server 10 stores the export record, it will generate a corresponding record identifier for the export record. When notifying the client 20 to download the file, the corresponding target record identifier will be placed in the file download message and sent. After receiving the file download message, the client 20 can obtain the target record identifier by parsing the message.
[0107] Sub-step S21-11: the target client sends an HTTP request to the server, wherein the HTTP request includes a target record identifier.
[0108] Sub-step S21 - 12 : the server determines the target data file according to the received target record identifier, and sends the target data file to the target client.
[0109] In this embodiment, in order to establish a long connection communication channel between the server and the client, the embodiment of the present invention also provides a specific implementation method, please refer to Figure 7 , Figure 7 Schematic diagram of a data export method for a data export system provided in an embodiment of the present invention Figure 6 , the method comprises the following steps:
[0110] Step S22: The client initiates an HTTP upgrade request to the server.
[0111] Step S23: The server receives the HTTP upgrade request initiated by the client.
[0112] In this embodiment, in order to ensure the legitimacy of the identity of the client 20 and avoid malicious connection of the client 20, the server 10 first verifies the legitimacy of the client 20 after receiving the HTTP upgrade request initiated by the client. If it is legitimate, it continues with the subsequent steps, otherwise, the connection of the client 20 is rejected.
[0113] Step S24: The server completes establishing a Web Socket connection with the client based on the HTTP upgrade request.
[0114] In this embodiment, the server 10 upgrades the HTTP connection to a Web Socket connection based on the HTTP connection request, thereby establishing a Web Socket connection with the client.
[0115] In step S25, the server binds the client and the current operating user to the Web Socket connection and stores them in a preset connection pool to establish a long connection communication channel between the server and the client.
[0116] In order to maintain the communication connection between the client 20 and the server 10 so that the two do not need to reconnect when they communicate in the future, the server 10 binds the client 20 and the Web Socket connection. The specific binding method can be: obtaining the client identifier and the current operating user of the client 20 from the HTTP connection request, establishing a corresponding relationship between the client identifier and the current operating user and the corresponding Web Socket connection, and finally realizing the binding of the client and the Web Socket connection.
[0117] It should be noted that the server 10 can be bound to multiple clients 20 at the same time. For ease of management, the server 10 uses a preset connection pool to store the bound Web Socket connections in the preset connection pool for management.
[0118] The embodiment of the present invention also provides a data export method applied to the server 10 side of the data export system, please refer to Figure 8 , Figure 8 A flow chart of a data export method applied to a server in a data export system provided by an embodiment of the present invention, the method comprising the following steps:
[0119] Step S200, receiving a data export message sent by a client by monitoring a persistent connection communication channel, wherein the data export message includes an export rule and a data source.
[0120] Step S201 : acquiring target data from a data source according to an export rule, and generating a target data file including the target data.
[0121] Step S202: Send a file download message to the client through the persistent connection communication channel, so that the client receives the file download message by monitoring the persistent connection communication channel and downloads the target data file based on the file download message.
[0122] The detailed implementation process of steps S200 to S202 can refer to the description of the previous embodiment and will not be repeated here.
[0123] The embodiment of the present invention also provides a data export method applied to the server 10 side of the data export system, please refer to Fig. 9 , Fig. 9 A flow chart of a data export method for a client in a data export system provided by an embodiment of the present invention, the method comprising the following steps:
[0124] Step S300, sending a data export request message to the server through the long connection communication channel, the data export request message includes export rules and data sources, to instruct the server to obtain target data from the data source according to the export rules, and generate a target data file including the target data; and instruct the server to send a file download message to the client through the long connection communication channel.
[0125] Step S301, receiving a file download message by monitoring a long connection communication channel and downloading a target data file based on the file download message.
[0126] The detailed implementation process of step S300 to step S301 can refer to the description of the previous embodiment and will not be repeated here.
[0127] In order to execute the corresponding steps in the above embodiments and various possible implementations, an implementation method of a first data export device 100 applied to a server in a data export system is provided below. Fig.10 , Fig.10The block diagram of the first data export device 100 provided by the embodiment of the present invention is shown. It should be noted that the basic principle and technical effects of the first data export device 100 provided by this embodiment are the same as those of the above embodiment, and are not mentioned in this embodiment for the sake of brief description.
[0128] The first data exporting device 100 includes a first receiving module 110 , a generating module 120 , a first sending module 130 , and a storage module 140 .
[0129] The first receiving module 110 is used to receive a data export message sent by a client by monitoring a long connection communication channel, wherein the data export message includes an export rule and a data source.
[0130] Furthermore, the server is communicatively connected to at least one client, and the server stores at least one export record, each export record including a user identifier of at least one client logged in and a corresponding data file that has not been downloaded. The first receiving module 110 is also used to: receive an initialization message sent by a target client in at least one client by monitoring a long connection communication channel, and the initialization message includes a target user identifier of the target client logged in.
[0131] The generating module 120 is used to obtain target data from a data source according to the export rule and generate a target data file including the target data.
[0132] Furthermore, the generating module 120 is specifically used to: obtain target data from the data source according to the output parameters; and generate a target data file according to the export type and the target data.
[0133] Furthermore, the export parameters include query parameters and conversion parameters. When the generation module 120 is used to obtain target data from a data source according to the export parameters, it is specifically used to: obtain initial data from the data source according to the query parameters; and convert the initial data into target data according to the conversion parameters.
[0134] Furthermore, the data source includes a target address and a target address type. When the generation module 120 is used to obtain initial data from the data source according to the query parameters, it is specifically used to: if the target address type is a database type, then according to the query parameters, query the initial data from the target database represented by the target address; if the target address type is an interface type, then according to the query parameters, call the target interface represented by the target address to obtain the initial data returned by the target interface.
[0135] Furthermore, the export rules also include an export configuration corresponding to the export type. When the export type is a data type, the export configuration is used to characterize the target file format of the export. When the export type is a data chart type, the export configuration is used to characterize the target chart type. The generation module 120 is used to generate a target data file according to the export type and target data. Specifically, if the export type is a data type, a target data file consistent with the target file format is generated according to the target data; if the export type is a data chart type, a target chart file is generated according to the target data and the target chart type, and the target chart file is used as the target data file.
[0136] The first sending module 130 is used to send a file download message to the client through the persistent connection communication channel, so that the client receives the file download message by monitoring the persistent connection communication channel and downloads the target data file based on the file download message.
[0137] Furthermore, the first sending module 130 is also used to: if there is a target export record corresponding to the target user identifier in the export record, send a file download message to the target client, so that the target client receives the file download message by monitoring the long connection communication channel and downloads the data file corresponding to the target export record based on the file download message.
[0138] Furthermore, the data export request includes a user identifier of a login client, and the storage module 140 is used to: generate an export record according to the user identifier and the target data file; generate a record identifier of the export record; and store the export record and the corresponding record identifier.
[0139] It should be noted that in addition to the above-mentioned module division method of the first data export device 100, the first data export device 100 can also be divided into modules according to the specific functions implemented. For example, as a specific implementation method, the first data export device 100 can also include the following modules: a first communication module, a data loading module, a data conversion module, a file generation module, an export management module and a configuration management module.
[0140] The first communication module is mainly used to communicate with the client 20, including receiving connection requests, heartbeat messages, data export messages, and data file download requests sent by the client 20, and also includes responding to the heartbeat confirmation message and sending the data export message to the client 20.
[0141] The data loading module is mainly used to obtain target data from the target data source, including using different methods to obtain target data according to different target address types.
[0142] The data conversion module is mainly used to convert the target data into the data required by the user according to the data conversion rules set by the user, including counting, summing, averaging, minimum, maximum, merging, converting, and customizing the columns or rows of the target data.
[0143] The file generation module is mainly used to generate target data files, including generating corresponding data files and charts according to the export type. File types include but are not limited to Excel, text, CSV, JSON, and chart types include but are not limited to line charts, column charts, bar charts, pie charts, donut charts, bubble charts, radar charts, rectangular tree charts, curve area charts, distribution charts, funnel charts, and scatter charts.
[0144] The export management module is mainly used to manage export records, including historical export information and export records that have not been downloaded.
[0145] The configuration management module is mainly used to configure the necessary operating conditions for the data export system, which may include database login information, RPC mapping address information, etc.
[0146] In order to execute the corresponding steps in the above embodiment and various possible implementations, an implementation method of a second data export device 200 applied to a client in a data export system is provided below. Fig.11 , Fig.11 The block diagram of the second data export device 200 provided by the embodiment of the present invention is shown. It should be noted that the basic principle and technical effects of the second data export device 200 provided by this embodiment are the same as those of the above embodiment, and are not mentioned in this embodiment for the sake of brief description.
[0147] The second data exporting device 200 includes a second sending module 210 and a second receiving module 220 .
[0148] The second sending module 210 is used to send a data export request message to the server through the long connection communication channel, where the data export request message includes an export rule and a data source, so as to instruct the server to obtain target data from the data source according to the export rule and generate a target data file including the target data; and to instruct the server to send a file download message to the client through the long connection communication channel.
[0149] The second receiving module 220 is used to receive a file download message by monitoring the long connection communication channel and download a target data file based on the file download message.
[0150] Furthermore, the second receiving module 220 is specifically used to: parse the file download message to obtain the target record identifier; send an HTTP request to the server, wherein the HTTP request includes the target record identifier, so that the server determines the target data file according to the target record identifier and sends the target data file to the client.
[0151] It should be noted that in addition to the above-mentioned module division method of the second data export device 200, the second data export device 200 can also be divided into modules according to the specific functions implemented. For example, as a specific implementation method, the second data export device 200 can also include the following modules: a second communication module, a configuration module and an interaction module.
[0152] The second communication module mainly provides communication functions with the server 10, including actively establishing a WebSocket communication connection with the server 10, sending heartbeat information to the server 10 after the communication is established, and receiving message notifications from the server 10. It also includes sending data export messages to the server 10 and downloading target data files to the server 10 when the user triggers the data export operation.
[0153] The configuration module mainly provides developers with data source information and default export rules corresponding to specific business functions, and stores user-defined export rules.
[0154] The interactive module mainly provides user interactive operations, supports users to select export rules, set export parameters in export rules, and set export types.
[0155] It should be noted that the second data export device 200 can also be used as a program plug-in running on the client 20. Developers of other projects can directly introduce the program plug-in, call the program plug-in where the corresponding business function is implemented, and configure the corresponding business data source and all exportable indicators to realize data export, which can greatly improve the development efficiency of developers.
[0156] based on Figure 1 The data export system described herein, the embodiment of the present invention further provides a block diagram of an electronic device 30, the electronic device 30 may be Figure 1 The server 10 in the Figure 1 For client 20, please refer to Fig.12 , Fig.12 A block diagram of an electronic device 30 provided in an embodiment of the present invention is shown. The electronic device 30 includes a processor 31 , a memory 32 and a bus 33 . The processor 31 and the memory 32 are connected via the bus 33 .
[0157] The processor 31 may be an integrated circuit chip having the ability to process signals. In the implementation process, each step of the data export method applied to the server 10 or the data export method applied to the client 20 may be completed by an integrated logic circuit of hardware in the processor 31 or by instructions in the form of software. The above-mentioned processor 31 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0158] The memory 32 is used to store programs, such as the first data export device applied to the server 10 or the second data export device applied to the client 20 in this embodiment. The first data export device and the second data export device each include at least one software function module that can be stored in the memory 32 in the form of software or firmware or solidified in the operating system (OS) of the electronic device 30. After receiving the execution instruction, the processor 31 executes the program to implement the data export method applied to the server 10 or the data export method applied to the client 20 disclosed in the following embodiments.
[0159] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the data export method applied to the server 10 or the data export method applied to the client 20 as described above is implemented.
[0160] In summary, the embodiment of the present invention provides a data export method, device, system, electronic device and storage medium, the method is applied to the server, the server is connected to the client, there is a long connection communication channel between the server and the client, the method includes: by monitoring the long connection communication channel, receiving the data export message sent by the client, wherein the data export message includes the export rule and the data source; according to the export rule, the target data is obtained from the data source, and a target data file including the target data is generated; through the long connection communication channel, a file download message is sent to the client, so that the client receives the file download message by monitoring the long connection communication channel and downloads the target data file based on the file download message. Compared with the prior art, the embodiment of the present invention uses the long connection communication channel between the server and the client, and after the server generates the target data file required by the client, it uses the asynchronous notification method to realize the asynchronous data export, so that when the data is exported, the user does not need to wait for a long time in a function interface, which improves the user's work efficiency. At the same time, the developer does not need to develop the corresponding back-end interaction interface and configure the data export template for the data export requirements of different business functions. It only needs to configure the corresponding business data source and all exportable indicators when implementing the specific business data export function, which can greatly improve the development efficiency of the developer.
[0161] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A data export method, characterized in that: The method is applied to a server, the server is in communication connection with at least one client, a long connection communication channel exists between the server and the client, the server stores at least one export record, each of the export records includes a user identifier for logging into the at least one client and a corresponding data file that has not been downloaded, the method includes: By monitoring the persistent connection communication channel, receiving a data export message sent by the client, wherein the data export message includes an export rule and a data source; According to the export rule, acquiring target data from the data source and generating a target data file including the target data; Sending a file download message to the client through the persistent connection communication channel, so that the client receives the file download message by monitoring the persistent connection communication channel and downloads the target data file based on the file download message; The method further comprises: receiving, by monitoring the persistent connection communication channel, an initialization message sent by a target client among the at least one client, wherein the initialization message includes a target user identifier for logging into the target client, and the initialization message is sent to the server when the target client recovers after a communication interruption with the server, recovers after an exception, or re-enters a function interface for exporting data; If there is a target export record corresponding to the target user identifier in the export record, a file download message is sent to the target client, so that the target client receives the file download message by monitoring the long connection communication channel and downloads the data file corresponding to the target export record based on the file download message.
2. The data export method according to claim 1, characterized in that: The export rule includes export parameters and export types. The step of acquiring target data from the data source and generating a target data file including the target data according to the export rule includes: Acquire the target data from the data source according to the export parameter; The target data file is generated according to the export type and the target data.
3. The data export method according to claim 2, characterized in that: The export parameters include query parameters and conversion parameters, and the step of acquiring the target data from the data source according to the export parameters includes: According to the query parameters, obtaining initial data from the data source; The initial data is converted into the target data according to the conversion parameters.
4. The data export method according to claim 3, characterized in that: The data source includes a target address and a target address type. The step of acquiring initial data from the data source according to the query parameter includes: If the target address type is a database type, querying the initial data from the target database represented by the target address according to the query parameter; If the target address type is an interface type, the target interface represented by the target address is called according to the query parameter to obtain the initial data returned by the target interface.
5. The data export method according to claim 2, characterized in that: The export rule also includes an export configuration corresponding to the export type. When the export type is a data type, the export configuration is used to characterize the target file format of the export. When the export type is a data chart type, the export configuration is used to characterize the target chart type. The step of generating the target data file according to the export type and the target data includes: If the export type is a data type, generating the target data file consistent with the target file format according to the target data; If the export type is a data chart type, a target chart file is generated according to the target data and the target chart type, and the target chart file is used as the target data file.
6. The data export method according to claim 1, characterized in that: The data export request includes a user identifier for logging into the client, and the method further includes: generating an export record according to the user identifier and the target data file; generating a record identifier of the export record; The export record and the corresponding record identifier are stored.
7. A data export method, characterized in that: The method is applied to a client, the client is connected to a server, the server stores at least one export record, each of the export records includes a user identifier for logging into the client and a corresponding data file that has not been downloaded, and there is a long connection communication channel between the server and the client. The method includes: Sending a data export request message to the server through the persistent connection communication channel, wherein the data export request message includes an export rule and a data source, so as to instruct the server to obtain target data from the data source according to the export rule and generate a target data file including the target data; and instructing the server to send a file download message to the client through the persistent connection communication channel; By monitoring the long connection communication channel, receiving the file download message and downloading the target data file based on the file download message; The method further comprises: An initialization message is sent to the server through the persistent connection communication channel, wherein the initialization message includes a target user identifier for logging into the client, and the initialization message is sent to the server when the client recovers after a communication interruption between the client and the server, recovers after an exception, or re-enters a function interface for data export, so that the server receives the initialization message from the client by monitoring the persistent connection communication channel, and when it is determined that there is a target export record corresponding to the target user identifier in the export record, a file download message is sent to the client; By monitoring the long connection communication channel, the file download message is received and the data file corresponding to the target export record is downloaded based on the file download message.
8. The data export method according to claim 7, characterized in that: The file download message includes a target record identifier, and the step of downloading the target data file based on the file download message includes: Parsing the file download message to obtain the target record identifier; An HTTP request is sent to the server, wherein the HTTP request includes the target record identifier, so that the server determines the target data file according to the target record identifier and sends the target data file to the client.
9. A data export method, characterized in that: Applied to a data export system, the data export system includes a server and a client, a long connection communication channel exists between the server and the client, the server stores at least one export record, each of the export records includes a user identifier for logging into the client and a corresponding data file that has not been downloaded, the method includes: The client sends a data export request message to the server, wherein the data export request message includes an export rule and a data source; The server receives the data export request message by monitoring the long connection communication channel; The server obtains the target data from the data source according to the export rule, and generates a target data file including the target data; The server sends a file download message to the client through the long connection communication channel; The client receives the file download message by monitoring the long connection communication channel and downloads the target data file based on the file download message; The method further comprises: An initialization message sent by the client, the initialization message including a target user identifier for logging into the client, and the initialization message is sent to the server when the client recovers after a communication interruption with the server, recovers after an exception, or re-enters a functional interface for data export; The server monitors the long connection communication channel, receives an initialization message from the client, and determines that there is a target export record corresponding to the target user identifier in the export record, and then sends a file download message to the client; The client receives the file download message by monitoring the long connection communication channel and downloads the data file corresponding to the target export record based on the file download message.
10. A data export device, characterized in that: The device is applied to a server, the server is connected to at least one client for communication, a long connection communication channel exists between the server and the client, the server stores at least one export record, each of the export records includes a user identifier for logging into the at least one client and a corresponding data file that has not been downloaded, and the device includes: A first receiving module, configured to receive a data export message sent by the client by monitoring the persistent connection communication channel, wherein the data export message includes an export rule and a data source; A generating module, configured to obtain target data from the data source according to the export rule, and generate a target data file including the target data; A first sending module, configured to send a file download message to the client through the persistent connection communication channel, so that the client receives the file download message by monitoring the persistent connection communication channel and downloads the target data file based on the file download message; The first receiving module is further used to receive an initialization message sent by a target client among the at least one client by monitoring the long connection communication channel, wherein the initialization message includes a target user identifier for logging into the target client, and the initialization message is sent to the server when the target client recovers after a communication interruption between the target client and the server, recovers after an exception, or re-enters a function interface for data export; The first sending module is also used to send a file download message to the target client if there is a target export record corresponding to the target user identifier in the export record, so that the target client receives the file download message by monitoring the long connection communication channel and downloads the data file corresponding to the target export record based on the file download message.
11. A data export device, characterized in that: The device is applied to a client, the client is connected to a server, the server stores at least one export record, each of the export records includes a user identifier for logging into the client and a corresponding data file that has not been downloaded, there is a long connection communication channel between the server and the client, and the device includes: a second sending module, configured to send a data export request message to the server through the persistent connection communication channel, wherein the data export request message includes an export rule and a data source, so as to instruct the server to obtain target data from the data source according to the export rule and generate a target data file including the target data; and to instruct the server to send a file download message to the client through the persistent connection communication channel; A second receiving module, configured to receive the file download message by monitoring the long connection communication channel and download the target data file based on the file download message; The second sending module is further used to send an initialization message, wherein the initialization message includes a target user identifier for logging in to the client, and the initialization message is sent to the server when the client recovers after a communication interruption between the client and the server, or recovers after an exception, or re-enters the data export function interface, so that the server receives the initialization message of the client by monitoring the long connection communication channel, and determines that there is a target export record corresponding to the target user identifier in the export record, and then sends a file download message to the client; The second receiving module is used to receive the file download message by monitoring the long connection communication channel and download the target data file based on the file download message.
12. A data export system, characterized in that: The system includes a server and a client, the server is in communication connection with the client, a long connection communication channel exists between the server and the client, the server stores at least one export record, each of the export records includes a user identifier for logging into the client and a corresponding data file that has not been downloaded; The server executes the data export method described in any one of claims 1-6, and the client executes the data export method described in any one of claims 7-8.
13. An electronic device, characterized in that: It comprises a processor and a memory; the memory is used to store a program; the processor is used to implement the data export method as described in any one of claims 1-6, or the data export method as described in any one of claims 7-8 when executing the program.
14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the data export method as described in any one of claims 1-6, or implements the data export method as described in any one of claims 7-8.
