General data exporting method and system, storage medium and electronic equipment

Abstract

The invention provides a general data exporting method and system, a storage medium and electronic equipment. The method comprises the following steps: asynchronously executing a plurality of general data exporting tasks; wherein the execution process of each general data export task comprises the following steps: analyzing a plurality of data sources, a data cleaning conversion rule and a data packaging encryption mode indicated by the general data export task; concurrently extracting target data from a plurality of data sources through a data collector; based on the data conversion rule, performing data cleaning conversion on the target data; based on a data packaging encryption mode, performing data packaging encryption on the target data after data cleaning conversion to obtain a packaging encryption file; and storing the packaged and encrypted file. According to the general data exporting method and system, higher flexibility is achieved, better user experience is provided, efficient performance is provided, and cross-platform compatibility is achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer data processing technology, and in particular to a general data export method, system, storage medium and electronic device. Background Art

[0002] At present, with the growing demand for data and technological advancement, more and more companies are aware of the importance of data for business decision-making, which has driven the demand for efficient data export tools and services. Existing data export services are increasingly focusing on automation, reducing manual intervention and improving efficiency.

[0003] The existing data export tools have the following disadvantages:

[0004] ETL tools still require code writing for complex data conversion logic; the configuration process may be cumbersome and not friendly to non-technical personnel; some advanced features may require additional fees.

[0005] Open source solutions require a certain skill level to deploy and maintain; documentation and support may not be as comprehensive as commercial products.

[0006] Dedicated data export software and services may lack sufficient flexibility to handle complex business logic; and have limited support for large amounts of data or high performance requirements.

[0007] The export tools that come with the database (such as MySQL Workbench, SQL Server Management Studio, etc.) can usually only process a single type of data source and are not suitable for cross-platform data export; the functions are relatively simple and are not suitable for complex data processing scenarios.

[0008] Therefore, in order to solve the above-mentioned shortcomings, a solution is needed urgently. Summary of the invention

[0009] One of the purposes of the present invention is to provide a universal data export method to solve the above-mentioned shortcomings in the prior art.

[0010] An embodiment of the present invention provides a general data export method, comprising:

[0011] Asynchronously execute multiple general data export tasks; wherein the execution process of each general data export task includes the following steps:

[0012] Parse multiple data sources, data cleaning and transformation rules, and data packaging and encryption methods indicated by general data export tasks;

[0013] Concurrently extract target data from multiple data sources through data collectors;

[0014] Based on data conversion rules, the target data is cleaned and converted;

[0015] Based on the data packaging encryption method, the target data after data cleaning and conversion is packaged and encrypted to obtain a packaged encrypted file;

[0016] Store the packaged encrypted files.

[0017] Optionally, the step of formulating the data cleaning conversion rule includes:

[0018] Allowing users to select data cleaning transformation rules from a first data cleaning transformation rule library;

[0019] and / or,

[0020] Allowing users to fine-tune the rules in the first data cleaning transformation rule library to obtain data cleaning transformation rules;

[0021] and / or,

[0022] Allow users to customize data cleaning and conversion rules;

[0023] and / or,

[0024] It allows users to perform visual combination or encoding to obtain data cleaning and transformation rules.

[0025] Optionally, the rules in the first data cleaning conversion rule library include at least: a date format conversion function and a string operation function.

[0026] Optionally, the formats of the packaged encrypted file include at least: CSV, JSON, Excel and Zip.

[0027] Optionally, the storage destination for storing the packaged encrypted file includes at least: an object storage system, an FTP server system, a cloud storage service system, an FSTP system, and a local file system.

[0028] Optionally, encapsulate the implementation details of different storage destinations through an abstraction layer; support different storage destinations through plug-in or adapter mode.

[0029] Optionally, users can configure common data export tasks through task configuration templates.

[0030] An embodiment of the present invention provides a general data export system, including:

[0031] The asynchronous execution module is used to asynchronously execute multiple general data export tasks; wherein the execution process of each general data export task includes the following steps:

[0032] Parse multiple data sources, data cleaning and transformation rules, and data packaging and encryption methods indicated by general data export tasks;

[0033] Concurrently extract target data from multiple data sources through data collectors;

[0034] Based on data conversion rules, the target data is cleaned and converted;

[0035] Based on the data packaging encryption method, the target data after data cleaning and conversion is packaged and encrypted to obtain a packaged encrypted file;

[0036] Store the packaged encrypted files.

[0037] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement any of the above methods.

[0038] An embodiment of the present invention provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any one of the methods described above.

[0039] This application has achieved the following beneficial effects:

[0040] 1. Higher flexibility: Supports more types of data sources and target storage systems, and is easy to add new data sources;

[0041] 2. Better user experience: Through the intuitive configuration interface, non-technical personnel can easily set up data export templates;

[0042] 3. Efficient performance: able to process large amounts of data in a short time;

[0043] 4. Cross-platform compatibility: It is not limited to a specific cloud platform or technology stack. By implementing applications that can run independently, it does not rely on the characteristics of a single cloud platform to ensure functional integrity. It uses cross-process interactive communication methods to solve the problem of technology stack dependence, making it not limited to a specific cloud platform or technology stack and has wide applicability.

[0044] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0045] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0047] Figure 1 is a schematic diagram of a general data export method in an embodiment of the present invention;

[0048] Figure 2 Schematic diagram of the application process of the general data export method in an embodiment of the present invention;

[0049] Figure 3 Schematic diagram of another application process of the general data export method in an embodiment of the present invention;

[0050] Figure 4 The present invention is another application flow diagram of the general data export method in the embodiment of the present invention. DETAILED DESCRIPTION

[0051] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0052] The embodiment of the present invention provides a general data export method, such as Figure 1 As shown, including:

[0053] Asynchronously execute multiple general data export tasks; wherein the execution process of each general data export task includes the following steps:

[0054] S1. Parse multiple data sources, data cleaning conversion rules, and data packaging and encryption methods indicated by the general data export task;

[0055] S2, extracting target data from multiple data sources concurrently through a data collector;

[0056] S3. Based on the data conversion rules, the target data is cleaned and converted;

[0057] S4. Based on the data packaging encryption method, the target data after data cleaning and conversion is packaged and encrypted to obtain a packaged encrypted file;

[0058] S5. Storing the packaged encrypted file;

[0059] The storage destination for storing the packaged encrypted files includes at least: an object storage system, an FTP server system, a cloud storage service system, an FSTP system, and a local file system;

[0060] Among them, users are supported to configure general data export tasks through task configuration templates.

[0061] like Figure 2-4 As shown, users can create multiple general data export tasks. For multiple general data export tasks, the system can execute asynchronously. When executing a general data export task, the general data export task will indicate the task execution requirements, namely multiple data sources, data cleaning conversion rules, and data packaging encryption methods. After parsing these task execution requirements, the target data is extracted from multiple data sources concurrently through the data collector to improve data extraction efficiency. Then, based on the data conversion rules, the target data is cleaned and converted. Then, based on the data packaging encryption method, the target data after data cleaning conversion is packaged and encrypted to obtain a packaged encrypted file. Finally, the packaged encrypted file is stored. When users configure general data export tasks through task configuration templates, they can perform configuration operations through the configuration interface.

[0062] This application has achieved the following beneficial effects:

[0063] 1. Higher flexibility: Supports more types of data sources and target storage systems, and is easy to add new data sources;

[0064] 2. Better user experience: Through the intuitive configuration interface, non-technical personnel can easily set up data export templates;

[0065] 3. Efficient performance: able to process large amounts of data in a short time;

[0066] 4. Cross-platform compatibility: Not limited to a specific cloud platform or technology stack, it has wide applicability.

[0067] In one embodiment, the step of formulating the data cleaning conversion rule includes:

[0068] Allowing users to select data cleaning transformation rules from a first data cleaning transformation rule library;

[0069] and / or,

[0070] Allowing users to fine-tune the rules in the first data cleaning transformation rule library to obtain data cleaning transformation rules;

[0071] and / or,

[0072] Allow users to customize data cleaning and conversion rules;

[0073] and / or,

[0074] Allow users to perform visual combination or coding to obtain data cleaning and transformation rules;

[0075] The rules in the first data cleaning conversion rule library at least include: a date format conversion function and a string operation function.

[0076] There are four ways to formulate data cleaning conversion rules. The first one is that the system provides built-in conversion rule functions, such as the specified date format conversion function (converting 2024-01-02 to 2024-01-02), string operation functions (such as removing prefixes), and users can click through the interface to add data processing functions to the specified field. The execution flow between functions is executed in the order of addition; the second one is that the system also provides some commonly used data conversion templates, and users can fine-tune the parameters to reuse the template; the third one also supports users to save custom data conversion functions and data templates; the fourth one supports users to combine them in a visual way as needed (which can also be achieved through coding). Improve humanization.

[0077] In one embodiment, the formats of the packaged encrypted file include at least: CSV, JSON, Excel and Zip.

[0078] In one embodiment, implementation details of different storage destinations are encapsulated through an abstraction layer; different storage destinations are supported through a plug-in or adapter pattern.

[0079] Different storage systems may use different protocols. We design an abstract layer to encapsulate the specific implementation details of different storage systems, and support different storage systems through plug-ins or adapter modes. Different storage systems (mysql, mssql, es, clickhouse, redis, plain, etc.) use different data storage protocols. We design an abstract layer for data acquisition to encapsulate the implementation details of different storage systems, so that different storage systems can be supported through plug-ins or adapter modes.

[0080] In one embodiment, the general data export task can also be obtained through the following steps:

[0081] Determine multiple auxiliary design materials based on the user's general data export requirements;

[0082] Based on various auxiliary design materials, assist users in designing general data export tasks.

[0083] In the above technical solution, the general data export requirement is the user's requirement for general data export, such as: what kind of data to export in batches, what kind of data to export regularly, what kind of data needs to be previewed before exporting, and how to record the export history when different data are exported; the auxiliary design material is the material that assists the user in designing general data export tasks according to the general data export requirement, such as: the design history of general data export tasks related to the general data export requirement that have been carried out in the past, a recommended table of export task parameters (such as a table containing encryption packaging methods suitable for different data types, etc.); based on the auxiliary design materials, the user is assisted in designing a general data export task.

[0084] Generally, when faced with a large amount of data that needs to be exported in batches or at regular intervals, the user needs to design the relevant general data export tasks in advance, which requires a large workload, high labor costs, and insufficient design efficiency. The embodiment of the present invention determines multiple auxiliary design materials based on the user's general data export needs, and assists the user in designing a general data export task based on each auxiliary design material, thereby reducing labor costs, improving design efficiency, and being more user-friendly.

[0085] In one embodiment, the step of assisting a user in designing a general data export task based on various auxiliary design materials includes:

[0086] Generate a material display timeline for each auxiliary design material;

[0087] Based on the material display timeline, each auxiliary design material is displayed to the user in sequence;

[0088] Continuously obtain the user's viewing history of the displayed auxiliary design materials;

[0089] Based on the viewing history, select a target auxiliary design material from the remaining auxiliary design materials that are not displayed;

[0090] When the interval between the time position of the target auxiliary design material on the material display timeline and the current time exceeds the interval threshold, the target auxiliary design material is displayed to the user in advance;

[0091] A generic data export task designed to receive user input.

[0092] In the above technical solution, the material display timeline is used to indicate how the system displays the timeline of each auxiliary design material to the user in sequence; when the user views the displayed auxiliary design material, a viewing history will be generated, and the viewing history reflects its specific design intention. Therefore, the target auxiliary design material can be selected from the remaining auxiliary design materials that have not been displayed; the interval threshold can be 5 minutes; when the interval between the time position and the current time exceeds the interval threshold, it means that if you continue to wait for the target auxiliary design material to be displayed, the waiting time will be long. In order to improve the user's design efficiency, the target auxiliary design material is displayed to the user in advance; if the interval does not exceed the interval threshold, it means that the target auxiliary design material will be displayed soon, and the display order of each auxiliary design material on the material display timeline is reasonably planned, so you can wait; when the user completes the general data export task, input will be made.

[0093] The embodiment of the present invention controls the display of various auxiliary design materials to the user in sequence based on the material display timeline, without the need for the user to browse all the auxiliary design materials one by one, which is very user-friendly and improves the user's design efficiency. When the auxiliary design materials are displayed to the user in sequence, the target auxiliary design material is selected based on the user's viewing history, and then it is determined whether to display it to the user in advance based on the relationship between the interval and the interval threshold, which further improves the user's design efficiency.

[0094] In one embodiment, generating a material display timeline of each auxiliary design material includes:

[0095] Based on the material cluster division constraints, each auxiliary design material is divided into multiple material clusters;

[0096] Determine the external sorting value and internal arrangement rules of each material cluster;

[0097] Traverse each material cluster in order from largest to smallest according to the external sort value;

[0098] Each time the traversal is performed, multiple auxiliary design materials in the traversed material cluster are arranged on the timeline based on the internal arrangement rules of the traversed material cluster;

[0099] After traversing each material cluster, the timeline after all auxiliary design materials are arranged is used as the material display timeline;

[0100] The material cluster division constraints include:

[0101] Constraint 1: The total number of auxiliary design materials in the same material cluster exceeds N; where N is a positive integer;

[0102] as well as,

[0103] Constraint 2: The auxiliary design materials of the same material category in the same material cluster are unique;

[0104] as well as,

[0105] Constraint 3: The material category set of the auxiliary design material in each material cluster matches the standard material category set;

[0106] The step of determining the external sorting value and internal arrangement rule of each material cluster includes:

[0107] Calculate the external ranking value of each material cluster using the following formula:

[0108]

[0109] Among them, P is the external ranking value of the material cluster, M is the total number of auxiliary design materials in the material cluster, H is the total number of all auxiliary design materials, P i is the number of categories of the same auxiliary design materials in the material cluster and other i-th material clusters, H-1 is the total number of other material clusters, and Y1 and Y2 are the preset weight values;

[0110] When the auxiliary design materials in the material cluster have not been used by users, the standard arrangement rules corresponding to the standard material category set that matches the material category set of the material cluster are used as internal arrangement rules; otherwise, the internal arrangement rules are generated as follows:

[0111] When arranging, the auxiliary design materials used by the user in the material cluster are arranged from large to small according to the degree of use, and then the first standard duration corresponding to the degree of use of the auxiliary design materials used by the user in the material cluster is used as the first span duration on the time axis, and the used auxiliary design materials are time spanned according to the first span duration;

[0112] The auxiliary design materials in the material cluster that have not been used by the user are arranged according to their relevance to the auxiliary design materials that have been used and from large to small, starting from the last auxiliary design material used by the user. Then, the relevance of the auxiliary design materials in the material cluster that have not been used by the user and the corresponding second standard duration are used as the second span duration on the timeline, and the used auxiliary design materials are set for time span according to the first span duration.

[0113] In the above technical solution, in constraint three, the standard material category set includes categories representing multiple auxiliary design materials that are displayed to users and have design auxiliary value to users; through the constraint of constraint three, when users view the auxiliary design materials in the same material cluster in turn, auxiliary value will be generated; secondly, through the constraints of constraints one and two, when users view the auxiliary design materials in the same material cluster in turn, they will view multiple materials and the material categories will not be repeated, which improves the rationality of the material cluster division; under the constraint of the material cluster division constraint, each time the user views the auxiliary design materials in a material cluster in turn, it is a design decision-making stage, and the user will get sufficient and appropriate assistance.

[0114] In the calculation formula of the external ranking value, M / H is the ratio of the number of auxiliary design materials in the material cluster to the total number of all auxiliary design materials. The larger the ratio, the more priority users can view, and it can also avoid the disgust caused by viewing too much content in the middle and later stages of viewing. It is the average number of categories of the same auxiliary design materials in the material cluster and other material clusters. The larger the average value is, the more likely it is that users can view it first and learn about it in advance. When they view the auxiliary design materials of the same category later, they can make comparisons and provide design assistance from a global perspective. Therefore, the two values ​​are given preset weights to calculate the external sorting value.

[0115] The standard material category set corresponds to a preset standard arrangement rule, which is to sort the auxiliary design materials in the material cluster from high to low according to the popularity of the materials used by different users. The popularity can be the total number of times they have been used by different users in history. When the auxiliary design materials in the material cluster have not been used by any user, the standard arrangement rule corresponding to the standard material category set that matches the material category set of the material cluster is used as the internal arrangement rule. Otherwise, the internal arrangement rule is generated. The usage degree refers to the total number of times the user has used the auxiliary design materials in history, etc. The usage degree corresponds to the preset first standard duration. The higher the usage degree, the more the user knows about the auxiliary design materials, the shorter the viewing time required, and the smaller the first standard duration. The first span duration refers to the length of the time interval where the auxiliary design materials are set on the timeline. The time span setting is to set the time interval of the first span duration on the timeline, and then set the corresponding auxiliary design materials within the interval. First, the auxiliary design materials that have been used are sorted according to the time interval. Arrange them from large to small according to the usage, and then set the time span of the used auxiliary design materials according to the first span time; so that users can view the auxiliary design materials they have used in priority with different reasonable spans, thereby improving their design decision-making efficiency; the correlation degree is the degree of correlation represented by the correlation relationship between the auxiliary design materials. The higher the correlation degree, the more the corresponding auxiliary design material needs to be displayed in priority, and then start arranging after the last auxiliary design material used by the user that has been arranged from large to small according to the correlation degree and the sum; the higher the correlation degree and the sum, the longer the user needs to spend to view, and the longer the corresponding second standard time; similarly, the correlation degree and the corresponding second standard time of the auxiliary design materials that have not been used by the user in the material cluster are used as the second span time on the timeline, and the used auxiliary design materials are set with a time span according to the first span time; then, start arranging the auxiliary design materials that have not been used by the user after the arranged auxiliary design materials that have been used by the user.

[0116] The material display timeline of each auxiliary design material generated by the embodiment of the present invention can help users view each auxiliary design material in the best viewing order, the best viewing rhythm, etc., which greatly improves the efficiency of the user's general data export task design.

[0117] In one embodiment, based on the viewing history, selecting a target auxiliary design material from the remaining undisplayed auxiliary design materials includes:

[0118] Determine the user's task design intent based on viewing history;

[0119] Determine the minimum intent difference between the task design intent and the design intent of multiple standard tasks;

[0120] generating a first material selection rule based on the minimum intention difference;

[0121] Based on the first material selection rule, select a target auxiliary design material from the remaining auxiliary design materials that are not displayed;

[0122] and / or,

[0123] Preprocess the viewing history to obtain a target history sequence; wherein the content in the odd-numbered order in the target history sequence is the first target content in the auxiliary design material that the user views and pays attention to, and the content in the even-numbered order is the second target content that the user views in the gap between the attention of two adjacent odd-numbered first target contents;

[0124] When the average of the first similarities between the second target contents exceeds the first similarity threshold and the second similarity between the first even-numbered second target content in the target history sequence and the trigger content exceeds the second similarity threshold, generating a second material selection rule based on each first target content;

[0125] Based on the second material selection rule, target auxiliary design materials are selected from the remaining auxiliary design materials that are not displayed.

[0126] In the above technical solution, there are two ways to select target auxiliary design materials: first, viewing the history will reflect the user's task design intent. The standard task design intent is the complete design intent that will appear when the user performs a general data export task design. Based on the minimum intent difference between the task design intent and multiple standard task design intentions, the remaining design intent that the user is most likely to reflect in the future can be further determined. Based on the minimum intent difference, a first material selection rule is generated, and the target auxiliary design material is selected based on the rule; the minimum intent difference refers to the intent difference with the smallest degree of difference, and the first material selection rule is to select the auxiliary design material that meets the minimum intent difference.

[0127] The second type is that the viewing history may not necessarily reflect the user's task design intention. The selection is based on the attention paid to the displayed auxiliary design materials; the odd order refers to the 1st, 3rd, 5th, etc., and the even order refers to the 2nd, 4th, 6th, etc.; generate a target history sequence, set the first target content in the odd order, and set the second target content in the even order. The attention gap refers to the gap time between the first target contents of the two adjacent odd orders without paying attention to any content. During the gap time, the user will perform a viewing operation, and the content of the viewing operation will be used as the second target content; the order of the first target content in each odd order is also the user's order of priority. The order of focusing on the first target contents in sequence; the triggering content is the content representing the auxiliary design materials that the user may want to use, for example: retrieving the data to be exported, comparing it with the content in the auxiliary design materials, etc.; when the average value of the first similarities between the second target contents exceeds the first similarity threshold and the second similarity between the first even-numbered second target content in the target history sequence and the triggering content exceeds the second similarity threshold, it indicates that all the first target contents can be used as the basis for selecting the target auxiliary design materials, and based on each first target content, a second material selection rule is generated, and the second material selection rule is to select the auxiliary design materials related to the first target content.

[0128] Through the above two methods of selecting target auxiliary design materials, the most suitable target auxiliary design materials can be selected in different situations, which improves the adaptability of the system and is also more intelligent.

[0129] An embodiment of the present invention provides a general data export system, including:

[0130] The asynchronous execution module is used to asynchronously execute multiple general data export tasks; wherein the execution process of each general data export task includes the following steps:

[0131] Parse multiple data sources, data cleaning and transformation rules, and data packaging and encryption methods indicated by general data export tasks;

[0132] Concurrently extract target data from multiple data sources through data collectors;

[0133] Based on data conversion rules, the target data is cleaned and converted;

[0134] Based on the data packaging encryption method, the target data after data cleaning and conversion is packaged and encrypted to obtain a packaged encrypted file;

[0135] Store the packaged encrypted files.

[0136] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. A processor executes the computer program to implement any of the above methods.

[0137] An embodiment of the present invention provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any one of the methods described above.

[0138] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A general data export method, characterized in that: include: Asynchronously execute multiple general data export tasks; wherein the execution process of each general data export task includes the following steps: Parse multiple data sources, data cleaning and transformation rules, and data packaging and encryption methods indicated by general data export tasks; Concurrently extract target data from multiple data sources through data collectors; Based on data conversion rules, the target data is cleaned and converted; Based on the data packaging encryption method, the target data after data cleaning and conversion is packaged and encrypted to obtain a packaged encrypted file; Store the packaged encrypted files.

2. The general data export method according to claim 1, characterized in that: The steps of formulating the data cleaning conversion rules include: Allowing users to select data cleaning transformation rules from a first data cleaning transformation rule library; and / or, Allowing users to fine-tune the rules in the first data cleaning transformation rule library to obtain data cleaning transformation rules; and / or, Allow users to customize data cleaning and conversion rules; and / or, It allows users to perform visual combination or encoding to obtain data cleaning and transformation rules.

3. The general data export method according to claim 2, characterized in that: The rules in the first data cleaning conversion rule library at least include: a date format conversion function and a string operation function.

4. The general data export method according to claim 1, characterized in that: The formats of the packaged encrypted files include at least: CSV, JSON, Excel and Zip.

5. The general data export method according to claim 1, characterized in that: The storage destination for storing the packaged encrypted files includes at least: an object storage system, an FTP server system, a cloud storage service system, an FSTP system, and a local file system.

6. The general data export method according to claim 5, characterized in that: Encapsulate the implementation details of different storage destinations through an abstraction layer; support different storage destinations through plug-in or adapter mode.

7. The general data export method according to claim 1, characterized in that: Supports users to configure common data export tasks through task configuration templates.

8. A general data export system, characterized in that: include: The asynchronous execution module is used to asynchronously execute multiple general data export tasks; wherein the execution process of each general data export task includes the following steps: Parse multiple data sources, data cleaning and transformation rules, and data packaging and encryption methods indicated by general data export tasks; Concurrently extract target data from multiple data sources through data collectors; Based on data conversion rules, the target data is cleaned and converted; Based on the data packaging encryption method, the target data after data cleaning and conversion is packaged and encrypted to obtain a packaged encrypted file; Store the packaged encrypted files.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the processor executes the computer program to implement the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method according to any one of claims 1 to 7.

Images