Data processing method, device, computer equipment and storage medium

Abstract

The present application relates to a data processing method, device, computer equipment and storage medium. The method comprises: in response to a received data processing instruction, entering a data processing initial configuration interface, the data processing initial configuration interface displays a data processing option list, and the data processing option list contains different types of data processing options; in response to a target data processing instruction corresponding to a determined target data processing option, entering a target data processing configuration interface, the target data processing option is a data processing option determined in the data processing option list according to data processing requirements; determining a target data processing rule in a processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and processing the data to be processed based on the target data processing rule. This method can be used to customize data processing rules and improve data processing flexibility.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, apparatus, computer equipment and storage medium. Background Art

[0002] With the development of information technology, there are more and more scenarios for data exchange in the form of files. When exchanging data between different systems, the file data needs to be formatted and uniformly imported into the database, so as to facilitate subsequent query and acquisition of standardized data for statistical analysis in the database. In the process of file data formatting, different types of processing are required for the file data.

[0003] In the traditional data processing process, multiple components need to be loaded to meet the multi-dimensional needs of data processing. However, there is a strong coupling (strong dependence) between the components. Once a problem occurs in a processing flow, the data cannot be processed and imported normally. Summary of the invention

[0004] Based on this, it is necessary to provide a data processing method, device, computer equipment and storage medium to address the above technical problems.

[0005] A data processing method, the method comprising:

[0006] In response to the received data processing instruction, entering a data processing initial configuration interface, wherein the data processing initial configuration interface displays a data processing option list, wherein the data processing option list includes different types of data processing options;

[0007] Entering a target data processing configuration interface in response to a target data processing instruction corresponding to a determined target data processing option, wherein the target data processing option is a data processing option determined in the data processing option list according to a data processing requirement;

[0008] The target data processing rule is determined in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and the data to be processed is processed based on the target data processing rule.

[0009] In one embodiment, the different types of data processing options included in the data processing option list specifically include: data correction options, data cleaning options, data conversion options and threshold alarm options, wherein each of the data processing options serves as a data processing node, and circulates the data to be processed based on the order of target data processing options in the received target configuration request, and the target configuration request contains at least one target data processing option.

[0010] In one embodiment, the target data processing option is a data correction option; in response to the target data processing instruction corresponding to the determined target data processing option, a target data processing configuration interface is entered, and the target data processing option is a data processing option determined according to data processing requirements in the data processing option list, including:

[0011] In response to the data correction instruction corresponding to the data correction option, a data correction configuration interface is entered, wherein the data correction configuration interface displays a list of data correction processing rules, wherein the list of data correction processing rules includes general rules, date data special rules, and numerical data special rules;

[0012] The step of determining a target data processing rule in a processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and processing the data to be processed based on the target data processing rule includes:

[0013] According to the data type and data content of the data to be processed, at least one data correction processing rule is determined in the data correction processing rule list as a target data correction processing rule;

[0014] Activate and enable the target data correction processing rule to perform correction processing on the data to be processed;

[0015] If the target data correction processing rule includes the general rule, the full amount of the data to be processed is processed according to the full amount of data format information specified in the general rule, so that the processed data to be processed all meet the full amount of data format specified in the general rule;

[0016] If the target data correction processing rule includes the date data special rule, the date data in the data to be processed is specially processed according to the date data special rule, and the date data is corrected to the standard date format preset in the date data special rule;

[0017] If the target data correction processing rule includes the numerical special rule, the numerical data column in the to-be-processed data is identified and verified according to the numerical special rule, and the non-numerical data in the numerical data column is converted into numerical data;

[0018] The data processed according to the target data correction processing rule is transferred to the next node.

[0019] In one embodiment, the target data processing option is a data cleaning option, and the target data processing instruction corresponding to the determined target data processing option is responded to, and a target data processing configuration interface is entered, and the target data processing option is a data processing option determined according to data processing requirements in the data processing option list, including:

[0020] In response to a data cleaning instruction corresponding to the data cleaning option, entering a data cleaning configuration interface, wherein the data cleaning configuration interface displays a list of data cleaning processing rules;

[0021] The step of determining a target data processing rule in a processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and processing the data to be processed based on the target data processing rule includes:

[0022] According to the data type and data content of the data to be processed, at least one data cleaning processing rule is determined in the data cleaning processing rule list as a target data processing rule;

[0023] The target data cleaning processing rule is activated and enabled, the data to be processed is cleaned, abnormal data in the data to be processed is screened and eliminated, and the cleaned data is transferred to the next node.

[0024] In one embodiment, the target data processing option is a threshold alarm option; the target data processing instruction corresponding to the determined target data processing option is responded to, and a target data processing configuration interface is entered, wherein the target data processing option is a data processing option determined according to data processing requirements in the data processing option list, including:

[0025] In response to a threshold alarm instruction corresponding to the threshold alarm option, entering a threshold alarm configuration interface, the threshold alarm configuration interface displays a list of threshold alarm processing rules;

[0026] The step of determining a target data processing rule in a processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and processing the data to be processed based on the target data processing rule includes:

[0027] According to the data type and data content of the data to be processed, determining at least one threshold alarm processing rule in the processing rule list in the threshold alarm configuration interface as the target data alarm processing rule;

[0028] Activate and enable the target data alarm processing rule, classify and count the amount of abnormal data in the data to be processed, and issue an alarm if the amount of abnormal data is greater than a preset threshold;

[0029] The data to be processed is modified according to the alarm prompt until the alarm is cancelled, and the processed normal data is transferred to the next node.

[0030] In one embodiment, the target data processing option is a data conversion option; in response to the target data processing instruction corresponding to the determined target data processing option, a target data processing configuration interface is entered, and the target data processing option is a data processing option determined according to data processing requirements in the data processing option list, including:

[0031] In response to a data conversion instruction corresponding to the data conversion option, entering a data conversion configuration interface, the data conversion configuration interface displays a list of data conversion processing rules;

[0032] The step of determining a target data processing rule in a processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and processing the data to be processed based on the target data processing rule includes:

[0033] According to the data type and data content of the data to be processed, at least one data conversion processing rule is determined in the data conversion rule list as a target data conversion processing rule;

[0034] Activate and enable the target data conversion processing rules, perform conversion processing on the data to be processed, establish a one-to-one mapping relationship between the integration layer fields in the data to be processed and the standard layer fields in the database, and import the integration layer fields in the data to be processed into the standard layer of the database according to the one-to-one mapping relationship.

[0035] A data processing device, comprising:

[0036] An initial configuration module, for entering a data processing initial configuration interface in response to a received data processing instruction, wherein the data processing initial configuration interface displays a data processing option list, wherein the data processing option list includes different types of data processing options;

[0037] a target configuration module, for entering a target data processing configuration interface in response to a target data processing instruction corresponding to a determined target data processing option, wherein the target data processing option is a data processing option determined according to a data processing requirement in the data processing option list;

[0038] The data processing module is used to determine the target data processing rule in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and process the data to be processed based on the target data processing rule.

[0039] In one embodiment, the different types of data processing options included in the data processing option list specifically include: data correction options, data cleaning options, data conversion options and threshold alarm options, wherein each of the data processing options serves as a data processing node, and circulates the data to be processed based on the order of target data processing options in the received target configuration request, and the target configuration request contains at least one target data processing option.

[0040] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0041] In response to the received data processing instruction, entering a data processing initial configuration interface, wherein the data processing initial configuration interface displays a data processing option list, wherein the data processing option list includes different types of data processing options;

[0042] Entering a target data processing configuration interface in response to a target data processing instruction corresponding to a determined target data processing option, wherein the target data processing option is a data processing option determined in the data processing option list according to data processing requirements;

[0043] The target data processing rule is determined in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and the data to be processed is processed based on the target data processing rule.

[0044] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0045] In response to the received data processing instruction, entering a data processing initial configuration interface, wherein the data processing initial configuration interface displays a data processing option list, wherein the data processing option list includes different types of data processing options;

[0046] Entering a target data processing configuration interface in response to a target data processing instruction corresponding to a determined target data processing option, wherein the target data processing option is a data processing option determined in the data processing option list according to data processing requirements;

[0047] The target data processing rule is determined in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and the data to be processed is processed based on the target data processing rule.

[0048] The above-mentioned data processing method, device, computer equipment and storage medium enter the data processing initial configuration interface in response to the received data processing instruction, and the data processing initial configuration interface displays a data processing option list, and the data processing option list contains different types of data processing options; in response to the target data processing instruction corresponding to the determined target data processing option, enter the target data processing configuration interface, and the target data processing option is the data processing option determined in the data processing option list according to the data processing requirements; determine the target data processing rule in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and process the data to be processed based on the target data processing rule. With this method, the data processing type and data processing rule can be configured autonomously, making data processing more flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a data processing method in one embodiment;

[0050] Figure 2 A schematic flow chart of a data correction processing method of a data correction option in an embodiment;

[0051] Figure 3 A schematic diagram of a data modification configuration interface in one embodiment;

[0052] Figure 4 A flowchart of a data cleaning method for a data cleaning option in an embodiment;

[0053] Figure 5 A schematic diagram of a data cleaning configuration interface in one embodiment;

[0054] Figure 6 A schematic flow chart of a threshold value alarm method of a threshold value alarm option in an embodiment;

[0055] Figure 7 A schematic diagram of a threshold alarm configuration interface in one embodiment;

[0056] Figure 8 A schematic flow chart of a data conversion method of a data conversion option in one embodiment;

[0057] Fig. 9 A schematic diagram of a data conversion configuration interface in one embodiment;

[0058] Fig.10 An example diagram of a data processing method in an embodiment;

[0059] Fig.11 is a structural block diagram of a data processing device in one embodiment;

[0060] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] In one embodiment, Figure 1 As shown, a data processing method is provided. This embodiment takes the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0063] Step 101, in response to a received data processing instruction, enter a data processing initial configuration interface, the data processing initial configuration interface displays a data processing option list, and the data processing option list includes different types of data processing options.

[0064] During implementation, the computer device enters a data processing initial configuration interface in response to the received data processing instructions, in which a data processing option list is displayed. The data processing option list includes different types of data processing options, such as data correction options, data cleaning options, data conversion options, and threshold alarm options.

[0065] Step 102, in response to the target data processing instruction corresponding to the determined target data processing option, enter the target data processing configuration interface, the target data processing option being the data processing option determined in the data processing option list according to the data processing requirement.

[0066] During implementation, the user selects the corresponding data processing option from the data processing option list displayed on the data processing initial configuration interface according to the data processing requirements, and inputs the corresponding trigger instruction, i.e., the target data processing instruction, by clicking. The computer device responds to the target data processing instruction and enters the target data processing configuration interface corresponding to the determined target data processing option.

[0067] Optionally, when there are multiple target data processing options determined, a list of target data processing options is formed according to the order in which the multiple target data processing options are determined, and the target data processing instructions corresponding to each target data processing option in the list are read and responded to in turn, and each target data processing option interface is entered in turn, and the data to be processed is processed.

[0068] Step 103 , determining a target data processing rule in a processing rule list of a target data processing configuration interface according to the data type and data content of the data to be processed, and processing the data to be processed based on the target data processing rule.

[0069] During implementation, a target data processing rule list corresponding to the target type data processing is provided in the target data processing configuration interface. The computer device determines the target data processing rule in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and processes the data to be processed based on the target data processing rule.

[0070] The above data processing method, in response to the received data processing instruction, enters the data processing initial configuration interface, the data processing initial configuration interface displays a data processing option list, and the data processing option list contains different types of data processing options; in response to the target data processing instruction corresponding to the determined target data processing option, enters the target data processing configuration interface, the target data processing option is the data processing option determined in the data processing option list according to the data processing requirements; according to the data type and data content of the data to be processed, the target data processing rule is determined in the processing rule list of the target data processing configuration interface, and the data to be processed is processed based on the target data processing rule. The method can be used to implement autonomous configuration of data processing types and data processing rules, so that data processing is more flexible, and each data processing option is independent, which reflects weak coupling in the process of freely configuring multiple data processing (type) options.

[0071] Optionally, the data processing rules may also be manually configured by the user, that is, by pulling down the processing rule list in the target data processing configuration interface to display the processing rule implementation details in the processing rule list, and completing the configuration of the target data processing rules through manual selection.

[0072] In one embodiment, the different types of data processing options included in the data processing option list specifically include: data correction options, data cleaning options, data conversion options and threshold alarm options, wherein each data processing option serves as a data processing node, and circulates the data to be processed based on the order of target data processing options in the received target configuration request, and the target configuration request contains at least one target data processing option.

[0073] In implementation, the data processing option list in the initial configuration interface includes different types of data processing options, specifically data correction options, data cleaning options, data conversion options and threshold alarm options, wherein each data processing option can be used as a data processing node, and each data processing node flows the data to be processed based on the order of the target data processing options in the received target configuration request. Optionally, the target configuration request includes at least one target data processing option.

[0074] In one embodiment, Figure 2 As shown, the target data processing option is a data correction option, and the specific processing process of step 102 is as follows:

[0075] Step 201, in response to the data correction instruction corresponding to the data correction option, enter the data correction configuration interface, the data correction configuration interface displays a data correction processing rule list, the data correction processing rule list includes general rules, date data special rules and numerical data special rules.

[0076] During implementation, the computer device responds to the data correction instruction corresponding to the data correction option and enters the data correction configuration interface, which displays a data correction processing rule list, which includes general rules, date data special rules and numerical data special rules.

[0077] Specifically, Figure 3 As shown, after entering the data correction configuration interface, select general rules for different rule types in the data correction processing rule list, and then obtain a general rule template interface, so that the general rules can be independently configured in the general rule template interface.

[0078] On this basis, the specific processing process of step 103 is as follows:

[0079] Step 202: According to the data type and data content of the data to be processed, at least one data correction processing rule is determined in the data correction processing rule list as the target data correction processing rule.

[0080] In implementation, the computer device determines at least one data correction processing rule in the data correction processing rule list as the target data correction processing rule according to the data type and data content of the data to be processed.

[0081] Specifically, in the data correction processing rule list (i.e., in the general rules, date data special rules, and numerical data special rules), select at least one data correction processing rule of a rule type, and then select specific rule implementation details under each rule type according to the selected data correction processing rules of different rule types, and use all the obtained rule implementation details as the target data correction processing rule. Taking the data correction processing rule of the general rule type determined in the data correction processing rule list as an example, at least one general rule implementation detail is determined in the general rule list corresponding to the general rule as the target data correction processing rule, so that subsequent data correction is performed according to the target data correction processing rule (target general rule implementation detail).

[0082] Step 203: Activate and enable the target data correction processing rule to perform correction processing on the data to be processed.

[0083] In implementation, the target data correction processing rule needs to be published before it can take effect. Therefore, for the determined target data correction processing rule, the computer device can save the determined target data correction processing rule into a relational database, and then activate and enable the target data correction processing rule so that it is displayed in the rule status bar, so that the data to be processed can be corrected according to the activated target data processing correction rule.

[0084] Step 204, if the target data correction processing rules include general rules, the full amount of data to be processed is processed according to the full amount of data format information specified in the general rules, so that the processed data to be processed all meet the full amount of data format specified in the general rules.

[0085] In implementation, if the target data correction processing rules include data processing rules whose rule type is a general rule, it means that the computer device can use the general rule to screen and process the data format information of the full data samples in the data to be processed, so that the processed data to be processed meets the data format of the full data specified in the general rule.

[0086] Specifically, if the target data correction processing rules include the general rule implementation details checked under the general rule type, such as Figure 3As shown, in the general rule template of the general rule, check and configure specific general rule implementation details, and then traverse, screen and process the entire amount of data to be processed according to the general rule implementation details. For example, the general rule implementation details can be "Songti" for the font style of the data to be processed, and "Small Four" for the font size. Then, according to the requirements in the implementation details, the font style and font size of the data in the data to be processed are adjusted accordingly, so that the data in the data to be processed meets the requirements in the general rule implementation details. For another example, the general rule implementation details can also be for correcting and deleting specific character content contained in the entire amount of data samples. Then, according to the general rule implementation details, the entire amount of data to be processed is screened, and the specific characters contained therein are corrected and deleted. Therefore, the specific implementation content of the general rule implementation details can be varied, and this embodiment does not limit it.

[0087] Step 205, if the target data correction processing rule includes a special rule for date data, then the date data in the data to be processed is specially processed according to the special rule for date data, and the date data is corrected to the standard date format preset in the special rule for date data.

[0088] During implementation, if the target data correction processing rules include the implementation details of the special date data processing rules checked under the special date data rules, pull down the special date data rule configuration box in the displayed correction processing interface, and configure specific special date data processing rule implementation details (each implementation rule displays the corresponding rule number and rule status, rule description and other information), it means that the computer device can use the selected special date data rule implementation details to perform special processing on the date data in the data to be processed, and correct the date data to the standard date format defined in the special date data rule, that is, correct the date type data in the data to be processed in the source layer (the source layer stores the most original data obtained from the business system, and is the source data of other upper-layer data) to a standardized date format.

[0089] Step 206, if the target data correction processing rules include special numerical rules, the numerical data columns in the data to be processed are identified and verified according to the special numerical rules, and the non-numerical data in the numerical data columns are converted into numerical data.

[0090] In implementation, if the target data correction processing rules include the implementation details of the special numerical processing rules checked under the special numerical rules, it means that the computer equipment can identify and verify the numerical column data in the processed data according to the implementation details of the special numerical processing rules, and convert the non-numerical data in the numerical column data into numerical data, that is, convert the non-numerical data (for example, strings, etc.) existing in the numerical column data in the processed data of the source layer into corresponding integer, floating-point and other numerical data expressions.

[0091] Step 207, transferring the data processed according to the target data correction processing rule to the next node.

[0092] In implementation, since each data processing option can serve as a data processing node, the computer device will transfer the data flow node that has been processed by specific rule implementation details (target data correction rules) such as general rules, date data special processing rules and numerical data special processing rules under the data correction processing option (one of the target data processing options) to the next node.

[0093] In one embodiment, Figure 4 As shown, the target data processing option is a data correction option, and the specific processing process of step 102 is as follows:

[0094] Step 401, in response to a data cleaning instruction corresponding to a data cleaning option, enter a data cleaning configuration interface, where a list of data cleaning processing rules is displayed.

[0095] During implementation, the computer device responds to the data cleaning instruction corresponding to the data cleaning option and enters the data cleaning configuration interface, in which a list of data cleaning processing rules is displayed. The list of data cleaning processing rules also corresponds to different types of data cleaning rules for different types of data, for example, cleaning rules corresponding to date data, cleaning rules for tax data, cleaning rules for identity data, etc., which are not limited to the embodiments of the present application.

[0096] On this basis, the specific processing process of step 103 is as follows:

[0097] Step 402: According to the data type and data content of the data to be processed, at least one data cleaning processing rule is determined in the data cleaning processing rule list as a target data processing rule.

[0098] In implementation, the computer device determines at least one data cleaning processing rule in the data cleaning processing rule list as the target data processing rule according to the data type and data content of the data to be processed.

[0099] Specifically, there are different types of data cleaning rules for different types of data, such as Figure 5 As shown, taking date data as an example, in the date cleaning rule configuration display box, at least one date data cleaning rule implementation detail is determined as the target data processing rule to clean the date data. Optionally, in addition to determining the date data cleaning rule implementation details according to the default configuration of the computer device, the user can also determine it by inputting instructions, which is not limited in the embodiment of the present application.

[0100] Step 403, activate and enable the target data cleaning processing rule, perform cleaning processing on the data to be processed, filter out abnormal data in the data to be processed, and transfer the cleaned data to the next node.

[0101] In implementation, the target data cleaning processing rules need to be published before they can take effect. Therefore, the computer equipment activates and enables the determined target data cleaning rules (data cleaning rule implementation details), cleans the data to be processed, and screens out abnormal data in the data to be processed. Since each data processing option can be used as a data processing node, the data cleaning processing option is also one of the nodes. After the cleaning process, the processed data will be transferred to the next node.

[0102] In one embodiment, Figure 6 As shown, if the target data processing option is a threshold alarm option, the specific processing process of step 102 is as follows:

[0103] Step 601, in response to a threshold alarm instruction corresponding to a threshold alarm option, entering a threshold alarm configuration interface, the threshold alarm configuration interface displays a threshold alarm processing rule list.

[0104] In implementation, the computer device responds to the threshold alarm instruction corresponding to the threshold alarm option and enters the threshold alarm configuration interface, which displays a threshold alarm processing rule list in which different types of alarm prompt conditions can be customized and selected.

[0105] On this basis, the specific processing process of step 103 is as follows:

[0106] Step 602: According to the data type and data content of the data to be processed, at least one threshold alarm processing rule is determined in the processing rule list of the threshold alarm configuration interface as the target data alarm processing rule.

[0107] In implementation, the computer device determines at least one threshold alarm processing rule in the processing rule list of the threshold alarm configuration interface as the target data alarm processing rule according to the data type and data content of the data to be processed.

[0108] Specifically, the processing rule list of the threshold alarm configuration interface consists of implementation details of different threshold alarm rules, such as Figure 7As shown, the scientific notation of the numerical data in the data to be processed is used as one of the alarm conditions determined, that is, when the amount of the numerical data in the scientific notation form in the data exceeds the preset threshold, an alarm prompt is given to the data sample to be processed. In addition, a data amount threshold can also be set for the amount of null value (missing) data in the data to be processed as a null value data alarm condition. Therefore, this embodiment does not limit the specific threshold alarm rule.

[0109] Step 603, activating and enabling the target data alarm processing rule, classifying and counting the amount of abnormal data in the data to be processed, and issuing a prompt alarm if the amount of abnormal data is greater than a preset threshold.

[0110] In implementation, the target data alarm processing rules need to be published before they can take effect. Therefore, the computer equipment activates and enables the target data alarm processing rules, classifies and counts the amount of abnormal data in the data to be processed, and if the amount of abnormal data is greater than the preset threshold, an alarm is issued.

[0111] Step 604, modify the data to be processed according to the alarm prompt until the alarm is cancelled, and transfer the processed normal data to the next node.

[0112] During implementation, the computer device modifies the data to be processed according to the alarm prompt until the alarm is cancelled. Since each data processing option can be used as a data processing node, after the threshold alarm option is processed, the computer device transfers the normal data obtained after processing to the next node.

[0113] Optionally, the computer device can also select a default processing flow, that is, directly block or end the process, or the user can select a subsequent processing flow, that is, ignore the alarm and continue the data processing flow of the next node, or modify and adjust the data as in step 604, eliminate the alarm and enter the next node. Therefore, this embodiment is not limited.

[0114] In one embodiment, Figure 8 As shown, the target data processing option is a data conversion option, and the specific processing process of step 102 is as follows:

[0115] Step 801, in response to a data conversion instruction corresponding to a data conversion option, enter a data conversion configuration interface, which displays a list of data conversion processing rules.

[0116] In implementation, the computer device enters a data conversion configuration interface in response to a data conversion instruction corresponding to the data conversion option, and a list of data conversion processing rules is displayed on the data conversion configuration interface.

[0117] Step 802: According to the data type and data content of the data to be processed, at least one data conversion processing rule is determined in the data conversion rule list as the target data conversion processing rule.

[0118] In implementation, the computer device determines at least one data conversion processing rule in the data conversion rule list as the target data conversion processing rule according to the data type and data content of the data to be processed.

[0119] Specifically, after one or more of the above-mentioned data correction processing, data cleaning processing, threshold alarm processing, etc., the data enters the integration layer. When converting the data to be processed in the integration layer, it is first necessary to configure the data conversion processing rules. Each data conversion processing rule corresponds to an integration model. The integration model is used to realize the integration mapping of the integration layer data table of the data to be processed and the standard layer data table in the database, such as Fig. 9 As shown, the user can select the standard layer data table to be entered after the conversion in the drop-down box of the data conversion rule list in the display interface (data standardization configuration template), so as to determine the mapping method between the integration layer data table and the standard layer data table of the data to be processed according to the corresponding data conversion rule (using the integration model).

[0120] Step 803, activate and enable the target data conversion processing rules, perform conversion processing on the data to be processed, establish a one-to-one mapping relationship between the integration layer fields in the data to be processed and the standard layer fields in the database, and import the integration layer fields in the data to be processed into the standard layer of the database according to the one-to-one mapping relationship.

[0121] In implementation, the target data conversion processing rules need to be published before they can take effect. Therefore, the computer equipment activates and enables the target data conversion processing rules, converts the data to be processed, and uses the integration model to establish a one-to-one mapping relationship between the integration layer fields in the data to be processed and the standard layer fields in the database. According to the one-to-one mapping relationship, the integration layer fields (data) in the data to be processed are imported into the standard layer of the database to complete the data entry.

[0122] In one embodiment, Fig.10 As shown, this application provides a data processing example, and the specific steps are as follows:

[0123] Step 1001, the computer device enters the data processing initial configuration interface in response to the received data processing instruction, and the data processing initial configuration interface displays a data processing option list, which includes data correction options, data cleaning options, threshold alarm options and data conversion options.

[0124] Step 1002, in response to the data correction instruction corresponding to the data correction option, enter the data correction configuration interface, the data correction configuration interface displays a list of data correction processing rules, and the data to be processed are corrected according to the processing rule implementation details configured in the general rules, date data special rules and numerical special rules in the data correction processing rule list, and the processed first data is obtained, and then transferred to the next data processing option.

[0125] Step 1003, in response to the data cleaning instruction corresponding to the data cleaning option, enter the data cleaning configuration interface, configure at least one target data cleaning processing rule in the data cleaning processing rule list according to the data type and data content of the first data, activate and enable the target data cleaning processing rule, perform cleaning processing on the first data, screen and eliminate abnormal data in the data to be processed, and transfer the cleaned second data to the next node.

[0126] Step 1004, in response to the threshold alarm instruction corresponding to the threshold alarm option, enter the threshold alarm configuration interface, the threshold alarm configuration interface displays a threshold alarm processing rule list, according to the data type and data content of the second data, configure at least one threshold alarm processing rule in the processing rule list of the threshold alarm configuration interface, activate and enable the target data alarm processing rule, classify and count the amount of abnormal data in the second data in batches, if the amount of abnormal data is greater than the preset threshold, issue an alarm and modify the second data according to the alarm prompt until the alarm is canceled, and transfer the processed third data to the next node.

[0127] Step 1005, in response to the data conversion instruction corresponding to the data conversion option, enter the data conversion configuration interface, the data conversion configuration interface displays a list of data conversion processing rules, and at least one data conversion processing rule is determined in the data conversion rule list according to the data type and data content of the third data as the target data conversion processing rule, and the target data conversion processing rule is activated to perform conversion processing on the third data, establish a one-to-one mapping relationship between the integration layer fields in the third data and the standard layer fields in the database, and import the integration layer fields in the third data into the standard layer of the database according to the one-to-one mapping relationship.

[0128] It should be understood that although Figure 1-2 , Figure 4 , Figure 6 , Figure 8 , Fig.10 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-2 , Figure 4 , Figure 6 , Figure 8 , Fig.10 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0129] In one embodiment, Fig.11 As shown, a data processing device 1100 is provided, comprising: a data initial configuration module 1110, a target configuration module 1120 and a data processing module 1130, wherein:

[0130] The initial configuration module 1110 is used to enter the data processing initial configuration interface in response to the received data processing instruction, and the data processing initial configuration interface displays a data processing option list, and the data processing option list includes different types of data processing options;

[0131] A target configuration module 1120, for entering a target data processing configuration interface in response to a target data processing instruction corresponding to a determined target data processing option, wherein the target data processing option is a data processing option determined in a data processing option list according to a data processing requirement;

[0132] The data processing module 1130 is used to determine the target data processing rule in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and process the data to be processed based on the target data processing rule.

[0133] The above-mentioned data processing device 1100, wherein the initial configuration module 1110 enters the data processing initial configuration interface in response to the received data processing instruction, and the data processing initial configuration interface displays a data processing option list, and the data processing option list includes different types of data processing options; the target configuration module 1120 enters the target data processing configuration interface in response to the target data processing instruction corresponding to the determined target data processing option, and the target data processing option is a data processing option determined in the data processing option list according to the data processing requirements; the data processing module 1130 determines the target data processing rule in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and processes the data to be processed based on the target data processing rule. The device can be used to implement autonomous configuration of data processing types and data processing rules, so that data processing is more flexible and each data processing option is independent, and weak coupling is embodied in the process of freely configuring multiple data processing (type) options.

[0134] In one embodiment, the different types of data processing options included in the data processing option list specifically include: data correction options, data cleaning options, data conversion options and threshold alarm options, wherein each data processing option serves as a data processing node, and circulates the data to be processed based on the order of target data processing options in the received target configuration request, and the target configuration request contains at least one target data processing option.

[0135] In one embodiment, the target data processing option is a data correction option, and the target configuration module 1120 is specifically used to respond to the data correction instruction corresponding to the data correction option to enter the data correction configuration interface. The data correction configuration interface displays a list of data correction processing rules, and the data correction processing rule list includes general rules, date data special rules and numerical data special rules.

[0136] The data processing module 1130 is specifically used to determine at least one data correction processing rule in the data correction processing rule list as the target data correction processing rule according to the data type and data content of the data to be processed;

[0137] Activate and enable the target data correction processing rules to correct the data to be processed;

[0138] If the target data correction processing rules include general rules, the full amount of data to be processed is processed according to the full amount of data format information specified in the general rules, so that the processed data to be processed all meet the full amount of data format specified in the general rules;

[0139] If the target data correction processing rules include special rules for date data, the date data in the data to be processed is specially processed according to the special rules for date data, and the date data is corrected to the standard date format preset in the special rules for date data;

[0140] If the target data correction processing rule includes a special numerical rule, the numerical data column in the data to be processed is identified and verified according to the special numerical rule, and the non-numerical data in the numerical data column is converted into numerical data;

[0141] The data processed according to the target data correction processing rules will be transferred to the next node.

[0142] In one embodiment, the target data processing option is a data cleaning option, and the target configuration module 1120 is specifically used to respond to a data cleaning instruction corresponding to the data cleaning option and enter a data cleaning configuration interface, which displays a list of data cleaning processing rules.

[0143] The data processing module 1130 is specifically used to determine at least one data cleaning processing rule in the data cleaning processing rule list as the target data processing rule according to the data type and data content of the data to be processed;

[0144] Activate and enable the target data cleaning processing rules, clean the data to be processed, filter out abnormal data in the data to be processed, and transfer the cleaned data to the next node.

[0145] In one embodiment, the target data processing option is a threshold alarm option, and the target configuration module 1120 is specifically used to respond to the threshold alarm instruction corresponding to the threshold alarm option and enter the threshold alarm configuration interface, which displays a threshold alarm processing rule list.

[0146] The data processing module 1130 is specifically used to determine at least one threshold alarm processing rule in the processing rule list of the threshold alarm configuration interface as the target data alarm processing rule according to the data type and data content of the data to be processed;

[0147] Activate the target data alarm processing rules, classify and count the amount of abnormal data in the data to be processed, and issue an alarm if the amount of abnormal data is greater than the preset threshold;

[0148] Modify the data to be processed according to the alarm prompt until the alarm is cancelled, and transfer the processed normal data to the next node.

[0149] In one embodiment, the target data processing option is a data conversion option, and the target configuration module 1120 is specifically used to respond to the data conversion instruction corresponding to the data conversion option and enter the data conversion configuration interface, which displays a list of data conversion processing rules;

[0150] According to the data type and data content of the data to be processed, at least one data conversion processing rule is determined in the data conversion rule list as the target data conversion processing rule;

[0151] Activate and enable the target data conversion processing rules, convert the data to be processed, establish a one-to-one mapping relationship between the integration layer fields in the data to be processed and the standard layer fields in the database, and import the integration layer fields in the data to be processed into the standard layer of the database based on the one-to-one mapping relationship.

[0152] For the specific definition of the data processing device, please refer to the definition of the data processing method above, which will not be repeated here. Each module in the above data processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0153] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a data processing method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0154] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0155] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0156] In response to the received data processing instruction, entering the data processing initial configuration interface, the data processing initial configuration interface displays a data processing option list, the data processing option list includes different types of data processing options;

[0157] In response to the target data processing instruction corresponding to the determined target data processing option, entering the target data processing configuration interface, the target data processing option being the data processing option determined in the data processing option list according to the data processing requirement;

[0158] The target data processing rule is determined in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and the data to be processed is processed based on the target data processing rule.

[0159] In one embodiment, the different types of data processing options included in the data processing option list specifically include: data correction options, data cleaning options, data conversion options and threshold alarm options, wherein each data processing option serves as a data processing node, and circulates the data to be processed based on the order of target data processing options in the received target configuration request, and the target configuration request contains at least one target data processing option.

[0160] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0161] In response to the data correction instruction corresponding to the data correction option, a data correction configuration interface is entered, and a data correction processing rule list is displayed on the data correction configuration interface. The data correction processing rule list includes general rules, date data special rules and numerical data special rules;

[0162] According to the data type and data content of the data to be processed, determining at least one data correction processing rule in the data correction processing rule list as the target data correction processing rule;

[0163] Activate and enable the target data correction processing rules to correct the data to be processed;

[0164] If the target data correction processing rules include general rules, the full amount of data to be processed is processed according to the full amount of data format information specified in the general rules, so that the processed data to be processed all meet the full amount of data format specified in the general rules;

[0165] If the target data correction processing rules include special rules for date data, the date data in the data to be processed is specially processed according to the special rules for date data, and the date data is corrected to the standard date format preset in the special rules for date data;

[0166] If the target data correction processing rule includes a special numerical rule, the numerical data column in the data to be processed is identified and verified according to the special numerical rule, and the non-numerical data in the numerical data column is converted into numerical data;

[0167] The data processed according to the target data correction processing rules will be transferred to the next node.

[0168] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0169] In response to a data cleaning instruction corresponding to the data cleaning option, a data cleaning configuration interface is entered, and a list of data cleaning processing rules is displayed on the data cleaning configuration interface;

[0170] According to the data type and data content of the data to be processed, at least one data cleaning processing rule is determined in the data cleaning processing rule list as the target data processing rule;

[0171] Activate and enable the target data cleaning processing rules, clean the data to be processed, filter out abnormal data in the data to be processed, and transfer the cleaned data to the next node.

[0172] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0173] In response to the threshold alarm instruction corresponding to the threshold alarm option, a threshold alarm configuration interface is entered, and the threshold alarm configuration interface displays a list of threshold alarm processing rules;

[0174] According to the data type and data content of the data to be processed, at least one threshold alarm processing rule is determined in the processing rule list of the threshold alarm configuration interface as the target data alarm processing rule;

[0175] Activate the target data alarm processing rules, classify and count the amount of abnormal data in the data to be processed, and issue an alarm if the amount of abnormal data is greater than the preset threshold;

[0176] Modify the data to be processed according to the alarm prompt until the alarm is cancelled, and transfer the processed normal data to the next node.

[0177] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0178] In response to the data conversion instruction corresponding to the data conversion option, enter the data conversion configuration interface, which displays a list of data conversion processing rules;

[0179] According to the data type and data content of the data to be processed, at least one data conversion processing rule is determined in the data conversion rule list as the target data conversion processing rule;

[0180] Activate and enable the target data conversion processing rules, convert the data to be processed, establish a one-to-one mapping relationship between the integration layer fields in the data to be processed and the standard layer fields in the database, and import the integration layer fields in the data to be processed into the standard layer of the database based on the one-to-one mapping relationship.

[0181] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0182] In response to the received data processing instruction, entering the data processing initial configuration interface, the data processing initial configuration interface displays a data processing option list, the data processing option list includes different types of data processing options;

[0183] In response to the target data processing instruction corresponding to the determined target data processing option, entering the target data processing configuration interface, the target data processing option being the data processing option determined in the data processing option list according to the data processing requirement;

[0184] The target data processing rule is determined in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and the data to be processed is processed based on the target data processing rule.

[0185] In one embodiment, the different types of data processing options included in the data processing option list specifically include: data correction options, data cleaning options, data conversion options and threshold alarm options, wherein each data processing option serves as a data processing node, and circulates the data to be processed based on the order of target data processing options in the received target configuration request, and the target configuration request contains at least one target data processing option.

[0186] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0187] In response to the data correction instruction corresponding to the data correction option, a data correction configuration interface is entered, and a data correction processing rule list is displayed on the data correction configuration interface. The data correction processing rule list includes general rules, date data special rules and numerical data special rules;

[0188] According to the data type and data content of the data to be processed, determining at least one data correction processing rule in the data correction processing rule list as the target data correction processing rule;

[0189] Activate and enable the target data correction processing rules to correct the data to be processed;

[0190] If the target data correction processing rules include general rules, the full amount of data to be processed is processed according to the full amount of data format information specified in the general rules, so that the processed data to be processed all meet the full amount of data format specified in the general rules;

[0191] If the target data correction processing rules include special rules for date data, the date data in the data to be processed is specially processed according to the special rules for date data, and the date data is corrected to the standard date format preset in the special rules for date data;

[0192] If the target data correction processing rule includes a special numerical rule, the numerical data column in the data to be processed is identified and verified according to the special numerical rule, and the non-numerical data in the numerical data column is converted into numerical data;

[0193] The data processed according to the target data correction processing rules will be transferred to the next node.

[0194] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0195] In response to a data cleaning instruction corresponding to the data cleaning option, a data cleaning configuration interface is entered, and a list of data cleaning processing rules is displayed on the data cleaning configuration interface;

[0196] According to the data type and data content of the data to be processed, at least one data cleaning processing rule is determined in the data cleaning processing rule list as the target data processing rule;

[0197] Activate and enable the target data cleaning processing rules, clean the data to be processed, filter out abnormal data in the data to be processed, and transfer the cleaned data to the next node.

[0198] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0199] In response to the threshold alarm instruction corresponding to the threshold alarm option, a threshold alarm configuration interface is entered, and the threshold alarm configuration interface displays a list of threshold alarm processing rules;

[0200] According to the data type and data content of the data to be processed, at least one threshold alarm processing rule is determined in the processing rule list of the threshold alarm configuration interface as the target data alarm processing rule;

[0201] Activate the target data alarm processing rules, classify and count the amount of abnormal data in the data to be processed, and issue an alarm if the amount of abnormal data is greater than the preset threshold;

[0202] Modify the data to be processed according to the alarm prompt until the alarm is cancelled, and transfer the processed normal data to the next node.

[0203] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0204] In response to the data conversion instruction corresponding to the data conversion option, enter the data conversion configuration interface, which displays a list of data conversion processing rules;

[0205] According to the data type and data content of the data to be processed, at least one data conversion processing rule is determined in the data conversion rule list as the target data conversion processing rule;

[0206] Activate and enable the target data conversion processing rules, convert the data to be processed, establish a one-to-one mapping relationship between the integration layer fields in the data to be processed and the standard layer fields in the database, and import the integration layer fields in the data to be processed into the standard layer of the database based on the one-to-one mapping relationship.

[0207] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0208] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0209] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A data processing method, characterized in that: The method comprises: In response to the received data processing instruction, entering a data processing initial configuration interface, wherein the data processing initial configuration interface displays a data processing option list, wherein the data processing option list includes different types of data processing options; Entering a target data processing configuration interface in response to a target data processing instruction corresponding to a determined target data processing option, wherein the target data processing option is a data processing option determined in the data processing option list according to a data processing requirement; Determine the target data processing rule in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and process the data to be processed based on the target data processing rule; The different types of data processing options included in the data processing option list specifically include: data correction options, data cleaning options, data conversion options and threshold alarm options, wherein each of the data processing options serves as a data processing node, and circulates the data to be processed based on the target data processing option sequence in the received target configuration request, and the target configuration request contains at least one target data processing option.

2. The method according to claim 1, characterized in that The target data processing option is a data correction option; the target data processing instruction corresponding to the determined target data processing option is responded to, and a target data processing configuration interface is entered, wherein the target data processing option is a data processing option determined according to data processing requirements in the data processing option list, including: In response to the data correction instruction corresponding to the data correction option, a data correction configuration interface is entered, wherein the data correction configuration interface displays a list of data correction processing rules, wherein the list of data correction processing rules includes general rules, date data special rules, and numerical data special rules; The step of determining a target data processing rule in a processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and processing the data to be processed based on the target data processing rule includes: According to the data type and data content of the data to be processed, at least one data correction processing rule is determined in the data correction processing rule list as a target data correction processing rule; Activate and enable the target data correction processing rule to perform correction processing on the data to be processed; If the target data correction processing rule includes the general rule, the full amount of the data to be processed is processed according to the full amount of data format information specified in the general rule, so that the processed data to be processed all meet the full amount of data format specified in the general rule; If the target data correction processing rule includes the date data special rule, the date data in the data to be processed is specially processed according to the date data special rule, and the date data is corrected to the standard date format preset in the date data special rule; If the target data correction processing rule includes a numerical special rule, the numerical data column in the to-be-processed data is identified and verified according to the numerical special rule, and the non-numerical data in the numerical data column is converted into numerical data; The data processed according to the target data correction processing rule is transferred to the next node.

3. The method according to claim 1, characterized in that The target data processing option is a data cleaning option; the target data processing instruction corresponding to the determined target data processing option is responded to, and a target data processing configuration interface is entered, wherein the target data processing option is a data processing option determined according to data processing requirements in the data processing option list, including: In response to a data cleaning instruction corresponding to the data cleaning option, entering a data cleaning configuration interface, wherein the data cleaning configuration interface displays a list of data cleaning processing rules; The step of determining a target data processing rule in a processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and processing the data to be processed based on the target data processing rule includes: According to the data type and data content of the data to be processed, at least one data cleaning processing rule is determined in the data cleaning processing rule list as a target data processing rule; Activate and enable the target data cleaning processing rule, perform cleaning processing on the data to be processed, filter out abnormal data in the data to be processed, and transfer the cleaned data to the next node.

4. The method according to claim 1, characterized in that: The target data processing option is a threshold alarm option; the target data processing instruction corresponding to the determined target data processing option is responded to, and a target data processing configuration interface is entered, wherein the target data processing option is a data processing option determined according to data processing requirements in the data processing option list, including: In response to a threshold alarm instruction corresponding to the threshold alarm option, entering a threshold alarm configuration interface, the threshold alarm configuration interface displays a list of threshold alarm processing rules; The step of determining a target data processing rule in a processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and processing the data to be processed based on the target data processing rule includes: According to the data type and data content of the data to be processed, determining at least one threshold alarm processing rule in the processing rule list in the threshold alarm configuration interface as the target data alarm processing rule; Activate and enable the target data alarm processing rule, classify and count the amount of abnormal data in the data to be processed, and issue an alarm if the amount of abnormal data is greater than a preset threshold; The data to be processed is modified according to the alarm prompt until the alarm is cancelled, and the processed normal data is transferred to the next node.

5. The method according to claim 1, characterized in that: The target data processing option is a data conversion option; the target data processing instruction corresponding to the determined target data processing option is responded to, and a target data processing configuration interface is entered, wherein the target data processing option is a data processing option determined according to data processing requirements in the data processing option list, including: In response to a data conversion instruction corresponding to the data conversion option, entering a data conversion configuration interface, the data conversion configuration interface displays a list of data conversion processing rules; The step of determining a target data processing rule in a processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and processing the data to be processed based on the target data processing rule includes: According to the data type and data content of the data to be processed, at least one data conversion processing rule is determined in the data conversion rule list as a target data conversion processing rule; Activate and enable the target data conversion processing rules, perform conversion processing on the data to be processed, establish a one-to-one mapping relationship between the integration layer fields in the data to be processed and the standard layer fields in the database, and import the integration layer fields in the data to be processed into the standard layer of the database according to the one-to-one mapping relationship.

6. A data processing device, characterized in that: The device comprises: An initial configuration module, for entering a data processing initial configuration interface in response to a received data processing instruction, wherein the data processing initial configuration interface displays a data processing option list, wherein the data processing option list includes different types of data processing options; a target configuration module, for entering a target data processing configuration interface in response to a target data processing instruction corresponding to a determined target data processing option, wherein the target data processing option is a data processing option determined according to a data processing requirement in the data processing option list; A data processing module, used to determine a target data processing rule in the processing rule list of the target data processing configuration interface according to the data type and data content of the data to be processed, and process the data to be processed based on the target data processing rule; The different types of data processing options included in the data processing option list specifically include: data correction options, data cleaning options, data conversion options and threshold alarm options, wherein each of the data processing options serves as a data processing node, and circulates the data to be processed based on the target data processing option sequence in the received target configuration request, and the target configuration request contains at least one target data processing option.

7. The device according to claim 6, characterized in that The target data processing option is a data correction option; the target configuration module is specifically used to respond to the data correction instruction corresponding to the data correction option and enter the data correction configuration interface, the data correction configuration interface displays a data correction processing rule list, and the data correction processing rule list includes general rules, date data special rules and numerical data special rules; The data processing module is specifically used to determine at least one data correction processing rule in the data correction processing rule list as a target data correction processing rule according to the data type and data content of the data to be processed; Activate and enable the target data correction processing rule to perform correction processing on the data to be processed; If the target data correction processing rule includes the general rule, the full amount of the data to be processed is processed according to the full amount of data format information specified in the general rule, so that the processed data to be processed all meet the full amount of data format specified in the general rule; If the target data correction processing rule includes the date data special rule, the date data in the data to be processed is specially processed according to the date data special rule, and the date data is corrected to the standard date format preset in the date data special rule; If the target data correction processing rule includes a numerical special rule, the numerical data column in the to-be-processed data is identified and verified according to the numerical special rule, and the non-numerical data in the numerical data column is converted into numerical data; The data processed according to the target data correction processing rule is transferred to the next node.

8. The device according to claim 6, characterized in that The target data processing option is a data cleaning option; the target configuration module is specifically used to respond to the data cleaning instruction corresponding to the data cleaning option and enter the data cleaning configuration interface, wherein the data cleaning configuration interface displays a list of data cleaning processing rules; The data processing module is specifically used to determine at least one data cleaning processing rule in the data cleaning processing rule list as a target data processing rule according to the data type and data content of the data to be processed; Activate and enable the target data cleaning processing rule, perform cleaning processing on the data to be processed, filter out abnormal data in the data to be processed, and transfer the cleaned data to the next node.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images