Data processing method, device and server based on data anomaly

Abstract

The present invention discloses a data processing method, apparatus and server based on data anomalies. The method includes: processing upstream data based on a pre-designed calculation logic to obtain calculation result data; determining whether there are abnormal results in the calculation result data; if so, extracting and storing first upstream data associated with the abnormal results from the upstream data; correcting the abnormal data in the first upstream data to obtain second upstream data; and re-executing the pre-designed calculation logic based on the second upstream data to obtain correct calculation results. The above solution can timely roll back abnormal data when big data calculation results are abnormal, obtain correct calculation results, and does not prevent the processing of correct calculation results during the process. Therefore, by independently correcting and recalculating abnormal data, data self-healing can be efficiently and quickly performed.

Description

Technical Field

[0001] The present invention relates to data processing technologies, and more specifically, to a data processing method, apparatus, and server based on data anomalies. Background Art

[0002] In the big data calculation of fund returns, when upstream data participating in the calculation is missing, or there are errors in the business logic of the system itself, various abnormal scenarios may occur, such as the termination of the calculation program, the storage of dirty data in the database, and the inability to update user fund returns in a timely manner.

[0003] When some enterprises face scenarios of large-scale abnormal fund return calculations, they usually adopt a relatively large number of manual intervention measures. For example, first, conduct a search and analysis judgment of abnormal data, propose a solution, then manually delete abnormal database data through scripts, and finally recalculate and store all data. However, taking manual intervention and manually correcting database data not only affects the timeliness of fund return calculations but also brings certain risks to the accuracy of data repair. Summary of the Invention

[0004] In view of this, the present invention provides the following technical solutions:

[0005] A data processing method based on data anomalies, including:

[0006] Process upstream data based on pre-designed calculation logic to obtain calculation result data;

[0007] Determine whether there are abnormal results in the calculation result data;

[0008] If so, extract and store the first upstream data associated with the abnormal results from the upstream data;

[0009] Correct the abnormal data in the first upstream data to obtain second upstream data;

[0010] Re-execute the pre-designed calculation logic based on the second upstream data to obtain the correct calculation result.

[0011] Optionally, before processing the upstream data based on the pre-designed calculation logic to obtain the calculation result data, it further includes:

[0012] Obtain a parameter configuration table, where the parameter configuration table includes algorithm operation configuration information;

[0013] Read the corresponding upstream data based on the parameter configuration table.

[0014] Optionally, the obtaining of the parameter configuration table includes:

[0015] Read the parameter configuration table based on the set parameter ID, where the parameter ID is the unique identifier providing algorithm parameter configuration information.

[0016] Optionally, it further includes:

[0017] Set the first upstream data as the latest upstream data reading source in the parameter configuration table.

[0018] Optionally, the abnormal result includes a calculation abnormal result, and further includes:

[0019] Recycle the calculation abnormal result and store the abnormal result in a temporary table;

[0020] Store the non-abnormal result in the calculation result data in a result table.

[0021] Optionally, the abnormal result includes a service abnormal result, and further includes:

[0022] Store the service abnormal result in a result table.

[0023] Optionally, after re-executing the pre-designed calculation logic based on the second upstream data to obtain a correct calculation result, it further includes:

[0024] Overwrite the corresponding service abnormal result in the result table with the correct calculation result.

[0025] Optionally, correcting the abnormal data in the first upstream data to obtain the second upstream data includes:

[0026] Correct the abnormal data in the first upstream data based on the user's input information to obtain the second upstream data.

[0027] This application also discloses a data processing device based on data anomalies, including:

[0028] A data calculation module for processing upstream data based on a pre-designed calculation logic to obtain calculation result data;

[0029] An anomaly determination module for determining whether there is an abnormal result in the calculation result data;

[0030] A data extraction module for extracting and storing the first upstream data associated with the abnormal result from the upstream data when the determination result of the anomaly determination module is that there is an anomaly;

[0031] A data correction module for correcting the abnormal data in the first upstream data to obtain the second upstream data;

[0032] The data calculation module is further configured to: re-execute the pre-designed calculation logic based on the second upstream data to obtain a correct calculation result.

[0033] Furthermore, the present application also discloses a server, including:

[0034] a processor;

[0035] a memory for storing executable instructions of the processor;

[0036] Wherein, the executable instructions include: processing upstream data based on a pre-designed calculation logic to obtain calculation result data; determining whether there is an abnormal result in the calculation result data; if so, extracting and storing first upstream data associated with the abnormal result from the upstream data; correcting the abnormal data in the first upstream data to obtain second upstream data; and re-executing the pre-designed calculation logic based on the second upstream data to obtain a correct calculation result.

[0037] As can be seen from the above technical solutions, embodiments of the present invention disclose a data processing method, apparatus and server based on data anomalies. The method includes: processing upstream data based on a pre-designed calculation logic to obtain calculation result data; determining whether there is an abnormal result in the calculation result data; if so, extracting and storing first upstream data associated with the abnormal result from the upstream data; correcting the abnormal data in the first upstream data to obtain second upstream data; and re-executing the pre-designed calculation logic based on the second upstream data to obtain a correct calculation result. The above solution can roll back abnormal data in a timely manner when the big data calculation result is abnormal, obtain a correct calculation result, and the process will not interfere with the processing of the correct calculation result. Therefore, by independently correcting and recalculating abnormal data, data self-healing can be performed efficiently and quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0039] Figure 1 It is a flowchart of a data processing method based on data anomalies disclosed in an embodiment of the present invention;

[0040] Figure 2 It is a flowchart of obtaining upstream data disclosed in an embodiment of the present invention;

[0041] Figure 3 It is a schematic diagram of the specific implementation process of a data processing method based on data anomalies disclosed in an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of a data processing device based on data anomalies disclosed in an embodiment of the present invention;

[0043] Figure 5 Schematic diagram of a server disclosed in an embodiment of the present application. Detailed implementation manners

[0044] For the sake of citation and clarity, the explanations, abbreviations or acronyms of professional terms and technical nouns used hereinafter are summarized as follows:

[0045] Daily calculation: A calculation task executed according to natural days.

[0046] Data recalculation: A calculation task executed according to historical dates, usually occurring in scenarios where natural day calculations are incorrect and data repair and rerunning are required.

[0047] Data source switching: A data calculation program may need to read data from multiple data sources. For example, a calculation task may read data from various databases such as Oracle, MySQL, and GreenPlum.

[0048] Upstream table switching: For this device, the upstream data tables can be dynamically switched. For example, if it is specified in the parameters to read the dividend data table A, the device will obtain dividend data from table A after startup; the next time it is specified in the parameters to read the dividend data table B, the device will similarly switch to table B to obtain dividend data.

[0049] Clearing institution: An institution that provides fund data.

[0050] Dirty data storage in the database: Dirty data refers to incorrect data, which means that due to upstream data errors or problems with the system itself, calculation errors occur, and ultimately incorrect data is stored in the database.

[0051] Calculation anomaly: It refers to an error that can be detected when the program is performing operations. For example, a division by zero operation in mathematical calculations.

[0052] Business anomaly: It refers to an error that the program cannot identify and requires manual discovery. For example, when calculating the fund income on a certain day, the transaction data involved in the calculation should have been 10 shares of the fund bought, but due to a mistake in the upstream data push, this data became 100 shares of the fund bought.

[0053] Data rollback: Data rollback refers to the act of restoring data to the previous correct state when data processing is incorrect.

[0054] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

[0055] Figure 1 It is a flowchart of a data processing method based on data anomalies disclosed in an embodiment of the present invention. Refer to Figure 1 As shown, the data processing method based on data anomalies may include:

[0056] Step 101: Process the upstream data based on the pre-designed calculation logic to obtain the calculation result data.

[0057] Among them, there can be various pre-designed calculation logics, and the upstream data and calculation methods required for different calculation logics are also different. For example, one calculation logic is used to calculate the total amount of a user's funds, then the amounts of all his funds need to be added up; another example is that one calculation logic is used to calculate the interest of a user's funds, then the amount of the funds needs to be multiplied by the interest. In this embodiment, the number of types of pre-designed calculation logics is not fixedly limited and can be determined based on the actual application scenario.

[0058] Step 102: Determine whether there is an abnormal result in the calculation result data.

[0059] In this embodiment, the abnormal result may include at least one of a calculation abnormal result and a business abnormal result. Among them, the calculation abnormal result refers to an abnormality that can be recognized by the calculation system. Usually, the code logic in the calculation system can judge, such as a division-by-zero exception, or an exception specified in some specific business scenarios. And the business abnormal result, or the business abnormal data, refers to the situation that the system cannot judge some business anomalies. For example, a user originally bought 100 shares of funds, but the data was incorrect and recorded as buying 10 shares of funds. This situation cannot be distinguished by the system, and the resulting abnormal result or abnormal data is the business abnormal result.

[0060] Step 103: If there is, extract and store the first upstream data associated with the abnormal result from the upstream data.

[0061] If it is determined that there is an abnormal result, relevant processing needs to be carried out in a timely manner to correct the abnormal result to the correct result. In this embodiment, when it is determined that there is an abnormal result, the first upstream data associated with the abnormal result will be extracted and stored from the upstream data, and subsequent correction processing will be carried out specifically for the problematic upstream data.

[0062] Step 104: Correct the abnormal data in the first upstream data to obtain the second upstream data.

[0063] Relevant staff can check the data in the first upstream data to determine which data or which data has problems. After discovering the problem data, that is, the abnormal data, the abnormal data can be corrected manually to facilitate obtaining accurate calculation results based on the corrected data for recalculation.

[0064] Step 105: Re-execute the pre-designed calculation logic based on the second upstream data to obtain the correct calculation result.

[0065] The second upstream data is the correct data after modification. Therefore, executing the pre-designed calculation logic based on the correct data will obtain the correct calculation result.

[0066] When the calculation result of big data is abnormal, the data processing method based on data abnormality in this embodiment can roll back the abnormal data in time to obtain the correct calculation result, and the processing of the correct calculation result will not be hindered during the process. Thus, by independently correcting and recalculating the abnormal data, data self-healing can be carried out efficiently and quickly.

[0067] Figure 2 It is a flowchart for obtaining upstream data disclosed in an embodiment of the present invention. Refer to Figure 2 As shown, in the data processing method based on data abnormality in this embodiment, before processing the upstream data based on the pre-designed calculation logic to obtain the calculation result data, it may further include:

[0068] Step 201: Obtain a parameter configuration table, and the parameter configuration table includes algorithm operation configuration information.

[0069] Before processing the upstream data, it is necessary to obtain the upstream data. In this embodiment, during the process of obtaining the upstream data, it is necessary to first obtain a parameter configuration table, and the specific operation configuration is set in the parameter configuration table, which may include, but is not limited to: batch job type (daily calculation or data recalculation), batch date, data source switch, upstream table switch, batch settlement institution, etc.

[0070] In one implementation, obtaining the parameter configuration table may include: reading the parameter configuration table based on a set parameter ID, where the parameter ID is the unique identifier providing algorithm parameter configuration information. Reading the parameter configuration table is to obtain the details of the batch job configuration. For example, before a program starts, multiple configuration parameters need to be set to tell the system which parameter indicators to rely on after startup, such as the batch job type, batch date, tables involved in the calculation during batch processing, and so on. These parameter information can be as few as a few items, or as many as dozens or even hundreds of items. It is impossible to directly pass such a large number of complex parameters to the system when the program starts. Therefore, in the solution of this application: write these parameter configurations in advance in a row of a data table and assign an id value to this row of data. When the corresponding program starts, pass this simple id to the system, and the system will query the table according to this id to locate the specific row and finally read all the configuration information.

[0071] Step 202: Read the corresponding upstream data based on the parameter configuration table.

[0072] After obtaining the parameter configuration table, according to the parameter configuration information therein, the corresponding upstream data table can be read to synchronize the upstream data.

[0073] The above content details the implementation of obtaining upstream data so that when the system needs to execute the pre-designed calculation logic, it can quickly obtain the upstream data to carry out the calculation and processing work.

[0074] Based on the above content, the data processing method for data anomalies may further include: setting the first upstream data as the latest upstream data reading source in the parameter configuration table. Since the first upstream data is only the upstream data associated with the abnormal result, setting it as the latest upstream data reading source can subsequently perform targeted recalculation on the problem data. Compared with rerunning all the upstream data, this implementation can significantly save a large amount of computing resources and time.

[0075] In one implementation, the abnormal result includes a calculation abnormal result. The data processing method based on data anomalies may further include: recycling the calculation abnormal result, storing the abnormal result in a temporary table; storing the non-abnormal results in the calculation result data in a result table.

[0076] Recycling the identified calculation abnormal result belongs to the implementation of preventing dirty data from being stored in the database. Storing the abnormal result in a temporary table is for facilitating subsequent analysis of the abnormal data in the upstream data. In this implementation, if a calculation anomaly occurs, the system will automatically extract the data involved in this calculation process into the temporary table. The purpose is that when the system recalculates this part of the data, it can directly obtain the data involved in the calculation without having to manually extract the data, improving the processing efficiency.

[0077] For example, calculating the returns of all funds depends on upstream transaction data. Assume this transaction data is stored in Table A. If the transaction data of a certain user in Table A is incorrect and is recognized by the system, then this piece of data of this user will be extracted into Table B, which is a temporary table created by the system. The purpose of extracting it into the temporary table is, on the one hand, to more conveniently troubleshoot problems, and on the other hand, to facilitate correction and recalculation. Suppose the original incorrect data was that a certain user bought 100 shares of a fund. After being drawn into B and it is located that the data is incorrect and should be 200 shares, then it can be directly modified in Table B. Then when the system reruns this piece of data, it only reads Table B for the transaction data and no longer reads Table A.

[0078] In one implementation, the abnormal result includes a business abnormal result. The data processing method based on data abnormality may further include: storing the business abnormal result in a result table.

[0079] If a business exception occurs and the system simply cannot recognize it, it will directly calculate and store the data in the result table as it is. This situation can be recognized manually, and relevant data participating in the calculation can be extracted manually. This part of the extracted data can also be stored in a temporary table.

[0080] Based on the above, after re-executing the pre-designed calculation logic based on the second upstream data and obtaining the correct calculation result, it may further include: covering the corresponding business abnormal result in the result table with the correct calculation result.

[0081] To better understand the foregoing content, an example is given below.

[0082] Assume that there are 1 million pieces of data in the upstream transaction table on a certain day and they are stored in Table A. Then when the system calculates, it will read Table A and take out all these 1 million pieces of data for calculation. As a result, it is found during this process that:

[0083] (1) Xiaoming's transaction data is selling 100 shares of a certain fund, but Xiaoming has never bought this fund before. Then the system will recognize this abnormality and temporarily store this piece of Xiaoming's data in Table B (temporary upstream table) among the 1 million transactions. At the same time, this piece of data of Xiaoming will be skipped and no longer calculated.

[0084] (2) There is also an error in a piece of Xiaohong's data. The transaction table shows that Xiaohong bought 1000 shares of the fund, but actually this data was misrecorded upstream and should actually be 10 shares. However, the system cannot determine the problem and continues the calculation, and also calculates and stores the incorrect result.

[0085] Regarding (1) in the above content, it refers to a calculation exception, and (2) refers to a business exception.

[0086] Next, the developer needs to handle these two types of exceptions. For (1), the developer needs to correct the transaction data of Xiaoming in Table B; for (2), the developer needs to insert the correct transaction data of Xiaohong into B, then configure the transaction data table to be read by the system as B in the parameter configuration table, assign an id to this configuration, and then restart the system. After the system restarts, according to this id, it reads the relevant configuration and knows that the transaction table read during this calculation is B. Then, it will only read the two pieces of data of Xiaoming and Xiaohong this time and recalculate these two pieces. After the calculation is completed, since the data of Xiaohong has been calculated in the result table before, the redundant data of Xiaohong that has been stored in the result before will be deleted, and finally the two correct pieces of data of Xiaohong and Xiaoming will fall into the result table.

[0087] The above example is a simple description of the entire patent process.

[0088] In one implementation, correcting the abnormal data in the first upstream data to obtain the second upstream data may include: correcting the abnormal data in the first upstream data based on the input information of the user to obtain the second upstream data. Wherein, the user may refer to the technical personnel for troubleshooting problem data.

[0089] With the above solution, by using flexible parameter configuration as the basis for data acquisition of the fund data calculation program, when an exception occurs, the program can automatically roll back and recycle the error data and record the associated data. These records can be repeatedly applied to the parameter configuration during the restorative rerun of the program, and the automatic cleaning of the redundant data that has been stored in the database during the rerun can be achieved.

[0090] Figure 3 This is a schematic diagram of the specific implementation process of the data processing method based on data exceptions disclosed in the embodiments of the present invention. Figure 3 Taking the example of fund income calculation to show the data processing process, combined with Figure 3 as shown in

[0091] Regarding the processing of abnormal data in fund income calculation:

[0092] S11. During the execution of fund income calculation, for the calculation abnormal results, data is recycled, and at the same time, the collected abnormal results are stored in the abnormal result table;

[0093] S12. Extract the upstream data associated with the abnormal results and participating in the calculation of the error results, and store them in a specific temporary upstream table;

[0094] S13. For the non-calculation abnormal result data, it is not affected at all, and the calculation continues and is stored in the original database;

[0095] S14. For business exception data that cannot be recognized by the calculation program, it will still be stored in the result table.

[0096] Regarding the repair of abnormal data of fund returns:

[0097] S21. Technicians conduct a troubleshooting of the abnormal causes based on the abnormal result table and the associated upstream table data in steps S11 and S12;

[0098] S22. For business exception data, manually extract the associated upstream data into the upstream data temporary table involved in step S12;

[0099] S23. Add a parameter configuration and set the table name of the upstream temporary table involved in the above step S12 as the new upstream table reading configuration;

[0100] S24. Pass in the parameter id, start the program, and the program reads the parameter to obtain the configuration of the new data source, data table, etc.

[0101] S25. According to the new data table configuration, synchronize the upstream synchronization data involved in this rerun. Clean up the redundant data in the result table based on the users involved in the synchronized data (at this time, the dirty data generated by step S14 is cleaned up).

[0102] The solution described in the embodiments of the present application realizes that when abnormal results occur, it will protect the non-abnormal data and continue the calculation, ensuring that the entire process will not be blocked; while collecting abnormal data, it also recovers the associated upstream data participating in its calculation and stores it in the specified temporary table for use as a new upstream data table when repairing and rerunning the data later; in the implementation, through parameter configuration, the upstream data table for rerunning is specified, and there is no need to manually delete the abnormal result data in the database. When rerunning, the redundant data can be located according to the rerun data and automatically deleted.

[0103] For the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0104] In the above embodiments publicly disclosed by the present invention, the method is described in detail. The method of the present invention can be implemented by means of devices in various forms. Therefore, the present invention also discloses a device, and specific embodiments are given below for detailed description.

[0105] Figure 4 It is a structural schematic diagram of a data processing device based on data anomalies publicly disclosed in the embodiments of the present invention. SeeFigure 4 As shown in Figure 4 , the data processing device 40 based on data anomalies may include:

[0106] A data calculation module 401, configured to process upstream data based on a pre-designed calculation logic to obtain calculation result data.

[0107] An anomaly determination module 402, configured to determine whether there is an abnormal result in the calculation result data.

[0108] A data extraction module 403, configured to extract and store first upstream data associated with the abnormal result from the upstream data when the determination result of the anomaly determination module is that there is an abnormal result.

[0109] A data correction module 404, configured to correct the abnormal data in the first upstream data to obtain second upstream data.

[0110] The data calculation module 401 is further configured to: re-execute the pre-designed calculation logic based on the second upstream data to obtain a correct calculation result.

[0111] When the data processing device based on data anomalies in this embodiment has an anomaly in the big data calculation result, it can roll back the abnormal data in a timely manner to obtain a correct calculation result, and the process will not interfere with the processing of the correct calculation result. Therefore, by independently correcting and recalculating the abnormal data, data self-healing can be performed efficiently and quickly.

[0112] In one implementation, the device may further include: a configuration table acquisition module, configured to acquire a parameter configuration table, where the parameter configuration table includes algorithm operation configuration information; a data reading module, configured to read corresponding upstream data based on the parameter configuration table.

[0113] In one implementation, the configuration table acquisition module may specifically be configured to: acquire a parameter configuration table based on a set parameter ID, where the parameter ID is a unique identifier for providing algorithm parameter configuration information.

[0114] In one implementation, the device may further include: a configuration modification module, configured to set the first upstream data as the latest upstream data reading source in the parameter configuration table.

[0115] In one implementation, when the abnormal result includes a calculation abnormal result, the device may further include: a data storage module, configured to recycle the calculation abnormal result, store the abnormal result in a temporary table; and store the non-abnormal result in the calculation result data in a result table.

[0116] In one implementation, when the abnormal result includes a service abnormal result, the data storage module may further be configured to: store the service abnormal result in a result table.

[0117] In one implementation, the device may further include: a result updating module, configured to overwrite the corresponding abnormal business result in the result table with the correct calculation result.

[0118] In one implementation, the data correction module may be specifically configured to correct abnormal data in the first upstream data based on user input information to obtain second upstream data.

[0119] For the specific implementation of the above data processing device based on data anomaly and its various modules, please refer to the corresponding part of the method embodiment, which will not be repeated here.

[0120] Any one of the data processing devices based on data anomalies described in the above embodiments includes a processor and a memory. The data calculation module, anomaly determination module, data extraction module, data correction module, etc. in the above embodiments are all stored in the memory as program modules, and the processor executes the above program modules stored in the memory to implement corresponding functions.

[0121] The processor includes a kernel, which retrieves the corresponding program module from the memory. One or more kernels can be set, and the processing of the access data can be realized by adjusting the kernel parameters.

[0122] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0123] In an exemplary embodiment, a computer-readable storage medium is also provided, which can be directly loaded into the internal memory of a computer and contains software code. After being loaded and executed by a computer, the computer program can implement the steps shown in any embodiment of the above-mentioned data processing method based on data anomalies.

[0124] In an exemplary embodiment, a computer program product is also provided, which can be directly loaded into the internal memory of a computer, wherein the computer program contains software code, and after being loaded and executed by a computer, the computer program can implement the steps shown in any embodiment of the data processing method based on data anomalies described above.

[0125] Furthermore, an embodiment of the present invention provides a server. Figure 5 This is a schematic diagram of the structure of a server disclosed in an embodiment of the present application. Figure 5As shown, the server 50 includes at least one processor 501, at least one memory 502 connected to the processor, and a bus 503; wherein, the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the above-mentioned data processing method based on data anomalies.

[0126] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same and similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method section.

[0127] It should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0128] The steps of the methods or algorithms described in connection with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0129] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method based on data anomalies, characterized in that, including: processing upstream data based on a pre-designed calculation logic to obtain calculation result data; determining whether there is an abnormal result in the calculation result data; if so, extracting and storing the first upstream data associated with the abnormal result from the upstream data; correcting the abnormal data in the first upstream data to obtain second upstream data; re-executing the pre-designed calculation logic based on the second upstream data to obtain a correct calculation result; the abnormal result includes a business abnormal result and a calculation abnormal result, the calculation abnormal result is an abnormal result recognizable by the calculation system, and the business abnormal result is an abnormal result unrecognizable by the calculation system; the step of if so, extracting and storing the first upstream data associated with the abnormal result from the upstream data includes: in response to the existence of the calculation abnormal result, recovering the calculation abnormal result and storing the calculation abnormal result in a temporary table; storing the business abnormal result, the business normal result, and the calculation normal result in the calculation result data in a result table; after the step of storing the business abnormal result, the business normal result, and the calculation normal result in the calculation result data in the result table, further includes: storing the correct transaction data corresponding to the business abnormal result in the temporary table; after restarting the system, reading the temporary table and repairing the calculation abnormal result in the temporary table correctly to obtain repaired correct data; storing the correct transaction data and the repaired correct data in the result table, and automatically deleting the business abnormal result corresponding to the correct transaction data.

2. The data processing method based on data anomaly according to claim 1, wherein before processing the upstream data based on the pre-designed calculation logic to obtain calculation result data, further includes: obtaining a parameter configuration table, the parameter configuration table includes algorithm operation configuration information; reading the corresponding upstream data based on the parameter configuration table.

3. The data processing method based on data anomaly according to claim 2, wherein the obtaining the parameter configuration table includes: reading the parameter configuration table based on a set parameter ID, the parameter ID is the only identifier providing algorithm parameter configuration information.

4. The data processing method based on data anomalies according to claim 2, wherein further includes: setting the first upstream data as the latest upstream data reading source in the parameter configuration table.

5. The data processing method based on data anomalies according to claim 1, characterized in that after re-executing the pre-designed calculation logic based on the second upstream data to obtain a correct calculation result, further includes: overwriting the business abnormal result corresponding to the correct calculation result in the result table.

6. The data processing method based on data anomaly according to claim 1, wherein the correcting the abnormal data in the first upstream data to obtain second upstream data includes: correcting the abnormal data in the first upstream data based on the user's input information to obtain second upstream data.

7. A data processing device based on data anomalies, characterized in that, including: a data calculation module for processing upstream data based on a pre-designed calculation logic to obtain calculation result data; an abnormal determination module for determining whether there is an abnormal result in the calculation result data; a data extraction module for extracting and storing the first upstream data associated with the abnormal result from the upstream data when the determination result of the abnormal determination module is yes; a data correction module for correcting the abnormal data in the first upstream data to obtain second upstream data; The data calculation module is further configured to: re - execute the pre - designed calculation logic based on the second upstream data to obtain a correct calculation result; The abnormal results include business abnormal results and calculation abnormal results. The calculation abnormal results are abnormal results that can be recognized by the calculation system, and the business abnormal results are abnormal results that cannot be recognized by the calculation system. The data extraction module is further configured to: In response to the existence of the calculation abnormal result, recycle the calculation abnormal result and store the calculation abnormal result in a temporary table; Store the business abnormal results, business normal results, and calculation normal results in the calculation result data in a result table; The data extraction module is further configured to: Store the correct transaction data corresponding to the business abnormal result in the temporary table; After restarting the system, read the temporary table and repair the calculation abnormal results in the temporary table to be correct to obtain repaired - correct data; Store the correct transaction data and the repaired - correct data in the result table, and automatically delete the business abnormal result corresponding to the correct transaction data.

8. A server, characterized in that, Comprising: A processor; A memory for storing executable instructions of the processor; Wherein, the executable instructions include: processing upstream data based on a pre - designed calculation logic to obtain calculation result data; determining whether there are abnormal results in the calculation result data; if so, extracting and storing first upstream data associated with the abnormal results from the upstream data; correcting the abnormal data in the first upstream data to obtain second upstream data; re - executing the pre - designed calculation logic based on the second upstream data to obtain a correct calculation result; The abnormal results include business abnormal results and calculation abnormal results. The calculation abnormal results are abnormal results that can be recognized by the calculation system, and the business abnormal results are abnormal results that cannot be recognized by the calculation system. The executable instructions further include: In response to the existence of the calculation abnormal result, recycle the calculation abnormal result and store the calculation abnormal result in a temporary table; Store the business abnormal results, business normal results, and calculation normal results in the calculation result data in a result table; The executable instructions further include: Store the correct transaction data corresponding to the business abnormal result in the temporary table; after restarting the system, read the temporary table and repair the calculation abnormal results in the temporary table to be correct to obtain repaired - correct data; Store the correct transaction data and the repaired - correct data in the result table, and automatically delete the business abnormal result corresponding to the correct transaction data.

Images