Flexible Data Processing Method, Apparatus, Storage Medium, and Electronic Device

Abstract

An embodiment of the present application provides a flexible data processing method, apparatus, storage medium, and electronic device. The method includes: obtaining multiple business date year-month tables to be processed; sequentially performing storage processing on the transaction data in the multiple business date year-month tables in a preset order; when performing storage processing on the transaction data in a business date year-month table, determining whether there is an abnormal business date year-month table with abnormal storage processing of transaction data. If so, suspend processing the abnormal business date year-month table and store the abnormal business date year-month table in an abnormal cache queue; sequentially process the business date year-month tables that have not yet undergone storage processing of transaction data in a preset order; if all business date year-month tables except the abnormal business date year-month tables have completed the storage processing of transaction data, obtain the abnormal business date year-month table from the abnormal cache queue and re-perform storage processing on the transaction data in the abnormal business date year-month table.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to a flexible data processing method, device, storage medium and electronic device. Background Art

[0002] In a test environment, a peripheral system needs to process as many transaction data in business date-year-month tables as possible within a specified time (perform warehousing processing on the transaction data in the business date-year-month tables). Among them, the peripheral system can only perform warehousing processing on the transaction data in one business date-year-month table at the same time. In the prior art, after the warehousing processing of the transaction data in the currently processed business date-year-month table by the peripheral system is successful, it will perform warehousing processing on the transaction data in the next business date-year-month table.

[0003] There will be a problem, that is, when the transaction data in the business date-year-month table currently being processed by the peripheral system fails to be warehoused due to an exception (some of the transaction data in the business date-year-month table do not belong to the transaction data in this business date-year-month table, or all of them do not belong to the transaction data in this business date-year-month table. This is because of an error in determining the transaction data corresponding to the business date-year-month table from the natural date-year-month table, or an error in migrating the transaction data from the natural date-year-month table to the business date-year-month table), since the peripheral system needs to successfully warehouse the transaction data in the currently processed business date-year-month table before performing warehousing processing on the transaction data in the next business date-year-month table, and it takes a relatively long time to successfully warehouse the transaction data in the business date-year-month table with abnormal transaction data (it is necessary to check and correct the transaction data in this business date-year-month table, etc.). In a test environment, the peripheral system needs to process as many transaction data in business date-year-month tables as possible within a specified time. If a large amount of time is spent on the business date-year-month table with abnormal data, it will affect the processing efficiency of the peripheral system.

[0004] Therefore, the prior art has defects and needs to be improved and developed. Summary of the Invention

[0005] The embodiments of this application provide a flexible data processing method, device, storage medium and electronic device, which can flexibly process the transaction data in multiple business date-year-month tables and improve the data warehousing volume of the transaction data in the business date-year-month tables by the peripheral system within a limited time.

[0006] The embodiments of this application provide a flexible data processing method, and the method includes:

[0007] Obtain multiple business date year - month tables to be processed, where each of the business date year - month tables contains a batch of transaction data;

[0008] Receive the data files sent by the core system when posting the transaction data in each of the business date year - month tables, and according to the information in the data files, sequentially perform warehousing processing on the transaction data in the business date year - month table corresponding to each data file in a preset order;

[0009] When performing warehousing processing on the transaction data in the business date year - month table, determine whether there is an abnormal business date year - month table with abnormal warehousing processing of transaction data. If so, determine whether the abnormal business date year - month table is the last business date year - month table processed among the multiple business date year - month tables;

[0010] If not, suspend the warehousing processing of the transaction data in the abnormal business date year - month table, and store the abnormal business date year - month table in the abnormal cache queue;

[0011] Sequentially perform warehousing processing on the transaction data in the business date year - month tables that have not undergone warehousing processing of transaction data in the preset order;

[0012] If all business date year - month tables except the abnormal business date year - month table have completed the warehousing processing of transaction data, obtain the abnormal business date year - month table from the abnormal cache queue, and correct the transaction data in the abnormal business date year - month table to obtain a corrected business date year - month table;

[0013] According to the information in the data file corresponding to the corrected business date year - month table, perform warehousing processing on the transaction data in the modified business date year - month table to complete the joint debugging test with the core system.

[0014] In the flexible data processing method described in the embodiments of the present application, after the step of if so, determine whether the abnormal business date year - month table is the last business date year - month table processed among the multiple business date year - month tables, it further includes:

[0015] If so, directly correct the transaction data in the abnormal business date year - month table to obtain a corrected business date year - month table.

[0016] In the flexible data processing method described in the embodiments of the present application, before the step of obtain multiple business date year - month tables to be processed, it further includes:

[0017] Migrate the transaction data in each of the business date-year-month tables from the natural date-year-month table to the corresponding business date-year-month table, where all the transaction data in the business date-year-month tables are recorded in the natural date-year-month table.

[0018] In the flexible data processing method described in the embodiments of the present application, if the transaction data entry processing for all the business date-year-month tables except the abnormal business date-year-month table is completed, then obtain the abnormal business date-year-month table from the abnormal cache queue, and correct the transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table, including:

[0019] Wait until the transaction data entry processing for all the business date-year-month tables is completed, then obtain the abnormal business date-year-month table from the abnormal cache queue;

[0020] Delete the incorrect transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table.

[0021] In the flexible data processing method described in the embodiments of the present application, if the transaction data entry processing for all the business date-year-month tables except the abnormal business date-year-month table is completed, then obtain the abnormal business date-year-month table from the abnormal cache queue, and correct the transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table, further including:

[0022] Wait until the transaction data entry processing for all the business date-year-month tables is completed, then obtain the abnormal business date-year-month table from the abnormal cache queue;

[0023] Delete the incorrect transaction data in the abnormal business date-year-month table, and obtain the correct transaction data from the natural date-year-month table and migrate it to the abnormal business date-year-month table to obtain a corrected business date-year-month table.

[0024] In the flexible data processing method described in the embodiments of the present application, the step of migrating the transaction data in each of the business date-year-month tables from the natural date-year-month table to the corresponding business date-year-month table includes:

[0025] Determine the transaction data in each of the business date-year-month tables from the natural date-year-month table;

[0026] Migrate the transaction data from the natural date-year-month table to the corresponding business date-year-month table.

[0027] In the flexible data processing method described in the embodiments of the present application, if there are multiple abnormal business date-year-month tables cached in the abnormal cache queue, if all the business date-year-month tables except the abnormal business date-year-month tables have completed the storage process of transaction data, then obtain the abnormal business date-year-month tables from the abnormal cache queue, and correct the transaction data in the abnormal business date-year-month tables to obtain corrected business date-year-month tables; including:

[0028] If all the business date-year-month tables except the abnormal business date-year-month tables have completed the storage process of transaction data, then obtain multiple abnormal business date-year-month tables from the abnormal cache queue in the order of their storage in the abnormal cache queue.

[0029] According to the acquisition order of the multiple abnormal business date-year-month tables, correct the transaction data in the multiple abnormal business date-year-month tables in sequence to obtain multiple corrected business date-year-month tables.

[0030] The storage process of the transaction data in the modified business date-year-month table according to the information in the data file corresponding to the modified business date-year-month table and the completion of the joint debugging test with the core system include:

[0031] According to the information in the data file corresponding to each corrected business date-year-month table, and in the generation order of the multiple corrected business date-year-month tables, perform the storage process of the transaction data in the multiple corrected business date-year-month tables in sequence to complete the joint debugging test with the core system.

[0032] The embodiments of the present application further provide a flexible data processing device, the device includes:

[0033] An acquisition module, configured to acquire multiple business date-year-month tables to be processed, where each business date-year-month table contains a batch of transaction data;

[0034] A first processing module, configured to receive the data file sent by the core system when posting the transaction data in each business date-year-month table, and according to the information in the data file, perform the storage process of the transaction data in the business date-year-month table corresponding to each data file in a preset order.

[0035] A determination module, configured to determine whether there is an abnormal business date-year-month table with abnormal storage process of transaction data when performing the storage process of the transaction data in the business date-year-month table. If so, determine whether the abnormal business date-year-month table is the last processed business date-year-month table among the multiple business date-year-month tables.

[0036] A storage module, which is used to, if not, suspend the storage process of the transaction data in the abnormal business date-year-month table and store the abnormal business date-year-month table in an abnormal cache queue;

[0037] A second processing module, which is used to perform the storage process of the transaction data in the business date-year-month table that has not undergone the storage process of transaction data in sequence according to the preset order;

[0038] A calibration module, which is used to, if the storage process of transaction data in all business date-year-month tables except the abnormal business date-year-month table is completed, obtain the abnormal business date-year-month table from the abnormal cache queue, and calibrate the transaction data in the abnormal business date-year-month table to obtain a calibrated business date-year-month table;

[0039] A third processing module, which is used to perform the storage process of the transaction data in the modified business date-year-month table according to the information in the data file corresponding to the calibrated business date-year-month table, and complete the joint debugging test with the core system.

[0040] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer is enabled to execute the flexible data processing method described in any embodiment.

[0041] An embodiment of the present application further provides an electronic device, which includes a processor and a memory. A computer program is stored in the memory. The processor is used to execute the flexible data processing method described in any embodiment by calling the computer program stored in the memory.

[0042] When the embodiment of the present application processes multiple business date-year-month tables in sequence, it determines whether there is an abnormal business date-year-month table with abnormal storage process of transaction data. If so, it stores the abnormal business date-year-month table in an abnormal cache queue, and then first processes the business date-year-month tables that have not undergone the storage process of transaction data in sequence according to the preset order. After the storage process of transaction data in all business date-year-month tables except the abnormal business date-year-month table is completed, it obtains the abnormal business date-year-month table from the abnormal cache queue, then calibrates the transaction data in the abnormal business date-year-month table, and after calibration, performs the storage process of the transaction data in the abnormal business date-year-month table, flexibly processes the transaction data in multiple business date-year-month tables, and improves the data storage volume of the transaction data in the business date-year-month table by the peripheral system within a limited time. Description of the Drawings

[0043] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those skilled in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0044] Figure 1 It is a schematic flowchart of a flexible data processing method provided by an embodiment of the present application.

[0045] Figure 2 It is a schematic structural diagram of a flexible data processing device provided by an embodiment of the present application.

[0046] Figure 3 It is another schematic structural diagram of a flexible data processing device provided by an embodiment of the present application.

[0047] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the protection scope of the present application.

[0049] An embodiment of the present application provides a flexible data processing method, and the flexible data processing method can be applied to an electronic device.

[0050] Please refer to Figure 1 , Figure 1 It is a schematic flowchart of a flexible data processing method provided by an embodiment of the present application. The flexible data processing method is applied to an electronic device, and the method may include the following steps:

[0051] Step 101, obtain multiple business date year-month tables to be processed, where each of the business date year-month tables contains a batch of transaction data.

[0052] Among them, each business date year-month table corresponds to a business date, and for the transaction data in the same business date year-month table, their business dates are the same.

[0053] In some embodiments, before obtaining the multiple business date year-month tables to be processed, it further includes:

[0054] Migrate the transaction data in each of the business date-year-month tables from the natural date-year-month table to the corresponding business date-year-month table, where all the transaction data in all business date-year-month tables are recorded in the natural date-year-month table.

[0055] Among them, all transaction data of all business dates are recorded in the natural date-year-month table, that is, all transaction data in all business date-year-month tables are recorded.

[0056] In some embodiments, the migrating the transaction data in each of the business date-year-month tables from the natural date-year-month table to the corresponding business date-year-month table includes:

[0057] Determine the transaction data in each of the business date-year-month tables from the natural date-year-month table;

[0058] Migrate the transaction data from the natural date-year-month table to the corresponding business date-year-month table.

[0059] Among them, since all transaction data of all business dates are recorded in the natural date-year-month table, therefore, before migrating the transaction data from the natural date-year-month table to the corresponding business date-year-month table, it is necessary to first determine the transaction data corresponding to the business date-year-month table in the natural date-year-month table. Since each transaction data corresponds to a business date, each business date-year-month table corresponds to a business date, and the transaction data in the same business date-year-month table have the same business date, therefore, as long as the business date corresponding to the transaction data is used, the transaction data corresponding to the business date-year-month table can be determined from the natural date-year-month table.

[0060] Step 102: Receive the data file sent by the core system when posting the transaction data in each of the business date-year-month tables, and perform the warehousing process on the transaction data in the business date-year-month table corresponding to each data file in the preset order according to the information in the data file.

[0061] Among them, each business date-year-month table corresponds to a data file, that is, the transaction data in the business date-year-month table is corresponding to the data file. When performing the warehousing process on the transaction data in the business date-year-month table, it is necessary to perform it according to the information in the data file. When the transaction data in the business date-year-month table is incorrect, it will cause the failure of warehousing the transaction data in the business date-year-month table.

[0062] Step 103: When performing the warehousing process on the transaction data in the business date-year-month table, determine whether there is an abnormal business date-year-month table with abnormal warehousing process of transaction data. If so, judge whether the abnormal business date-year-month table is the last processed business date-year-month table among multiple business date-year-month tables;

[0063] Among them, the transaction data warehousing process in the business date-year-month table is abnormal, indicating that there is an error in the transaction data in the business date-year-month table. It may be that some of the transaction data in the business date-year-month table does not belong to the transaction data in this business date-year-month table, or all of them do not belong to the transaction data in this business date-year-month table.

[0064] Step 104, if not, then suspend the warehousing process of the transaction data in the abnormal business date-year-month table, and store the abnormal business date-year-month table in the abnormal cache queue.

[0065] Step 105, perform the warehousing process on the transaction data in the business date-year-month table that has not undergone the transaction data warehousing process in the preset order.

[0066] That is, first suspend the warehousing process of the transaction data in the abnormal business date-year-month table, and give priority to processing the transaction data in other business date-year-month tables, so as to increase the data warehousing volume of the transaction data in the business date-year-month table by the peripheral system within a limited time.

[0067] Step 106, if all the business date-year-month tables except the abnormal business date-year-month table have completed the warehousing process of the transaction data, then obtain the abnormal business date-year-month table from the abnormal cache queue, and correct the transaction data in the abnormal business date-year-month table to obtain the corrected business date-year-month table.

[0068] That is, give priority to performing the warehousing process on the transaction data in the normal business date-year-month table. After all the normal business date-year-month tables have completed the warehousing process of the transaction data, then obtain the abnormal business date-year-month table from the abnormal cache queue and process the abnormal business date-year-month table, so as to increase the data warehousing volume of the transaction data in the business date-year-month table by the peripheral system within a limited time.

[0069] Among them, since the warehousing of the transaction data in the abnormal business date-year-month table fails because the transaction data in the abnormal business date-year-month table is incorrect (some of the transaction data in the abnormal business date-year-month table does not belong to the transaction data in this business date-year-month table, or all of them do not belong to the transaction data in this business date-year-month table), therefore, after obtaining the abnormal business date-year-month table from the abnormal cache queue, it is necessary to correct the transaction data in the abnormal business date-year-month table.

[0070] In some embodiments, if all of the business date-year-month tables except the abnormal business date-year-month table have completed the transaction data storage process, then obtain the abnormal business date-year-month table from the abnormal cache queue, and correct the transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table, including:

[0071] Wait until all of the business date-year-month tables have completed the transaction data storage process, then obtain the abnormal business date-year-month table from the abnormal cache queue;

[0072] Delete the incorrect transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table.

[0073] Wherein, when the storage of the transaction data in the abnormal business date-year-month table fails because some of the transaction data in the abnormal business date-year-month table do not belong to the transaction data in this abnormal business date-year-month table, then delete the incorrect transaction data in the abnormal business date-year-month table to generate a business date-year-month table with all correct transaction data.

[0074] In some embodiments, if all of the business date-year-month tables except the abnormal business date-year-month table have completed the transaction data storage process, then obtain the abnormal business date-year-month table from the abnormal cache queue, and correct the transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table, further including:

[0075] Wait until all of the business date-year-month tables have completed the transaction data storage process, then obtain the abnormal business date-year-month table from the abnormal cache queue;

[0076] Delete the incorrect transaction data in the abnormal business date-year-month table, and obtain the correct transaction data from the natural date-year-month table and migrate it to the abnormal business date-year-month table to obtain a corrected business date-year-month table.

[0077] Wherein, when the storage of the transaction data in the abnormal business date-year-month table fails because all of the transaction data in the abnormal business date-year-month table do not belong to the transaction data in this abnormal business date-year-month table, then delete the incorrect transaction data in the abnormal business date-year-month table, and obtain the correct transaction data from the natural date-year-month table and migrate it to the abnormal business date-year-month table to generate a business date-year-month table with all correct transaction data.

[0078] In some embodiments, if it exists, after determining whether the abnormal business date-year-month table is the last processed business date-year-month table among multiple business date-year-month tables, it further includes:

[0079] If so, directly correct the transaction data in the abnormal business date and month table to obtain a corrected business date and month table.

[0080] Among them, when it is determined that the abnormal business date and month table is the last processed business date and month table among multiple business date and month tables, since it is the last business date and month table to be processed, there is no need to pause the storage process of the transaction data in the abnormal business date and month table, and there is no need to store the abnormal business date and month table in the abnormal cache queue. Just directly correct the transaction data in the abnormal business date and month table to generate a business date and month table with all correct transaction data.

[0081] Step 107: According to the information in the data file corresponding to the corrected business date and month table, perform an inbound process on the transaction data in the modified business date and month table to complete the joint debugging test with the core system.

[0082] In some embodiments, if there are multiple abnormal business date and month tables cached in the abnormal cache queue, if all the business date and month tables except the abnormal business date and month table have completed the inbound process of transaction data, then obtain the abnormal business date and month table from the abnormal cache queue and correct the transaction data in the abnormal business date and month table to obtain a corrected business date and month table; it includes:

[0083] If all the business date and month tables except the abnormal business date and month table have completed the inbound process of transaction data, then sequentially obtain multiple abnormal business date and month tables from the abnormal cache queue according to the order in which the multiple abnormal business date and month tables are stored in the abnormal cache queue;

[0084] According to the acquisition order of the multiple abnormal business date and month tables, sequentially correct the transaction data in the multiple abnormal business date and month tables to obtain multiple corrected business date and month tables;

[0085] The performing an inbound process on the transaction data in the modified business date and month table according to the information in the data file corresponding to the modified business date and month table to complete the joint debugging test with the core system includes:

[0086] According to the information in the data file corresponding to each corrected business date and month table, sequentially perform an inbound process on the transaction data in the multiple corrected business date and month tables according to the generation order of the multiple corrected business date and month tables to complete the joint debugging test with the core system.

[0087] Any combination of the above optional technical solutions can form an optional embodiment of the present application, which will not be elaborated one by one here.

[0088] In specific implementation, the present application is not limited by the execution order of the described steps. Without conflict, some steps can also be performed in other orders or simultaneously.

[0089] As can be seen from the above, in the flexible data processing method provided by the embodiment of the present application, when processing multiple business date-year-month tables in sequence, it is determined whether there is an abnormal business date-year-month table with abnormal transaction data warehousing processing. If so, the abnormal business date-year-month table is stored in the abnormal cache queue. Then, first, the business date-year-month tables that have not undergone transaction data warehousing processing are processed in sequence according to the preset order. After all the business date-year-month tables except the abnormal business date-year-month table have completed the transaction data warehousing processing, the abnormal business date-year-month table is retrieved from the abnormal cache queue, and then the transaction data in the abnormal business date-year-month table is corrected. After correction, the transaction data in the abnormal business date-year-month table is warehoused, flexibly processing the transaction data in multiple business date-year-month tables, and improving the data warehousing volume of the transaction data in the business date-year-month table by the peripheral system within a limited time.

[0090] The embodiment of the present application further provides a flexible data processing device, and the flexible data processing device can be integrated in an electronic device.

[0091] Please refer to Figure 2 , Figure 2 which is a schematic structural diagram of the flexible data processing device provided by the embodiment of the present application. The flexible data processing device 30 may include:

[0092] An obtaining module 31, configured to obtain multiple business date-year-month tables to be processed, where each business date-year-month table contains a batch of transaction data;

[0093] A first processing module 32, configured to receive a data file sent by the core system when posting the transaction data in each business date-year-month table, and perform warehousing processing on the transaction data in the business date-year-month table corresponding to each data file in sequence according to the information in the data file;

[0094] A determining module 33, configured to determine whether there is an abnormal business date-year-month table with abnormal transaction data warehousing processing when performing warehousing processing on the transaction data in the business date-year-month table. If so, it is determined whether the abnormal business date-year-month table is the last business date-year-month table processed among the multiple business date-year-month tables;

[0095] The storage module 34 is used to, if not, suspend the storage process of the transaction data in the abnormal business date-year-month table and store the abnormal business date-year-month table in the abnormal cache queue;

[0096] The second processing module 35 is used to perform the storage process of the transaction data in the business date-year-month table that has not been processed for storing transaction data in sequence according to the preset order;

[0097] The calibration module 36 is used to, if the storage process of the transaction data in all business date-year-month tables except the abnormal business date-year-month table is completed, obtain the abnormal business date-year-month table from the abnormal cache queue and calibrate the transaction data in the abnormal business date-year-month table to obtain a calibrated business date-year-month table;

[0098] The third processing module 37 is used to perform the storage process of the transaction data in the modified business date-year-month table according to the information in the data file corresponding to the calibrated business date-year-month table to complete the joint debugging test with the core system.

[0099] In some embodiments, the calibration module 36 is used to, when the storage process of the transaction data in all business date-year-month tables is completed, obtain the abnormal business date-year-month table from the abnormal cache queue; delete the incorrect transaction data in the abnormal business date-year-month table to obtain a calibrated business date-year-month table.

[0100] In some embodiments, the calibration module 36 is used to, when the storage process of the transaction data in all business date-year-month tables is completed, obtain the abnormal business date-year-month table from the abnormal cache queue; delete the incorrect transaction data in the abnormal business date-year-month table and obtain the correct transaction data from the natural date-year-month table and migrate it to the abnormal business date-year-month table to obtain a calibrated business date-year-month table.

[0101] In some embodiments, the calibration module 36 is used to, if the storage process of the transaction data in all business date-year-month tables except the abnormal business date-year-month table is completed, obtain multiple abnormal business date-year-month tables from the abnormal cache queue in the order of their storage in the abnormal cache queue; calibrate the transaction data in the multiple abnormal business date-year-month tables in the order of their acquisition to obtain multiple calibrated business date-year-month tables.

[0102] In some embodiments, the third processing module 37 is configured to, according to the information in the data file corresponding to each of the corrected business date year-month tables, sequentially perform warehousing processing on the transaction data in the multiple corrected business date year-month tables in the generation order of the multiple corrected business date year-month tables, and complete the joint debugging test with the core system.

[0103] In specific implementation, each of the above modules may be implemented as an independent entity, or may be arbitrarily combined and implemented as the same or several entities.

[0104] As can be seen from the above, the flexible data processing device 30 provided by the embodiment of the present application obtains, through the obtaining module 31, multiple business date year-month tables to be processed, where each business date year-month table contains a batch of transaction data; receives, through the first processing module 32, a data file sent by the core system when performing posting processing on the transaction data in each business date year-month table, and sequentially performs warehousing processing on the transaction data in the business date year-month table corresponding to each data file according to a preset order; determines, through the determining module 33 when performing warehousing processing on the transaction data in the business date year-month table, whether there is an abnormal business date year-month table with abnormal warehousing processing of transaction data. If so, determines whether the abnormal business date year-month table is the last processed business date year-month table among the multiple business date year-month tables; suspends, through the storage module 34, the warehousing processing of the transaction data in the abnormal business date year-month table, and stores the abnormal business date year-month table in an abnormal cache queue; sequentially performs warehousing processing on the transaction data in the business date year-month tables that have not undergone warehousing processing of transaction data through the second processing module 35 according to the preset order; if all the business date year-month tables except the abnormal business date year-month table have completed the warehousing processing of transaction data, obtains, through the correction module 36, the abnormal business date year-month table from the abnormal cache queue, and corrects the transaction data in the abnormal business date year-month table to obtain a corrected business date year-month table; and performs warehousing processing on the transaction data in the modified business date year-month table according to the information in the data file corresponding to the corrected business date year-month table through the third processing module 37, and completes the joint debugging test with the core system.

[0105] Please refer to Figure 3 , Figure 3Another structural schematic diagram of the flexible data processing device provided by the embodiment of the present application. The flexible data processing device 30 includes a memory 120, one or more processors 180, and one or more application programs. The one or more application programs are stored in the memory 120 and are configured to be executed by the processor 180. The processor 180 may include an acquisition module 31, a first processing module 32, a determination module 33, a storage module 34, a second processing module 35, a calibration module 36, and a third processing module 37. For example, the structures and connection relationships of the above components may be as follows:

[0106] The memory 120 can be used to store application programs and data. The application programs stored in the memory 120 contain executable codes. The application programs can form various functional modules. The processor 180 executes various functional applications and data processing by running the application programs stored in the memory 120. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 120 may also include a memory controller to provide the processor 180 with access to the memory 120.

[0107] The processor 180 is the control center of the device, connects various parts of the entire terminal using various interfaces and lines, executes various functions of the device and processes data by running or executing the application programs stored in the memory 120 and calling the data stored in the memory 120, thereby monitoring the device as a whole. Optionally, the processor 180 may include one or more processing cores; preferably, the processor 180 may integrate an application processor and a modulation and demodulation processor. Among them, the application processor mainly processes the operating system, user interface, and application programs, etc.

[0108] Specifically in this embodiment, the processor 180 will load the executable codes corresponding to the processes of one or more application programs into the memory 120 according to the following instructions, and the processor 180 will run the application programs stored in the memory 120 to implement various functions:

[0109] The acquisition module 31 is used to acquire multiple business date year-month tables to be processed. Each of the business date year-month tables contains a batch of transaction data;

[0110] The first processing module 32 is used to receive the data files sent by the core system when posting the transaction data in each of the business date year-month tables, and perform warehousing processing on the transaction data in the business date year-month table corresponding to each of the data files in a preset order according to the information in the data files;

[0111] A determination module 33, configured to determine whether there is an abnormal business date and month table with abnormal transaction data storage processing when performing storage processing on the transaction data in the business date and month table, and if so, determine whether the abnormal business date and month table is the last processed business date and month table among multiple business date and month tables;

[0112] A storage module 34, configured to, if not, suspend the storage processing of the transaction data in the abnormal business date and month table, and store the abnormal business date and month table in an abnormal cache queue;

[0113] A second processing module 35, configured to perform storage processing on the transaction data in the business date and month table that has not undergone transaction data storage processing in the preset order;

[0114] A calibration module 36, configured to, if all business date and month tables except the abnormal business date and month table have completed the storage processing of transaction data, obtain the abnormal business date and month table from the abnormal cache queue, and calibrate the transaction data in the abnormal business date and month table to obtain a calibrated business date and month table;

[0115] A third processing module 37, configured to perform storage processing on the transaction data in the modified business date and month table according to the information in the data file corresponding to the calibrated business date and month table, and complete the joint debugging test with the core system.

[0116] In some embodiments, the calibration module 36 is configured to, after all business date and month tables have completed the storage processing of transaction data, obtain the abnormal business date and month table from the abnormal cache queue; delete the incorrect transaction data in the abnormal business date and month table to obtain a calibrated business date and month table.

[0117] In some embodiments, the calibration module 36 is configured to, after all business date and month tables have completed the storage processing of transaction data, obtain the abnormal business date and month table from the abnormal cache queue; delete the incorrect transaction data in the abnormal business date and month table, and obtain correct transaction data from the natural date and month table and migrate it to the abnormal business date and month table to obtain a calibrated business date and month table.

[0118] In some embodiments, the correction module 36 is configured to, if the transaction data has been stored in all the business date and month tables except the abnormal business date and month tables, obtain the multiple abnormal business date and month tables from the abnormal cache queue in the order of their storage in the abnormal cache queue; and correct the transaction data in the multiple abnormal business date and month tables in the order of their acquisition, so as to obtain multiple corrected business date and month tables.

[0119] In some embodiments, the third processing module 37 is configured to, according to the information in the data file corresponding to each corrected business date and month table, perform the storage process on the transaction data in the multiple corrected business date and month tables in the order of their generation, so as to complete the joint debugging test with the core system.

[0120] An embodiment of the present application further provides an electronic device. Please refer to Figure 4 , Figure 4 which shows a schematic structural diagram of the electronic device provided by the embodiment of the present application. The electronic device can be used to implement the flexible data processing method provided in the above embodiments.

[0121] As Figure 4 shown, the electronic device 1200 may include an RF (Radio Frequency) circuit 110, a memory 120 including one or more (only one is shown in the figure) computer-readable storage media, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a transmission module 170, a processor 180 including one or more (only one is shown in the figure) processing cores, and a power supply 190, etc. Those skilled in the art can understand that Figure 4 the structure of the electronic device 1200 shown in

[0122] does not constitute a limitation on the electronic device 1200, and it may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements. Among them:

[0123] The memory 120 can be used to store software programs and modules, such as the program instructions / modules corresponding to the flexible data processing method in the above embodiments. The processor 180 executes various functional applications and data processing by running the software programs and modules stored in the memory 120, and can flexibly process the transaction data in multiple business date-year-month tables, improving the data storage volume of the transaction data in the business date-year-month table by the peripheral system within a limited time. The memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 120 may further include a memory remotely disposed relative to the processor 180, and these remote memories can be connected to the electronic device 1200 through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.

[0124] The input unit 130 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls. Specifically, the input unit 130 may include a touch-sensitive surface 131 and other input devices 132. The touch-sensitive surface 131, also known as a touch display screen or a touchpad, can collect touch operations of a user thereon or nearby (such as operations of the user using a finger, a stylus, or any suitable object or accessory on or near the touch-sensitive surface 131), and drive corresponding connection devices according to a preset program. Optionally, the touch-sensitive surface 131 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the touch orientation of the user, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 180, and can receive commands sent by the processor 180 and execute them. In addition, multiple types such as resistive, capacitive, infrared, and surface acoustic wave can be used to implement the touch-sensitive surface 131. In addition to the touch-sensitive surface 131, the input unit 130 may further include other input devices 132. Specifically, the other input devices 132 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, joysticks, etc.

[0125] The display unit 140 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the electronic device 1200. These graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. The display unit 140 may include a display panel 141. Optionally, the display panel 141 can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface 131 can cover the display panel 141. When the touch-sensitive surface 131 detects a touch operation on or near it, it is transmitted to the processor 180 to determine the type of touch event. Subsequently, the processor 180 provides a corresponding visual output on the display panel 141 according to the type of touch event. Although in Figure 4 the touch-sensitive surface 131 and the display panel 141 are implemented as two independent components to perform input and output functions, in some embodiments, the touch-sensitive surface 131 and the display panel 141 can be integrated to implement input and output functions.

[0126] The electronic device 1200 may further include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. Among them, the ambient light sensor can adjust the brightness of the display panel 141 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 141 and / or the backlight when the electronic device 1200 is moved to the ear. As a kind of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in all directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used in applications for identifying the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as a pedometer, tapping), etc. As for other sensors that the electronic device 1200 can also be configured with, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., they will not be elaborated here.

[0127] The audio circuit 160, the speaker 161, and the microphone 162 can provide an audio interface between the user and the electronic device 1200. The audio circuit 160 can transmit the electrical signal converted from the received audio data to the speaker 161, and the speaker 161 converts it into a sound signal for output. On the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and converted into audio data. Then, after the audio data is output to the processor 180 for processing, it is sent through the RF circuit 110 to, for example, another terminal, or the audio data is output to the memory 120 for further processing. The audio circuit 160 may also include an earphone jack to provide communication between the peripheral earphone and the electronic device 1200.

[0128] The electronic device 1200 can help users send and receive emails, browse the web, and access streaming media, etc. through the transmission module 170 (such as a Wi-Fi module), which provides users with wireless broadband Internet access. Although Figure 4 the transmission module 170 is shown, it can be understood that it does not belong to an essential component of the electronic device 1200 and can be completely omitted within the scope of not changing the essence of the invention as needed.

[0129] The processor 180 is the control center of the electronic device 1200, connecting various parts of the entire mobile phone through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 120, and by calling data stored in the memory 120, it executes various functions of the electronic device 1200 and processes data. Optionally, the processor 180 may include one or more processing cores; in some embodiments, the processor 180 may integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 180 either.

[0130] The electronic device 1200 further includes a power supply 190 (such as a battery) for powering each component. In some embodiments, the power supply can be logically connected to the processor 180 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 190 may also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0131] Although not shown, the electronic device 1200 may also include a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be elaborated here. Specifically, in this embodiment, the display unit 140 of the electronic device 1200 is a touch screen display, and the electronic device 1200 further includes a memory 120, and one or more programs, where one or more programs are stored in the memory 120 and are configured to be executed by one or more processors 180. The one or more programs include instructions for performing the following operations:

[0132] Obtain multiple business date-year-month tables to be processed, where each of the business date-year-month tables contains a batch of transaction data;

[0133] The receiving core system sends the data files when posting the transaction data in each of the business date-year-month tables, and according to the information in the data files, sequentially processes the transaction data in the business date-year-month tables corresponding to each of the data files for storage in the preset order;

[0134] When processing the transaction data in the business date-year-month table for storage, determine whether there is an abnormal business date-year-month table with abnormal processing of transaction data storage. If so, determine whether the abnormal business date-year-month table is the last processed business date-year-month table among multiple business date-year-month tables;

[0135] If not, suspend the processing of the transaction data in the abnormal business date-year-month table and store the abnormal business date-year-month table in the abnormal cache queue;

[0136] Sequentially process the transaction data in the business date-year-month tables that have not been processed for transaction data storage in the preset order;

[0137] If all business date-year-month tables except the abnormal business date-year-month table have completed the processing of transaction data storage, obtain the abnormal business date-year-month table from the abnormal cache queue, and correct the transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table;

[0138] According to the information in the data file corresponding to the corrected business date-year-month table, process the transaction data in the modified business date-year-month table for storage to complete the joint debugging test with the core system.

[0139] In some embodiments, the processor 180 is used to, if so, directly correct the transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table.

[0140] In some embodiments, the processor 180 is used to migrate the transaction data in each business date-year-month table from the natural date-year-month table to the corresponding business date-year-month table, where all transaction data in all business date-year-month tables are recorded in the natural date-year-month table.

[0141] In some embodiments, the processor 180 is used to wait until all business date-year-month tables have completed the processing of transaction data storage, then obtain the abnormal business date-year-month table from the abnormal cache queue; delete the incorrect transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table.

[0142] In some embodiments, the processor 180 is configured to, when the storage process of the transaction data for all the business date-year-month tables is completed, obtain the abnormal business date-year-month tables from the abnormal cache queue; delete the incorrect transaction data in the abnormal business date-year-month tables, and obtain the correct transaction data from the natural date-year-month tables and migrate it to the abnormal business date-year-month tables to obtain the corrected business date-year-month tables.

[0143] In some embodiments, the processor 180 is configured to determine the transaction data in each of the business date-year-month tables from the natural date-year-month tables; and migrate the transaction data from the natural date-year-month tables to the corresponding business date-year-month tables.

[0144] In some embodiments, the processor 180 is configured to, when the storage process of the transaction data for all the business date-year-month tables except the abnormal business date-year-month tables is completed, obtain multiple abnormal business date-year-month tables from the abnormal cache queue in the order of their storage in the abnormal cache queue; correct the transaction data in the multiple abnormal business date-year-month tables in the order of their acquisition to obtain multiple corrected business date-year-month tables; and according to the information in the data files corresponding to each of the corrected business date-year-month tables, perform the storage process of the transaction data in the multiple corrected business date-year-month tables in the order of their generation to complete the joint debugging test with the core system.

[0145] As described above, the embodiment of the present application provides an electronic device 1200, and the electronic device 1200 performs the following steps: obtaining multiple business date year-month tables to be processed, wherein each of the business date year-month tables contains a batch of transaction data; receiving a data file sent by the core system when posting the transaction data in each of the business date year-month tables, and sequentially performing warehousing processing on the transaction data in the business date year-month table corresponding to each data file according to a preset order; when performing warehousing processing on the transaction data in the business date year-month table, determining whether there is an abnormal business date year-month table with abnormal warehousing processing of the transaction data, and if so, determining whether the abnormal business date year-month table is the last processed business date year-month table among the multiple business date year-month tables; if not, suspending the warehousing processing of the transaction data in the abnormal business date year-month table, and storing the abnormal business date year-month table in an abnormal cache queue; sequentially performing warehousing processing on the transaction data in the business date year-month tables that have not undergone warehousing processing of the transaction data according to the preset order; if all the business date year-month tables except the abnormal business date year-month table have completed the warehousing processing of the transaction data, obtaining the abnormal business date year-month table from the abnormal cache queue, and correcting the transaction data in the abnormal business date year-month table to obtain a corrected business date year-month table; according to the information in the data file corresponding to the corrected business date year-month table, performing warehousing processing on the transaction data in the modified business date year-month table to complete the joint debugging test with the core system.

[0146] The embodiment of the present application further provides a storage medium, in which a computer program is stored. When the computer program runs on a computer, the computer executes the flexible data processing method described in any of the above embodiments.

[0147] It should be noted that for the flexible data processing method of the present application, those of ordinary skill in the art can understand that all or part of the processes of implementing the flexible data processing method described in the embodiments of the present application can be controlled by a computer program to complete the relevant hardware. The computer program can be stored in a computer-readable storage medium, such as stored in the memory of an electronic device and executed by at least one processor in the electronic device. During the execution process, it can include the processes of the embodiments of the flexible data processing method. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM, Read Only Memory), a random access memory (RAM, Random Access Memory), etc.

[0148] For the flexible data processing device according to the embodiments of the present application, its various functional modules may be integrated in a processing chip, or each module may exist physically alone, or two or more modules may be integrated in one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk or an optical disc.

[0149] The flexible data processing method, device, storage medium and electronic device provided by the embodiments of the present application have been introduced in detail above. Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A flexible data processing method, characterized in that, Including: Obtain multiple business date-year-month tables to be processed, where each of the business date-year-month tables contains a batch of transaction data; Receive the data files sent by the core system when posting the transaction data in each of the business date-year-month tables, and perform warehousing processing on the transaction data in the business date-year-month table corresponding to each data file in the preset order according to the information in the data files; When performing warehousing processing on the transaction data in the business date-year-month table, determine whether there is an abnormal business date-year-month table with abnormal warehousing processing of transaction data. If so, determine whether the abnormal business date-year-month table is the last processed business date-year-month table among the multiple business date-year-month tables, where the transaction data in the abnormal business date-year-month table is abnormal; If so, directly correct the transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table; If not, suspend the warehousing processing of the transaction data in the abnormal business date-year-month table and store the abnormal business date-year-month table in the abnormal cache queue; Perform warehousing processing on the transaction data in the business date-year-month tables that have not undergone warehousing processing of transaction data in the preset order; If all the business date-year-month tables except the abnormal business date-year-month table have completed the warehousing processing of transaction data, obtain the abnormal business date-year-month table from the abnormal cache queue and correct the transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table; According to the information in the data file corresponding to the corrected business date-year-month table, perform warehousing processing on the transaction data in the corrected business date-year-month table to complete the joint debugging test with the core system.

2. The flexible data processing method according to claim 1, wherein Before obtaining the multiple business date-year-month tables to be processed, it further includes: Migrate the transaction data in each business date-year-month table from the natural date-year-month table to the corresponding business date-year-month table, where all the transaction data in the business date-year-month tables is recorded in the natural date-year-month table.

3. The flexible data processing method according to claim 2, wherein The step of "If all the business date-year-month tables except the abnormal business date-year-month table have completed the warehousing processing of transaction data, obtain the abnormal business date-year-month table from the abnormal cache queue and correct the transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table" includes: Wait until all the business date-year-month tables have completed the warehousing processing of transaction data, and then obtain the abnormal business date-year-month table from the abnormal cache queue; Delete the incorrect transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table.

4. The flexible data processing method according to claim 2, wherein The step of "If all the business date-year-month tables except the abnormal business date-year-month table have completed the warehousing processing of transaction data, obtain the abnormal business date-year-month table from the abnormal cache queue and correct the transaction data in the abnormal business date-year-month table to obtain a corrected business date-year-month table" further includes: After all the business date-year-month tables have completed the storage process of transaction data, obtain the abnormal business date-year-month tables from the abnormal cache queue; Delete the incorrect transaction data in the abnormal business date-year-month tables, and obtain the correct transaction data from the natural date-year-month tables and migrate it to the abnormal business date-year-month tables to obtain the corrected business date-year-month tables.

5. The flexible data processing method according to claim 2, characterized in that, The migration of the transaction data in each business date-year-month table from the natural date-year-month table to the corresponding business date-year-month table includes: Determine the transaction data in each business date-year-month table from the natural date-year-month table; Migrate the transaction data from the natural date-year-month table to the corresponding business date-year-month table.

6. The flexible data processing method according to claim 1, characterized in that If there are multiple abnormal business date-year-month tables cached in the abnormal cache queue, then if all business date-year-month tables except the abnormal business date-year-month tables have completed the storage process of transaction data, obtain the abnormal business date-year-month tables from the abnormal cache queue and correct the transaction data in the abnormal business date-year-month tables to obtain the corrected business date-year-month tables; including: If all business date-year-month tables except the abnormal business date-year-month tables have completed the storage process of transaction data, obtain multiple abnormal business date-year-month tables from the abnormal cache queue in the order of their storage in the abnormal cache queue; Correct the transaction data in multiple abnormal business date-year-month tables in the order of acquisition of the multiple abnormal business date-year-month tables to obtain multiple corrected business date-year-month tables; The storage process of the transaction data in the corrected business date-year-month tables according to the information in the data file corresponding to the corrected business date-year-month tables to complete the joint debugging test with the core system includes: According to the information in the data file corresponding to each corrected business date-year-month table, perform the storage process of the transaction data in multiple corrected business date-year-month tables in the order of generation of the multiple corrected business date-year-month tables to complete the joint debugging test with the core system.

7. A flexible data processing device, characterized in that, The device includes: An acquisition module, configured to acquire multiple business date-year-month tables to be processed, where each business date-year-month table contains a batch of transaction data; A first processing module, configured to receive the data files sent by the core system when posting the transaction data in each business date-year-month table, and perform the storage process of the transaction data in the business date-year-month table corresponding to each data file in a preset order according to the information in the data files; A determination module, configured to determine whether there is an abnormal business date and month table with abnormal transaction data storage processing when performing storage processing on the transaction data in the business date and month table. If so, it determines whether the abnormal business date and month table is the last processed business date and month table among the multiple business date and month tables, where the transaction data in the abnormal business date and month table is abnormal; if so, it directly corrects the transaction data in the abnormal business date and month table to obtain a corrected business date and month table; A storage module, configured to, if not, suspend the storage processing of the transaction data in the abnormal business date and month table and store the abnormal business date and month table in an abnormal cache queue; A second processing module, configured to perform storage processing on the transaction data in the business date and month table that has not undergone transaction data storage processing in the preset order; A correction module, configured to, if all business date and month tables except the abnormal business date and month table have completed the storage processing of transaction data, obtain the abnormal business date and month table from the abnormal cache queue and correct the transaction data in the abnormal business date and month table to obtain a corrected business date and month table; A third processing module, configured to perform storage processing on the transaction data in the corrected business date and month table according to the information in the data file corresponding to the corrected business date and month table to complete the joint debugging test with the core system.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when running on a computer, causes the computer to execute the flexible data processing method according to any one of claims 1 to 6.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory. The memory stores a computer program. The processor is configured to execute the flexible data processing method according to any one of claims 1 to 6 by calling the computer program stored in the memory.

Images