Problem data management method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a problem data management method and device, equipment and a storage medium. The method comprises the steps of determining an associated verification rule bucket of problem data from a plurality of preset verification rule buckets for each piece of problem data after each verification; by judging whether the problem data is recorded in the association verification rule bucket or not, adding the problem data into an unrectified to-be-updated list or a newly added list corresponding to the association verification rule bucket, and determining that the problem data is empty data in the association verification rule bucket; after verification of all problem data at this time is completed, adding the residual problem data in the non-rectified sub-buckets under each verification rule bucket into the rectified to-be-updated list corresponding to the verification rule bucket; and according to the unrectified to-be-updated list, the rectified to-be-updated list and the newly added list corresponding to each verification rule bucket, updating the problem data recording state in the verification rule bucket. According to the embodiment of the invention, timely and effective management of the problem data rectification state is ensured.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and specifically to a method, device, equipment and storage medium for managing problem data. Background Art

[0002] With the booming development of the global data economy and the rapid rise of technologies such as 5G, artificial intelligence, and the Internet of Things, the amount of global data is showing an explosive growth trend. The quality of data will directly determine the value of the data, thereby directly or indirectly affecting the development direction of a data industry. To this end, it is necessary to focus on the actual rectification of problematic data in various business data to ensure that data quality meets business requirements.

[0003] At present, after verifying the corresponding problem data from various business data, in addition to the problem data verification step, additional statistics and storage of the actual rectification information of a large amount of problem data are usually performed to support users to independently browse the problem data rectification information. However, due to the storage resource limitations and efficiency requirements of massive problem data, the additional statistics and storage of problem data rectification information also have the problems of high resource requirements and low efficiency, and cannot guarantee the timely and effective management of problem data after rectification, which is likely to reduce the timeliness of problem data management. Summary of the invention

[0004] The embodiments of the present application provide a method, device, equipment and storage medium for problem data management, which realizes immediate rectification management after problem data verification, reduces the time loss of problem data management, ensures timely and effective management of problem data rectification status, and improves the timeliness of problem data management.

[0005] In a first aspect, an embodiment of the present application provides a method for managing problem data, the method comprising:

[0006] For each problem data after each verification, determining the associated verification rule bucket of the problem data from a plurality of preset verification rule buckets;

[0007] By judging whether the problem data verified this time has been recorded in the association verification rule bucket, the problem data is added to the unrectified to-be-updated list or the newly added list corresponding to the association verification rule bucket, and determining that the problem data is empty data in the association verification rule bucket;

[0008] After completing the verification of all problematic data, the remaining problematic data that has not been verified in the unrectified sub-buckets under each verification rule bucket is added to the rectified pending update list corresponding to the verification rule bucket;

[0009] According to the unrectified list to be updated, the rectified list to be updated and the newly added list corresponding to each verification rule bucket, the problem data record status in the verification rule bucket is updated.

[0010] In a second aspect, an embodiment of the present application provides a problem data management device, the device comprising:

[0011] An association rule bucket determination module is used to determine, for each problem data after each verification, an association verification rule bucket of the problem data from a plurality of preset verification rule buckets;

[0012] An unrectified data determination module is used to determine whether the problem data verified this time has been recorded in the association verification rule bucket, add the problem data to the unrectified to-be-updated list or the newly added list corresponding to the association verification rule bucket, and determine that the problem data is empty data in the association verification rule bucket;

[0013] The rectified data determination module is used to add the remaining problematic data that have not been verified in the unrectified sub-bucket under each verification rule bucket to the rectified to-be-updated list corresponding to the verification rule bucket after completing the verification of all problematic data this time;

[0014] The problem data updating module is used to update the problem data record status in the verification rule bucket according to the unrectified list to be updated, the rectified list to be updated and the newly added list corresponding to each verification rule bucket.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0016] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the problem data management method provided in the first aspect of the present application.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to execute the problem data management method provided in the first aspect of the present application.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the problem data management method provided in the first aspect of the present application.

[0019] The embodiment of the present application provides a method, device, equipment and storage medium for managing problem data, which can pre-set multiple verification rule buckets to record historical problem data under different verification rules. Then, after each verification of a problem data, the associated verification rule bucket of the problem data can be immediately determined from multiple verification rule buckets. By judging whether the problem data verified this time has been recorded in the associated verification rule bucket, the problem data is added to the uncorrected to-be-updated list or the newly added list corresponding to the associated verification rule bucket, and it is ensured that the problem data is empty data in the associated verification rule bucket. Therefore, after completing the verification of all the problem data this time, the remaining problem data that has not been verified in the uncorrected sub-bucket under each verification rule bucket can be added to the rectified to-be-updated list corresponding to the verification rule bucket, so that after each verification of a problem data, the rectification status of the problem data can be immediately counted, and there is no need to perform additional statistics and storage on the actual rectification information of all the problem data after the verification of all the problem data is completed, thereby reducing the time loss and computing overhead of problem data management and ensuring efficient management of problem data. Finally, according to the unrectified list to be updated, the rectified list to be updated and the newly added list corresponding to each verification rule bucket, the problem data record status in the verification rule bucket can be updated to achieve accurate partition management of problem data in different verification rule buckets, ensure timely and effective management of the problem data rectification status, and improve the timeliness of problem data management. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of a method for managing problem data shown in an embodiment of the present application;

[0022] Figure 2 This is a schematic diagram showing the principle of recording problem data in different rectification states through multiple verification rule buckets shown in an embodiment of the present application;

[0023] Figure 3 A flowchart of another method for managing problem data according to an embodiment of the present application;

[0024] Figure 4 A principle block diagram of a problem data management device shown in an embodiment of the present application;

[0025] Figure 5 A schematic block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0028] Considering that in addition to the problem data verification step, when additional statistics and storage are performed on the actual rectification information of a large amount of problem data, there will be problems of high resource requirements and low efficiency, and timely and effective management of the problem data after rectification cannot be guaranteed. So, in order to solve the above problems, the embodiment of the present application designs a new problem data management solution. This application will pre-set multiple verification rule buckets according to the various verification rules of the problem data to record the historical problem data under different verification rules, so as to achieve efficient partition management of the problem data under different verification rules. Then, after each verification of a problem data, the problem data verified this time can be added to the unrectified to-be-updated list or the newly added list corresponding to the associated verification rule bucket by judging whether the associated verification rule bucket corresponding to the problem data in multiple verification rule buckets has recorded the problem data verified this time, and ensure that the problem data is empty data in the associated verification rule bucket. Then, after completing the verification of all the problem data, the remaining problem data that has not been verified in the unrectified sub-bucket under each verification rule bucket can be added to the rectified pending update list corresponding to the verification rule bucket. In this way, after each problem data is verified, the rectification status of the problem data can be immediately counted. There is no need to perform additional statistics and storage on the actual rectification information of all problem data after the verification of all problem data is completed, which reduces the time loss and computing overhead of problem data management and ensures efficient management of problem data. Finally, according to the unrectified pending update list, rectified pending update list and newly added list corresponding to each verification rule bucket, the problem data record status in the verification rule bucket can be updated to achieve accurate partition management of problem data in different verification rule buckets, ensure timely and effective management of the rectification status of problem data, and improve the timeliness of problem data management.

[0029] Figure 1 This is a flowchart of a method for managing problem data shown in an embodiment of the present application. Figure 1 , the method may include the following steps:

[0030] S110, for each problematic data after each verification, determining the associated verification rule bucket of the problematic data from a plurality of preset verification rule buckets.

[0031] For various business data in the business system, the quality of the data will directly determine the value of the data, thus directly or indirectly affecting the development direction of the entire business system. For this reason, in each business system, it is usually necessary to perform quality checks on various business data under various data rules to analyze whether there are problematic data in the business data.

[0032] It is understandable that, for the verification of business data, this application can pre-set a data verification interval, and verify the business data once every certain period of time to ensure the reliability of the business data. Alternatively, this application can support the management personnel of the business system to initiate a verification of the business data within the period from time to time to ensure the reliability of the business data. In other words, the verification timing of business data can be a regular passive verification, or it can be an active verification initiated randomly by humans, and this application does not limit this.

[0033] From the above content, it can be seen that in order to ensure comprehensive verification of business data, this application can pre-define multiple verification rules for business data by analyzing the data quality requirements of various business data, so as to verify the corresponding problem data by judging whether the actual value of each business data meets the requirements of each verification rule. Then, when verifying the business data each time, multiple problem data may be obtained.

[0034] In order to avoid the problem of high resource requirements and low efficiency in the additional statistics and storage of the actual rectification information of all problem data after the verification of all problem data is completed, the present application can immediately analyze the specific verification and rectification status of the problem data after each problem data is verified. There is no need to perform additional statistics and storage of the actual rectification information of all problem data after the verification of all problem data is completed, so as to ensure efficient management of problem data.

[0035] In the present application, it is considered that business data will become corresponding problem data when it does not meet the requirements of any verification rule. Then, in order to accurately analyze the specific verification situation of the problem data, the present application can pre-set a corresponding verification rule bucket for each defined verification rule, which is used to record the various historical problem data obtained during each verification process that do not meet the verification rule. In other words, if the problem data A does not meet the requirements of the first verification rule, then the problem data A can be recorded in the verification rule bucket corresponding to the first verification rule. It can be seen that each verification rule bucket can record various historical problem data that were verified during the historical verification process because they did not meet a certain verification rule corresponding to the verification rule bucket.

[0036] In some possible implementations, after verifying the problematic data, the problematic data is usually rectified to ensure the accuracy of the business data. Then, in order to accurately analyze the specific rectification of the problematic data, such as Figure 2As shown, the present application can divide each verification rule bucket into two sub-buckets: an unrectified sub-bucket and a rectified sub-bucket. The unrectified sub-bucket under each verification rule bucket can record historical problem data that is in an unrectified state due to non-compliance with the verification rule corresponding to the verification rule bucket. The rectified sub-bucket under each verification rule bucket can record historical problem data that is in a rectified state due to non-compliance with the verification rule corresponding to the verification rule bucket.

[0037] That is to say, through the unrectified sub-buckets and rectified sub-buckets under each verification rule bucket, each historical problem data in different rectification states under different verification rules can be partitioned and recorded to achieve efficient partition management of problem data under different verification rules and different rectification states. In this way, after each problem data is verified, by analyzing the verification rules that the problem data does not comply with and the records in the corresponding verification rule bucket, the actual rectification status of the problem data can be managed promptly and effectively, without the need to perform additional statistics and storage on the actual rectification information of all problem data after all problem data verification is completed, reducing the time loss and computing overhead of problem data management, and ensuring efficient management of problem data.

[0038] In some possible implementations, in order to ensure efficient recording and querying of problem data in each verification rule bucket, each verification rule bucket in the present application can use a bitmap index method, using the feature mark field of each historical problem data in the problem database as an index identifier to record historical problem data in different rectification states. Among them, the feature mark field of each historical problem data in the problem database can be obtained by combining multiple feature field values ​​of the historical problem data and performing hash processing.

[0039] That is to say, since each problem data can have multiple feature field columns, and each feature field column will have the actual value of the feature field of the problem data entered in it, and each problem data is uniquely identified by each feature field column, which greatly increases the query efficiency of the problem data. Moreover, in order to ensure the unified storage of problem data, each problem data obtained after each verification is usually recorded as historical problem data in the problem database one by one, so that each problem data in the problem database can be promptly and effectively rectified later.

[0040] Then, in order to ensure efficient query of problem data in each verification rule bucket, for each historical problem data recorded in the problem database, the present application can obtain the specific characteristic field values ​​entered under each characteristic field column of the historical problem data, and combine them into a comprehensive characteristic field value. Then, by hashing the combined comprehensive characteristic field value of the historical problem data, a hash value corresponding to the historical problem data can be obtained as the characteristic mark field of the historical problem data to uniquely identify the historical problem data. In the same manner as above, the characteristic mark field of each historical problem data recorded in the problem database can be obtained.

[0041] Furthermore, in order to ensure efficient query of problem data in each verification rule bucket, for each verification rule bucket, the present application can construct a corresponding unrectified bitmap for the unrectified sub-bucket under the verification rule bucket, and a corresponding rectified bitmap for the rectified sub-bucket under the verification rule bucket. Then, for each historical problem data recorded in the problem database, the present application can determine the verification rule bucket that each historical problem data needs to enter by analyzing the verification rules that each historical problem data does not meet, and analyze the current rectification status of each historical problem data. Then, in each verification rule bucket, for some historical problem data in the problem database that does not conform to a certain verification rule corresponding to the verification rule bucket, the historical problem data in the unrectified state and the historical problem data in the rectified state can be distinguished, and the characteristic mark field of each historical problem data is used as an index mark to record the historical problem data in the unrectified state into the unrectified bitmap constructed for the unrectified sub-bucket under the verification rule bucket, and record the historical problem data in the rectified state into the rectified bitmap constructed for the rectified sub-bucket under the verification rule bucket. In this way, by using the bitmap index method in each verification rule bucket, the characteristic mark field of each historical problem data can be used to replace multiple characteristic field columns of the historical problem data to record historical problem data in different rectification states, greatly reducing the resource loss and query efficiency for problem data records in each verification rule bucket.

[0042] It can be seen from the above content that in each verification process, after each problem data is verified, in order to manage the problem data in a timely and effective manner, the present application will first analyze which verification rule the problem data does not meet the requirements, and determine a verification rule bucket corresponding to the verification rule that the problem data does not meet from the pre-set multiple verification rule buckets, as the associated verification rule bucket for the problem data in the present application. Subsequently, the problem data needs to be recorded in the associated verification rule bucket for management.

[0043] S120, by determining whether the problem data verified this time has been recorded in the associated verification rule bucket, the problem data is added to the uncorrected to-be-updated list or the newly added list corresponding to the associated verification rule bucket, and it is determined that the problem data is empty data in the associated verification rule bucket.

[0044] Considering that each verification rule bucket will record the historical problem data in different rectification states in the problematic database in advance, it can be seen that the problem data detected each time may be the problem data newly added by the business system and has not been recorded in any verification rule bucket; it may also be the historical problem data that has been verified before but has not been rectified or has a problem again after rectification, and has been recorded in the associated verification rule bucket after the last verification. At this time, the problem data in different situations needs to be managed with different operations.

[0045] It is understandable that for each historical problem data in the problem database, in order to ensure the accuracy of business data, the historical problem data in the problem database is usually rectified accordingly. Then, after each verification, the problem data verified this time can be compared with the historical problem data verified previously to determine the newly added problem data in this verification, the problem data verified in the past that has not been rectified or has problems after rectification, and the problem data verified in the past but has been rectified in this verification, so as to timely update the problem data records of each verification rule bucket involved in this verification.

[0046] Therefore, in order to accurately analyze the newly added problem data that needs to be updated in each verification rule bucket after each verification, the problem data that has not been rectified or has problems after rectification, and the problem data that has been verified in the past but has been rectified in this verification, this application will pre-set the corresponding unrectified list to be updated, the newly added list, and the rectified list to be updated for each verification rule bucket. Among them, the unrectified list to be updated is used to record the problem data that has been verified before but has not been rectified or has problems after rectification and is verified again this time, the newly added list is used to record the newly added problem data verified this time, and the rectified list to be updated is used to record the problem data that has been verified before but has been rectified in this verification.

[0047] Therefore, for each problem data verified during each verification process, after determining the associated verification rule bucket of the problem data, the present application will first use the characteristic mark field of the problem data to query whether there is the problem data verified this time in the associated verification rule bucket, so as to determine whether the problem data verified this time has been recorded in the associated verification rule bucket, and thus determine whether the problem data verified this time is new problem data, or problem data that has been verified before but has not been rectified or has problems after rectification and is verified again this time. Then, according to the different types of the problem data verified this time after the above analysis, the problem data verified this time can be added to the unrectified to-be-updated list or the newly added list corresponding to the associated verification rule bucket, so as to update the problem data recorded in the associated verification rule bucket later.

[0048] Moreover, considering that the historical problem data recorded in the associated verification rule bucket are all previously verified problem data, then, in order to facilitate the effective update of the problem data verified this time in the associated verification rule bucket, this application will also set the problem data verified this time to empty data in the associated verification rule bucket, so as to re-record and manage the problem data verified this time according to the unrectified list to be updated and the newly added list corresponding to the associated verification rule bucket.

[0049] S130, after completing the verification of all the problem data this time, the remaining problem data that has not been verified this time in the unrectified sub-bucket under each verification rule bucket is added to the rectified to-be-updated list corresponding to the verification rule bucket.

[0050] Considering that different problem data detected each time will involve different verification rules, the problem data records in each verification rule bucket will be updated once after each verification. The uncorrected sub-bucket under each verification rule bucket will record the historical problem data in the uncorrected state, and in this verification process, the problem data detected this time has been set to empty data in each verification rule bucket, which means that the verification rule bucket does not have the problem data detected this time.

[0051] Then, after completing the verification of all the problem data this time, the present application can determine whether there is any problem data remaining in the unrectified sub-bucket under each verification rule bucket because it was not verified this time. The remaining problem data was not verified during this verification process and was not set to empty data, indicating that the remaining problem data has been rectified during this verification and is no longer in an unrectified state. Therefore, for each verification rule bucket, the present application can add the remaining problem data that was not verified this time in the unrectified sub-bucket under the verification rule bucket as the problem data that was previously verified but has been rectified during this verification to the rectified to-be-updated list corresponding to the verification rule bucket, so as to subsequently update the historical problem data in the rectified state in the verification rule bucket.

[0052] For example, assume that four problematic data items A, B, C, and F can be verified during this verification. The uncorrected sub-buckets under the associated verification rule buckets of these four problematic data items have recorded five problematic data items A, B, C, D, and E before this verification.

[0053] Then, when the problem data A is verified this time, it can be determined that the problem data A verified this time has been recorded in the associated verification rule bucket, indicating that the problem data A verified this time is the problem data that has been verified before but has not been rectified or has problems after rectification and is verified again this time. Then, the problem data A verified this time can be added to the unrectified pending update list corresponding to the associated verification rule bucket, and the problem data A recorded in the unrectified sub-bucket under the associated verification rule bucket is set to empty data, that is, the problem data A is deleted from the unrectified sub-bucket under the associated verification rule bucket.

[0054] When the problem data B and C are verified this time, the same processing operation as the problem data A verified this time can be performed to add the problem data B and C verified this time to the unrectified list to be updated corresponding to the associated verification rule bucket, and set the problem data B and C recorded in the unrectified sub-bucket under the associated verification rule bucket to empty data, that is, delete the problem data B and C from the unrectified sub-bucket under the associated verification rule bucket.

[0055] When the problem data F is verified this time, it can be determined that the problem data F verified this time is not recorded in the associated verification rule bucket, indicating that the problem data F verified this time is the newly added problem data in this verification. Then, the problem data F verified this time can be added to the newly added list corresponding to the associated verification rule bucket.

[0056] Therefore, after completing the verification of all the problem data A, B, C, and F, it can be determined that there are still two problem data D and E remaining in the unrectified sub-bucket under the associated verification rule bucket because they were not verified this time, indicating that the problem data D and E that were not verified this time had been rectified before this verification, and are no longer in the unrectified state. Then, the remaining problem data D and E that were not verified this time in the unrectified sub-bucket under the associated verification rule bucket can be added to the rectified pending update list corresponding to the verification rule bucket as the problem data that was previously verified but has been rectified during this verification.

[0057] S140, updating the problem data record status in each verification rule bucket according to the uncorrected to-be-updated list, the corrected to-be-updated list and the newly added list corresponding to each verification rule bucket.

[0058] After obtaining the unrectified list to be updated, the rectified list to be updated, and the newly added list corresponding to each verification rule bucket, for each verification rule bucket, this application can determine that the problem data in the unrectified list to be updated and the newly added list corresponding to the verification rule bucket are all in the unrectified state, and the problem data in the rectified list to be updated corresponding to the verification rule bucket are in the rectified state. Accordingly, each problem data in the unrectified list to be updated, the rectified list to be updated, and the newly added list corresponding to each verification rule bucket can be updated to different partitions under the verification rule bucket according to the current rectification status of the problem data, so as to accurately update the historical problem data in different rectification states recorded in the verification rule bucket.

[0059] The technical solution provided by the embodiment of the present application can pre-set multiple verification rule buckets to record historical problem data under different verification rules. Then, after each verification of a problem data, the associated verification rule bucket of the problem data can be immediately determined from multiple verification rule buckets. By judging whether the problem data verified this time has been recorded in the associated verification rule bucket, the problem data is added to the uncorrected to-be-updated list or the newly added list corresponding to the associated verification rule bucket, and it is ensured that the problem data is empty data in the associated verification rule bucket. Thus, after completing the verification of all the problem data this time, the remaining problem data that has not been verified in the uncorrected sub-bucket under each verification rule bucket can be added to the corrected to-be-updated list corresponding to the verification rule bucket, so that after each verification of a problem data, the rectification status of the problem data can be immediately counted, and there is no need to perform additional statistics and storage on the actual rectification information of all the problem data after the verification of all the problem data is completed, thereby reducing the time loss and computing overhead of problem data management and ensuring efficient management of problem data. Finally, according to the unrectified list to be updated, the rectified list to be updated and the newly added list corresponding to each verification rule bucket, the problem data record status in the verification rule bucket can be updated to achieve accurate partition management of problem data in different verification rule buckets, ensure timely and effective management of the problem data rectification status, and improve the timeliness of problem data management.

[0060] As an optional implementation scheme in the present application, the present application can uniformly store the problem data verified each time through a problem database, and partition and record the problem data in different rectification states through unrectified sub-buckets and rectified sub-buckets under multiple verification rule buckets. Thus, after each verification, the problem data in different rectification states recorded in the unrectified sub-buckets and rectified sub-buckets under each verification rule bucket can be updated to achieve accurate partition management of the problem data in different verification rule buckets. Therefore, in order to ensure the timely and effective management of the rectification status of the problem data, the present application can provide a detailed explanation of the specific update process of the problem data record status in each verification rule bucket after each verification and the update process of the problem database.

[0061] Figure 3 This is a flow chart of another method for managing problem data according to an embodiment of the present application. Figure 3 As shown, the method may specifically include the following steps:

[0062] S310, for each problem data after each verification, determining the associated verification rule bucket of the problem data from a plurality of preset verification rule buckets.

[0063] S320, if the problem data verified this time has been recorded in the associated verification rule bucket, the problem data is added to the uncorrected to-be-updated list corresponding to the associated verification rule bucket, and the problem data is removed from the associated verification rule bucket, so that the problem data is empty data in the associated verification rule bucket.

[0064] For each problem data verified during each verification process, after determining the associated verification rule bucket for the problem data, it is possible to analyze whether the problem data verified this time has been recorded in the associated verification rule bucket to determine whether the problem data is new problem data for this verification, or problem data that has been verified before but has not been rectified, or problem data that has reappeared after rectification and is verified again this time. This allows problem data in different situations to be managed under different operations to ensure the accuracy of problem data management.

[0065] Then, if the problem data verified this time has been recorded in the associated verification rule bucket, it means that the problem data was verified before but not rectified or had problems after rectification and was verified again this time. Then, this application can add the problem data to the unrectified pending update list corresponding to the associated verification rule bucket, thereby indicating that the latest rectification status of each problem data in the unrectified pending update list is the unrectified status, ensuring the accurate update of the rectification status of the problem data in the associated verification rule bucket in the future.

[0066] Moreover, in order to facilitate the effective update of the problem data verified this time in the associated verification rule bucket, this application will also directly remove the problem data verified this time in the associated verification rule bucket, and re-record and manage the problem data verified this time according to the unrectified list to be updated corresponding to the associated verification rule bucket.

[0067] S330: If the problem data verified this time is not recorded in the associated verification rule bucket, the problem data is added to the newly added list corresponding to the associated verification rule bucket.

[0068] However, the problem data verified in this verification is not recorded in the associated verification rule bucket, indicating that the problem data is newly added in this verification. Therefore, it can be directly determined that the problem data does not exist in the associated verification rule bucket, and the problem data can be directly added to the newly added list corresponding to the associated verification rule bucket, which indicates that the latest rectification status of each problem data in the newly added list is the unrectified state, ensuring the accurate update of the rectification status of the problem data in the associated verification rule bucket in the future.

[0069] It can be understood that S320 and S330 are specific steps executed in two different situations of whether the problematic data is recorded in the association verification rule bucket, so S320 and S330 can be executed one by one, but not at the same time.

[0070] S340, after completing the verification of all the problem data this time, the remaining problem data that has not been verified this time in the unrectified sub-bucket under each verification rule bucket is added to the rectified to-be-updated list corresponding to the verification rule bucket.

[0071] S350, for each verification rule bucket, determining the first type of problem data in the unrectified to-be-updated list and the newly added list corresponding to the verification rule bucket and the second type of problem data in the rectified to-be-updated list corresponding to the verification rule bucket.

[0072] After obtaining the unrectified list to be updated, the rectified list to be updated, and the newly added list corresponding to each verification rule bucket, for each verification rule bucket, the present application can determine that the problem data in the unrectified list to be updated and the newly added list corresponding to the verification rule bucket are all in an unrectified state, and the problem data in the rectified list to be updated corresponding to the verification rule bucket are in a rectified state. Accordingly, the present application can treat each problem data in the unrectified list to be updated and the newly added list corresponding to each verification rule bucket as first-category problem data, and the first-category problem data is problem data in an unrectified state. And treat each problem data in the rectified list to be updated corresponding to each verification rule bucket as second-category problem data, and the second-category problem data is problem data in a rectified state.

[0073] S360, update and record the first type of problem data into the unrectified sub-bucket under the verification rule bucket, and update and record the second type of problem data into the rectified sub-bucket under the verification rule bucket.

[0074] For the first type of problem data and the second type of problem data, the present application can update and record each first type of problem data in sequence into the unrectified sub-bucket under the verification rule bucket, and update and record each second type of problem data in sequence into the rectified sub-bucket under the verification rule bucket, so as to accurately update the historical problem data in different rectification states recorded in the verification rule bucket.

[0075] S370, associate the problem data in the newly added list corresponding to each verification rule bucket with the bucket identifier of the verification rule bucket, and write them into the problem database in batches, so as to rectify the historical problem data after verification in the problem database, and record the latest rectification status of each historical problem data in the problem database.

[0076] Considering that some problem data may be added after each verification, and the problem data after each verification will be uniformly stored in the problem database. The problem database usually records each historical problem data into different verification rule buckets according to the specific verification rules that each historical problem data does not meet, so as to partition the problem data in different rectification states.

[0077] Therefore, for each newly added list corresponding to a verification rule bucket, the newly added list can record the newly added problem data after each verification. Then, in order to facilitate the accurate record management of each historical problem data in the problem database to each verification rule bucket, for each newly added list corresponding to a verification rule bucket, the present application can associate the bucket identifier of the verification rule bucket with each problem data in the newly added list, for example, the bucket identifier of each verification rule bucket and each problem data in the newly added list corresponding to the verification rule bucket form a corresponding key-value pair.

[0078] Then, after the number of problem data in the newly added list corresponding to each verification rule bucket reaches the preset batch submission number, the bucket identifier of the verification rule bucket and the associated information of each problem data in the newly added list corresponding to the verification rule bucket can be written into the problem database in batches, so that the problem database can store the problem data after each verification and mark the bucket identifier corresponding to the problem data.

[0079] Moreover, in order to ensure the accuracy of business data, this application can regularly rectify historical problem data after verification in the problem database, and record the latest rectification status of each historical problem data in the problem database in real time, so that each verification rule bucket can accurately manage the rectification status of each problem data.

[0080] S380, traverse the historical problem data in the problem database, and update the problem data record status in each verification rule bucket according to the associated bucket identifier and the latest rectification status of each historical problem data.

[0081] For the problem database, in order to ensure accurate partition management of the rectification status of problem data in each verification rule bucket, this application will use the unrectified list to be updated, the rectified list to be updated and the newly added list corresponding to each verification rule bucket to make corresponding updates to the problem data record status in the verification rule bucket after each verification. At the same time, it will traverse the various historical problem data in the problem database from time to time to make further additional updates to the problem data record status in each verification rule bucket to ensure accurate management of the rectification status of problem data through each verification rule bucket.

[0082] At this time, each historical problem data in the problem database is traversed to determine the bucket identifier associated with the historical problem data, so as to determine in which verification rule bucket the historical problem data needs to be recorded, so as to complete the effective management of the problem data rectification status. In addition, by analyzing the specific rectification status of the historical problem data in the problem database, the latest rectification status of the historical problem data is determined.

[0083] Then, according to the bucket identifier associated with each historical problem data in the problem database, the associated verification rule bucket of the historical problem data can be determined. Furthermore, according to the latest rectification status of the historical problem data, the historical problem data is updated to the unrectified sub-bucket or the rectified sub-bucket under the associated verification rule bucket. During the traversal of the problem data in the problem database, the problem data record status in each verification rule bucket can be continuously updated according to the same steps as above, so as to realize the accurate management of the rectification status of the problem data through each verification rule bucket.

[0084] The technical solution provided by the embodiment of the present application can pre-set multiple verification rule buckets, and divide unrectified sub-buckets and rectified sub-buckets under each verification rule bucket, thereby partitioning and recording historical problem data in different rectification states under different verification rules through unrectified sub-buckets and rectified sub-buckets under each verification rule bucket. Then, after each verification of a problem data, the associated verification rule bucket of the problem data can be immediately determined from multiple verification rule buckets. By judging whether the problem data has been recorded in the associated verification rule bucket, the problem data is added to the unrectified to-be-updated list or the newly added list corresponding to the associated verification rule bucket, and it is ensured that the problem data is empty data in the associated verification rule bucket. Thus, after completing the verification of all the problem data this time, the remaining problem data in the unrectified sub-bucket under each verification rule bucket can be added to the rectified to-be-updated list corresponding to the verification rule bucket, so that after each verification of a problem data, the rectification status of the problem data can be immediately counted, and there is no need to perform additional statistics and storage on the actual rectification information of all the problem data after the verification of all the problem data is completed, thereby reducing the time loss and computing overhead of problem data management and ensuring efficient management of problem data. Finally, according to the unrectified list to be updated, the rectified list to be updated and the newly added list corresponding to each verification rule bucket, the problem data record status in the verification rule bucket can be updated to achieve accurate partition management of problem data in different verification rule buckets, ensure timely and effective management of the problem data rectification status, and improve the timeliness of problem data management.

[0085] Figure 4 This is a principle block diagram of a problem data management device shown in an embodiment of the present application. Figure 4 As shown, the apparatus 400 may include:

[0086] The associated rule bucket determination module 410 is used to determine, for each problem data after each verification, an associated verification rule bucket of the problem data from a plurality of preset verification rule buckets;

[0087] The unrectified data determination module 420 is used to determine whether the problem data verified this time has been recorded in the association verification rule bucket, add the problem data to the unrectified to-be-updated list or the newly added list corresponding to the association verification rule bucket, and determine that the problem data is empty data in the association verification rule bucket;

[0088] The rectified data determination module 430 is used to add the remaining problematic data that have not been verified in the unrectified sub-bucket under each verification rule bucket to the rectified to-be-updated list corresponding to the verification rule bucket after completing the verification of all problematic data this time;

[0089] The problem data updating module 440 is used to update the problem data record status in each verification rule bucket according to the uncorrected to-be-updated list, the corrected to-be-updated list and the newly added list corresponding to each verification rule bucket.

[0090] In some implementations, the unrectified data determination module 420 may be specifically configured to:

[0091] If the problem data verified this time has been recorded in the association verification rule bucket, the problem data is added to the unrectified to-be-updated list corresponding to the association verification rule bucket, and the problem data verified this time is removed from the association verification rule bucket, so that the problem data is empty data in the association verification rule bucket;

[0092] If the associated verification rule bucket does not record the problem data verified this time, the problem data is added to the newly added list corresponding to the associated verification rule bucket.

[0093] In some implementations, the question data updating module 440 may be specifically used to:

[0094] For each verification rule bucket, determine the first type of problem data in the unrectified to-be-updated list and the newly added list corresponding to the verification rule bucket and the second type of problem data in the rectified to-be-updated list corresponding to the verification rule bucket;

[0095] Update and record the first type of problem data into the uncorrected sub-bucket under the verification rule bucket;

[0096] The second type of problem data is updated and recorded in the rectified sub-bucket under the verification rule bucket.

[0097] In some possible implementations, the problem data management device 400 may further include:

[0098] The problem data storage module is used to associate the problem data in the newly added list corresponding to each verification rule bucket with the bucket identifier of the verification rule bucket, and write them into the problem database in batches, so as to rectify the historical problem data after verification in the problem database, and record the latest rectification status of each historical problem data in the problem database.

[0099] In some possible implementations, the problem data management device 400 may further include:

[0100] The problem database traversal module is used to traverse the historical problem data in the problem database, and update the problem data record status in each verification rule bucket according to the associated bucket identifier and the latest rectification status of each historical problem data.

[0101] In some possible implementations, each of the verification rule buckets uses a bitmap index method, with the characteristic mark field of each historical problem data in the problem database as an index identifier to record historical problem data in different rectification states; the characteristic mark field of each historical problem data in the problem database is obtained by combining multiple characteristic field values ​​of the historical problem data and performing hash processing.

[0102] In the embodiment of the present application, multiple verification rule buckets can be pre-set to record historical problem data under different verification rules. Then, after each verification of a problem data, the associated verification rule bucket of the problem data can be immediately determined from multiple verification rule buckets. By judging whether the problem data verified this time has been recorded in the associated verification rule bucket, the problem data is added to the uncorrected to-be-updated list or the newly added list corresponding to the associated verification rule bucket, and it is ensured that the problem data is empty data in the associated verification rule bucket. Thus, after completing the verification of all the problem data this time, the remaining problem data that has not been verified in the unrectified sub-bucket under each verification rule bucket can be added to the rectified to-be-updated list corresponding to the verification rule bucket, so that after each verification of a problem data, the rectification status of the problem data can be immediately counted, and there is no need to perform additional statistics and storage on the actual rectification information of all the problem data after the verification of all the problem data is completed, thereby reducing the time loss and computing overhead of problem data management and ensuring efficient management of problem data. Finally, according to the unrectified list to be updated, the rectified list to be updated and the newly added list corresponding to each verification rule bucket, the problem data record status in the verification rule bucket can be updated to achieve accurate partition management of problem data in different verification rule buckets, ensure timely and effective management of the problem data rectification status, and improve the timeliness of problem data management.

[0103] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Figure 4 The device 400 shown can execute any method embodiment in the present application, and the aforementioned and other operations and / or functions of each module in the device 400 are respectively for implementing the corresponding processes in each method in the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0104] The above describes the device 400 of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0105] Figure 5 A schematic block diagram of an electronic device according to an embodiment of the present application.

[0106] like Figure 5 As shown, the electronic device 500 may include:

[0107] The memory 510 and the processor 520, the memory 510 is used to store the computer program and transmit the program code to the processor 520. In other words, the processor 520 can call and run the computer program from the memory 510 to implement the method in the embodiment of the present application.

[0108] For example, the processor 520 may be configured to execute the above method embodiments according to instructions in the computer program.

[0109] In some embodiments of the present application, the processor 520 may include but is not limited to:

[0110] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0111] In some embodiments of the present application, the memory 510 includes but is not limited to:

[0112] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0113] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 510 and executed by the processor 520 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0114] like Figure 5 As shown, the electronic device may also include:

[0115] The transceiver 530 may be connected to the processor 520 or the memory 510 .

[0116] The processor 520 may control the transceiver 530 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 530 may include a transmitter and a receiver. The transceiver 530 may further include an antenna, and the number of antennas may be one or more.

[0117] It should be understood that the various components in the electronic device are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0118] The present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0119] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0120] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0121] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0122] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0123] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for managing problem data, characterized in that: include: For each problem data after each verification, determining the associated verification rule bucket of the problem data from a plurality of preset verification rule buckets; By judging whether the problem data verified this time has been recorded in the association verification rule bucket, the problem data is added to the unrectified to-be-updated list or the newly added list corresponding to the association verification rule bucket, and determining that the problem data is empty data in the association verification rule bucket; After completing the verification of all problematic data, the remaining problematic data that has not been verified in the unrectified sub-buckets under each verification rule bucket is added to the rectified pending update list corresponding to the verification rule bucket; According to the unrectified list to be updated, the rectified list to be updated and the newly added list corresponding to each verification rule bucket, the problem data record status in the verification rule bucket is updated.

2. The method according to claim 1, characterized in that: The step of judging whether the problem data verified this time has been recorded in the association verification rule bucket, adding the problem data to the unrectified to-be-updated list or the newly added list corresponding to the association verification rule bucket, and determining that the problem data is empty data in the association verification rule bucket includes: If the problem data verified this time has been recorded in the association verification rule bucket, the problem data is added to the unrectified to-be-updated list corresponding to the association verification rule bucket, and the problem data verified this time is removed from the association verification rule bucket, so that the problem data is empty data in the association verification rule bucket; If the associated verification rule bucket does not record the problem data verified this time, the problem data is added to the newly added list corresponding to the associated verification rule bucket.

3. The method according to claim 1, characterized in that The updating of the problem data record status in each verification rule bucket according to the uncorrected to-be-updated list, the corrected to-be-updated list, and the newly added list corresponding to each verification rule bucket includes: For each verification rule bucket, determine the first type of problem data in the unrectified to-be-updated list and the newly added list corresponding to the verification rule bucket and the second type of problem data in the rectified to-be-updated list corresponding to the verification rule bucket; Update and record the first type of problem data into the uncorrected sub-bucket under the verification rule bucket; The second type of problem data is updated and recorded in the rectified sub-bucket under the verification rule bucket.

4. The method according to claim 1, characterized in that The method further comprises: The problem data in the newly added list corresponding to each verification rule bucket is associated with the bucket identifier of the verification rule bucket, and written into the problem database in batches, so as to rectify the historical problem data after verification in the problem database, and record the latest rectification status of each historical problem data in the problem database.

5. The method according to claim 1, characterized in that The method further comprises: The historical problem data in the problem database are traversed, and the problem data record status in each verification rule bucket is updated according to the associated bucket identifier and the latest rectification status of each historical problem data.

6. The method according to claim 1, characterized in that Each of the verification rule buckets adopts a bitmap index method, with the characteristic mark field of each historical problem data in the problem database as an index identifier to record historical problem data in different rectification states; the characteristic mark field of each historical problem data in the problem database is obtained by combining multiple characteristic field values ​​of the historical problem data and performing hash processing.

7. A problem data management device, characterized in that: include: An association rule bucket determination module is used to determine, for each problem data after each verification, an association verification rule bucket of the problem data from a plurality of preset verification rule buckets; An unrectified data determination module is used to determine whether the problem data verified this time has been recorded in the association verification rule bucket, add the problem data to the unrectified to-be-updated list or the newly added list corresponding to the association verification rule bucket, and determine that the problem data is empty data in the association verification rule bucket; The rectified data determination module is used to add the remaining problematic data that have not been verified in the unrectified sub-bucket under each verification rule bucket to the rectified to-be-updated list corresponding to the verification rule bucket after completing the verification of all problematic data this time; The problem data updating module is used to update the problem data record status in the verification rule bucket according to the unrectified list to be updated, the rectified list to be updated and the newly added list corresponding to each verification rule bucket.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the problem data management method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the problem data management method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for managing problem data as described in any one of claims 1 to 6 is implemented.

Images