Data quality monitoring method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a data quality monitoring method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a plurality of processing stages of data according to a calculation rule of the data, and determining a monitoring range of a calculation unit and a quality inspection unit of the data in each processing stage; when the target data is the first data change identifier in the current processing stage, judging whether the target data is within the monitoring range of the calculation unit and the quality inspection unit or not; according to the judgment result, the calculation identifier of the calculation unit and the quality inspection identifier of the quality inspection unit are updated; wherein the first data change identifier is used for representing that the data is updated. By setting the change identifier, the calculation identifier and the quality inspection identifier for the data, the data quality problem can be conveniently positioned, the link length can be reduced and eliminated, and the delivery efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data quality monitoring method, device, electronic equipment and storage medium. Background Art

[0002] In the construction of digital twins of building models, a large amount of computer automatic calculations are added during the twin construction in order to improve the construction efficiency. The pre-calculation data of computer automatic calculations are generally delivered manually. When problems occur in the automatic calculations, it is necessary to quickly locate the pre-calculation data, find the problems and guide manual corrections.

[0003] However, due to the extremely large amount of data delivered and the long delivery cycle, the data quality check needs to be carried out in batches. When errors are found, it is impossible to provide the delivery personnel with sufficient and accurate information to locate the nodes that need to be adjusted.

[0004] Therefore, there is an urgent need to provide a data quality monitoring method to monitor the quality of data generated during the digital delivery process, so that delivery personnel can locate problematic data. Summary of the invention

[0005] The present invention provides a data quality monitoring method, device, electronic device and storage medium to monitor the quality of data and facilitate delivery personnel to locate problematic data.

[0006] According to one aspect of the present invention, a method for monitoring data quality is provided, the method comprising:

[0007] Determine multiple data processing stages according to the data calculation rules, and determine the monitoring scope of the data calculation unit and quality inspection unit in each processing stage;

[0008] When the target data is a first data change identifier in the current processing stage, determining whether the target data is within the monitoring range of the calculation unit and the quality inspection unit;

[0009] According to the judgment result, the calculation identification of the calculation unit and the quality inspection identification of the quality inspection unit are updated;

[0010] The first data change identifier is used to indicate that data has been updated.

[0011] According to another aspect of the present invention, a data quality monitoring device is provided, the device comprising:

[0012] A monitoring scope determination module is used to determine multiple processing stages of data according to the calculation rules of data, and determine the monitoring scope of the data calculation unit and quality inspection unit in each processing stage;

[0013] A data monitoring range determination module, used for determining whether the target data is within the monitoring range of the calculation unit and the quality inspection unit when the target data is a first data change identifier in the current processing stage;

[0014] An identification updating module, used for updating the calculation identification of the calculation unit and the quality inspection identification of the quality inspection unit according to the judgment result;

[0015] The first data change identifier is used to indicate that data has been updated.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data quality monitoring method described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data quality monitoring method described in any embodiment of the present invention when executed.

[0021] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the data quality monitoring method according to any embodiment of the present invention is implemented.

[0022] The technical solution of the embodiment of the present invention determines multiple processing stages of data according to the calculation rules of the data, and determines the monitoring range of the calculation unit and the quality inspection unit of the data in each processing stage; when in the current processing stage, the target data is the first data change identifier, it is judged whether the target data is within the monitoring range of the calculation unit and the quality inspection unit; according to the judgment result, the calculation identifier of the calculation unit and the quality inspection identifier of the quality inspection unit are updated; wherein, the first data change identifier is used to indicate that the data has been updated, which solves the problem of data quality monitoring during the construction of digital twins, and by setting change identifiers, calculation identifiers and quality inspection identifiers for the data, it is convenient to locate data quality problems, reduce the length of the exclusion link, and improve delivery efficiency.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a flow chart of a data quality monitoring method provided according to an embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of associating a data identifier with a computing identifier according to an embodiment of the present invention;

[0027] Figure 3 is a schematic diagram of a calculation identification change provided according to an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of associating a quality inspection mark with a data mark according to an embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of the linkage of various quality inspection units provided according to an embodiment of the present invention;

[0030] Figure 6 This is a diagram showing an example of application of a data quality monitoring method provided in an embodiment of the present invention;

[0031] Figure 7 is a schematic structural diagram of a data quality monitoring device provided according to an embodiment of the present invention;

[0032] Figure 8 It is a schematic diagram of the structure of an electronic device for implementing the data quality monitoring method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Figure 1 This is a flow chart of a data quality monitoring method provided according to an embodiment of the present invention. This embodiment is applicable to the situation where the quality of data in the calculation is monitored when the digital twin of the building model is established. The method can be executed by a data quality monitoring device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device such as a computer. Figure 1 As shown, the method includes:

[0036] Step 110: Determine multiple processing stages of the data according to the calculation rules of the data, and determine the monitoring scope of the data calculation unit and the quality inspection unit in each processing stage.

[0037] Among them, the data calculation rules can be the calculation rules for calculating the data in the twin construction when the digital twin is carried out in the building field. The data can be an instance of delivery information, such as objects, information, and relationships. Due to the large amount of delivered data, the specific calculation can be carried out in batches and in stages, so the data can have multiple processing stages. The calculation units and quality inspection units in different processing stages can have different monitoring ranges. The monitoring scope can be understood as which data the calculation unit and the quality inspection unit specifically monitor. For example, the calculation unit M monitors the data in the processing stage M-1. For another example, the quality inspection unit M+1 monitors the data in the processing stage M+1.

[0038] Furthermore, in addition to being related to the processing stage, the monitoring scope may also be related to the production unit of the data. For example, the same type of objects may be produced by different production units, but the calculation unit or quality inspection unit may only focus on a specific production unit. For example, object class X may be produced in different production units, such as M and N. The calculation unit A only focuses on the data Xn produced in the production unit N, and may not calculate the data Xm produced in the production unit M.

[0039] In the embodiment of the present invention, in order to ensure the correctness and reliability of data quality monitoring, the monitoring range of the calculation unit and the quality inspection unit in each processing stage can be determined according to the calculation rules, which can ensure the correct monitoring of the corresponding data.

[0040] Step 120: When the target data is the first data change identifier in the current processing stage, determine whether the target data is within the monitoring range of the calculation unit and the quality inspection unit.

[0041] The first data change flag is used to indicate that the data has been updated. Specifically, when the data is updated, that is, when the old data is changed to the new data, or when the new data is added, the data can be marked as the first data change flag. When determining whether the target data is within the monitoring scope, the processing stage of the target data can be matched with the monitoring stage corresponding to the computing unit and the quality inspection unit.

[0042] Furthermore, in an optional implementation of an embodiment of the present invention, when in the current processing stage, the target data is a first data change identifier, judging whether the target data is within the monitoring range of the calculation unit and the quality inspection unit, including: when in the current processing stage, the target data is a first data change identifier, judging whether the target data is within the monitoring range of the calculation unit and the quality inspection unit according to the production unit identifier of the target data and the current processing stage of the target data.

[0043] The data status identifier of the target data may include two types: one is a data change identifier and the other is a production unit identifier. The data change identifier is used to indicate whether the data has changed. The data change identifier may include a first data change identifier and a second data change identifier. The first data change identifier is used to indicate that the data has been updated. The second data change identifier is used to indicate that the data has completed quality inspection.

[0044] The production unit identifier is used to indicate the functional unit that generates the data. For example, the high-voltage equipment object generated by the questionnaire can add "DY052" as the production unit identifier, which indicates that this object is generated in DY052 and that the data needs to be maintained in DY052 later. It should be emphasized that the delivery information generated by automatic calculation also needs to add the production unit identifier. For example, the data that completes the data calculation in the DY111 data automatic calculation section needs to add the data production unit identifier "DY111".

[0045] When determining whether the target data is within the monitoring range of the computing unit and the quality inspection unit, the production unit concerned by the computing unit and the quality inspection unit can be further considered while considering the processing stage to accurately monitor the data.

[0046] Step 130: Update the calculation identification of the calculation unit and the quality inspection identification of the quality inspection unit according to the judgment result.

[0047] In an embodiment of the present invention, the calculation identifier and the quality inspection identifier can be used together as the process unit identifier of the data. The calculation identifier may include multiple identifiers. For example, the calculation identifier may include an identifier that needs to be calculated, an identifier that the calculation failed, and an identifier that the calculation is completed. The calculation identifier can indicate whether the calculation unit needs to be executed. The result of the calculation identifier can be displayed on the web-side tool so that the user can see it more clearly and remind the user to execute it. Specifically, the "identification that needs to be calculated" means that the system has not performed this automatic calculation under the changed conditions when the data identifier is the first data change identifier. The "identification of failed calculation" means that an error occurred during the automatic calculation of the data, resulting in a calculation failure, and troubleshooting is required. The "identification of completed calculation" means that the automatic calculation of the data is completed normally.

[0048] On the basis of the above implementation, optionally, the method further includes: when the calculation of the calculation unit in the current stage is completed, updating the calculation identifier of the calculation unit; wherein the calculation identifier includes: an identifier of calculation failure and an identifier of calculation completion.

[0049] Specifically, the change of the calculation flag can be: when the calculation unit is triggered for the first time and the calculation is completed, the value of the calculation flag "the flag that needs to be calculated" will change. When the calculation fails, the value of the calculation flag is changed to "the flag of calculation failure"; when the calculation is successful, the value of the calculation flag is changed to "the flag of calculation completion".

[0050] After that, no matter what the current value of the calculation identifier is, the calculation of the calculation unit can be triggered again, and the label value after inspection is still updated according to the rule that "when the calculation fails, the value of the calculation identifier is changed to "the identifier of the calculation failure"; when the automatic calculation is successful, the value of the calculation identifier is changed to "the identifier of the calculation completion"".

[0051] In an optional implementation of the embodiment of the present invention, the data identifier can be associated with the calculation identifier, and the calculation identifier can be adjusted when the data identifier changes. Optionally, based on the judgment result, the calculation identifier of the calculation unit and the quality inspection identifier of the quality inspection unit are updated, including: when the target data is within the monitoring range of the calculation unit, the calculation identifier of the calculation unit in the current processing stage is updated to the identifier that needs to be calculated; when the target data is not within the monitoring range of the calculation unit, the calculation identifier of the calculation unit in the current processing stage is kept unchanged.

[0052] The calculation identifier needs to monitor the main class of all input parameters of this calculation unit, and monitor the data production unit identifier of the main instance of each input parameter. Based on the data production unit identifier of the main class and the instance, determine whether the data change identifier of this data instance needs to be monitored. When the data change identifier changes from the "second data change identifier" to the "first data change identifier", that is, from "old data" to "new data", the calculation unit can change the calculation identifier of this data from the "calculation failure identifier" or "calculation completion identifier" to the "calculation required identifier".

[0053] Figure 2 FIG. 1 is a schematic diagram of associating a data identifier with a computing identifier according to an embodiment of the present invention. Figure 2 As shown, assuming that there is a calculation rule in the calculation unit, which requires an object class as an input parameter. When a user generates a new object instance of an object class, the data change identifier of the new object instance is "first data change identifier" or "new data", and the calculation identifier of the calculation unit will become "identifier that needs to be calculated".

[0054] Of course, object class data may be generated in different production units, and the calculation rules in the calculation unit only focus on the object instances generated in the specified production unit. Therefore, only when the object instance generated in the production unit changes, the calculation identification of the calculation unit will be changed to "identity that needs to be calculated". When the object instance generated in the production unit does not change, the calculation identification of the calculation unit will not change.

[0055] For example, suppose there is a calculation rule α in the calculation unit A, which requires object class X as an input parameter. When a user generates a new object instance X of object class X n , new object instance X n The data change flag of is "first data change flag" or "new data", then the calculation flag of calculation unit A will become "need to calculate". However, in practice, object class X may be generated in different production units, and there is a calculation rule α in calculation unit A that only focuses on object instance X generated in production unit N. n . Figure 3 1 is a schematic diagram of a calculation identifier change provided according to an embodiment of the present invention. Figure 3 As shown, according to the calculation rule α, only when the object instance X generated in the production unit N n When a change occurs, the unit status flag of the automatic calculation unit A will be changed to "the flag that needs to be calculated"; and when the object instance X generated in the production unit M m When a change occurs, the calculation identifier of calculation unit A will not change.

[0056] In the embodiment of the present invention, the quality inspection mark expresses whether the quality inspection unit needs to be executed. At the same time, when the data is verified to be incorrect, the user can be prompted that the data is abnormal and needs manual inspection. The quality inspection mark result of the quality inspection unit can be displayed on the web-side tool so that the user can see it more clearly and conveniently remind the user to perform quality inspection.

[0057] The quality inspection mark can include the mark of unverified, the mark of data error, and the mark of passed verification. The mark of "unverified" means that when the data mark is the first data change mark, the data has not been verified. The mark of "data error" means that data anomalies are found during the data quality verification process and need to be investigated. The mark of "passed verification" means that no data anomalies are found in the data within the data verification range (usually within a stage).

[0058] Based on the above-mentioned embodiment, optionally, the method further includes: when the quality inspection of the quality inspection unit in the current stage is completed, updating the quality inspection mark of the quality inspection unit and the data change mark of the data.

[0059] Specifically, the change of the quality inspection flag can be: when a staged data quality check or a global quality check is triggered for the first time and the calculation is completed, the value of the quality inspection flag "unverified flag" will change. When data anomalies are found in the results of the data quality check, the value of the quality inspection flag is changed to "data error flag". When no data anomalies are found in the results of the data quality check, the value of the quality inspection flag is changed to "passed verification flag".

[0060] Regardless of the current value of the quality inspection mark, the quality inspection unit can be triggered again to perform quality inspection, and the mark value after inspection can still be updated according to the rule that "when data anomalies are found in the results of the data quality inspection, the value of the quality inspection mark in the quality inspection stage is changed to "data error mark"; when no data anomalies are found in the results of the data quality inspection, the value of this quality inspection mark is changed to "pass verification mark"."

[0061] In an optional implementation of an embodiment of the present invention, the calculation identifier of the calculation unit and the quality inspection identifier of the quality inspection unit are updated according to the judgment result, including: when the target data is within the monitoring range of the quality inspection unit, the quality inspection identifier of the quality inspection unit in the current processing stage is updated to an unverified identifier; when the target data is not within the monitoring range of the quality inspection unit, the quality inspection identifier of the quality inspection unit in the current processing stage is kept unchanged.

[0062] The content change of the quality inspection mark at each stage depends on the data change mark of the data within the inspection scope of the quality inspection unit. Figure 4 FIG. 1 is a schematic diagram of the association between a quality inspection mark and a data mark according to an embodiment of the present invention. Figure 4As shown, when the data change mark is changed from "second data change mark" to "first data change mark", that is, from "old data" to "new data", the quality inspection mark of the quality inspection unit responsible for the data quality inspection will be changed from "data error mark" or "verified mark" to "unverified mark". Figure 4 As shown, after completing a data quality check in this stage, the change flag of all data in this stage is recorded as "second data change flag" or "old data". That is, regardless of the result of the data quality check, the data change flag of this data will be changed to "second data change flag" or "old data".

[0063] Digital business can include multiple stages of data quality checks and one global data quality check. Each data quality check function has its corresponding quality inspection mark. The quality inspection marks of these quality inspection units need to be linked.

[0064] In an optional implementation of an embodiment of the present invention, after updating the quality inspection flag of the quality inspection unit of the current processing stage to an unverified flag when the target data is within the monitoring range of the quality inspection unit, it also includes: updating the quality inspection flag of the quality inspection unit of each processing stage after the current processing stage to an unverified flag.

[0065] Figure 5 FIG. 1 is a schematic diagram of the linkage of various quality inspection units provided according to an embodiment of the present invention. Figure 5 As shown, the quality inspection mark of each quality inspection unit of the data quality inspection function needs to monitor the quality inspection mark of its previous quality inspection unit. When the quality inspection mark of the previous quality inspection unit is changed to "unverified mark" or "data error mark", the quality inspection mark of the quality inspection unit of this quality inspection function should be changed to "unverified mark".

[0066] The final result of this linkage is that when the quality inspection mark of a quality inspection unit is changed to "unverified mark" or "data error mark", the quality inspection marks of all subsequent quality inspection units will be changed to "unverified mark".

[0067] On the basis of the above implementation, optionally, the updating of the calculation identification of the calculation unit and the quality inspection identification of the quality inspection unit according to the judgment result also includes: in response to the user's skip instruction for the quality inspection unit of the target processing stage, updating the quality inspection identification of the quality inspection unit of the target processing stage and each processing stage after the target processing stage to an unverified identification, and prompting that the quality inspection processing is skipped in the target processing stage.

[0068] When the user skips an unverified quality inspection unit and triggers the next quality inspection unit, even if the verification result of the next quality inspection unit is "passed verification mark", because the quality inspection mark of the earlier quality inspection unit is "unverified", the quality inspection mark result of the next quality inspection unit cannot be marked as "passed verification mark", but should be marked as "unverified mark". At this time, the user can be provided with corresponding instructions, that is, the skipped quality inspection process is prompted to prevent the user from mistakenly thinking that the quality inspection unit function is incorrect.

[0069] It should be noted that if Figure 5 In the quality inspection unit linkage shown, the quality inspection mark of the stage quality inspection unit does not restrict the use of the delivery function, but if the quality inspection mark of the global quality inspection unit is "unverified mark" or "data error mark", acceptance related actions cannot be performed. In addition, several quality inspection units can also be "batch triggered" to perform data quality inspection at the same time.

[0070] The technical solution of this embodiment determines multiple processing stages of data according to the calculation rules of the data, and determines the monitoring range of the calculation unit and the quality inspection unit of the data in each processing stage; when in the current processing stage, the target data is the first data change identifier, it is judged whether the target data is within the monitoring range of the calculation unit and the quality inspection unit; according to the judgment result, the calculation identifier of the calculation unit and the quality inspection identifier of the quality inspection unit are updated; wherein, the first data change identifier is used to indicate that the data has been updated, which solves the problem of data quality monitoring during the construction of digital twins, and by setting change identifiers, calculation identifiers and quality inspection identifiers for the data, it is convenient to locate data quality problems, reduce the length of the exclusion link, and improve delivery efficiency.

[0071] In order to specifically illustrate the application of the embodiment of the present invention, Figure 6 FIG. 1 is an example diagram of an application of a data quality monitoring method provided according to an embodiment of the present invention. Figure 6 As shown, in scenario 1: in processing stage m-1, new data N1 is added, and the data identifier of N1 is the first data change identifier, that is, "new data". N1 is within the data monitoring range of computing unit M, so the calculation identifier of computing unit M is changed to "identifier that needs to be calculated". At this time, since N1 is not within the data monitoring range of computing unit M+1, the calculation identifier of computing unit M+1 remains unchanged. Similarly, the quality inspection identifier of quality inspection unit M+1 remains unchanged.

[0072] In scenario 2: Based on scenario 1, the calculation of computing unit M is triggered, and the calculation identifier of computing unit M is changed to "the identifier of the calculation completed", and the calculation result generates data N2. The data identifier of N2 is the first data change identifier, namely "new data". Since N2 is within the data monitoring range of computing unit M+1, the calculation identifier of computing unit M+1 is changed to "the identifier of the calculation required". At this time, it still does not affect the quality inspection identifier of quality inspection unit M+1.

[0073] In scenario 3: Based on scenario 2, the calculation of calculation unit M+1 is triggered, and the calculation flag of calculation unit M+1 is changed to "calculation completed flag", and the calculation result generates data N5. The data flag of N5 is the first data change flag, that is, "new data". N5 is within the monitoring range of quality inspection unit M+1, and the quality inspection flag of quality inspection unit M+1 is changed to "unverified flag".

[0074] In scenario 4: Based on scenario 3, the quality inspection units M-1, M and M+1 are triggered, and the quality inspection flag is changed to "verified flag". At the same time, the data flags of data N1, N2 and N5 are changed to the second data change flag, i.e. "old data". Figure 6 As shown in the figure, if the user delivers a new data N3, the calculation flags of calculation unit M and calculation unit M+1 will not change, but the quality inspection flag of quality inspection unit M+1 will change to "unverified flag". If the new data delivered by the user is not N3 but N4, the calculation flag of calculation unit M will not change, but the calculation flag of calculation unit M+1 will change to "need calculation flag", and the quality inspection flag of quality inspection unit M+1 will change to "unverified flag".

[0075] The technical solution of the embodiment of the present invention can realize automatic monitoring of data calculation and quality inspection by setting identifiers on data, computing units and quality inspection units, and automatically updating the computing identifiers and quality inspection identifiers according to the data change identifiers to perform information linkage. When there are abnormalities in the data or the calculation fails, the node that needs to be adjusted can be located according to the identifier, thereby reducing the link troubleshooting length when there are problems with the data and improving delivery efficiency. Through the linkage between the quality inspection units, it can be ensured that after the data changes, the delivered data can be fully covered by quality inspection, thereby improving the efficiency of digital twin construction.

[0076] Figure 7 Schematic diagram of a data quality monitoring device provided according to an embodiment of the present invention. Figure 7 As shown, the device includes: a monitoring range determination module 710, a data monitoring range determination module 720 and an identification update module 730; wherein:

[0077] A monitoring range determination module 710 is used to determine multiple processing stages of the data according to the calculation rules of the data, and to determine the monitoring range of the calculation unit and the quality inspection unit of the data in each processing stage;

[0078] The data monitoring range determination module 720 is used to determine whether the target data is within the monitoring range of the calculation unit and the quality inspection unit when the target data is the first data change identifier in the current processing stage;

[0079] The identification updating module 730 is used to update the calculation identification of the calculation unit and the quality inspection identification of the quality inspection unit according to the judgment result;

[0080] The first data change identifier is used to indicate that the data has been updated.

[0081] Optionally, the data monitoring range determination module 720 includes:

[0082] The data monitoring range judgment unit is used to judge whether the target data is within the monitoring range of the calculation unit and the quality inspection unit according to the production unit identifier of the target data and the current processing stage of the target data when the target data is the first data change identifier in the current processing stage.

[0083] Optionally, the identification updating module 730 includes:

[0084] A calculation identification first updating unit, used for updating the calculation identification of the calculation unit in the current processing stage to an identification that needs to be calculated when the target data is within the monitoring range of the calculation unit;

[0085] The calculation identification second updating unit is used to keep the calculation identification of the calculation unit in the current processing stage unchanged when the target data is not within the monitoring range of the calculation unit.

[0086] Optionally, the identification updating module 730 includes:

[0087] A first quality inspection mark updating unit, used to update the quality inspection mark of the quality inspection unit at the current processing stage to an unverified mark when the target data is within the monitoring range of the quality inspection unit;

[0088] The second quality inspection mark updating unit is used to keep the quality inspection mark of the quality inspection unit in the current processing stage unchanged when the target data is not within the monitoring range of the quality inspection unit.

[0089] Optionally, the device further includes:

[0090] The third quality inspection mark updating unit is used to update the quality inspection mark of the quality inspection unit in the current processing stage to an unverified mark when the target data is within the monitoring range of the quality inspection unit, and then update the quality inspection mark of the quality inspection unit in each processing stage after the current processing stage to an unverified mark.

[0091] Optionally, the device further includes:

[0092] The fourth updating unit of the quality inspection mark is used to execute the update of the calculation mark of the calculation unit and the quality inspection mark of the quality inspection unit according to the judgment result. In response to the user's skip instruction of the quality inspection unit of the target processing stage, the quality inspection mark of the quality inspection unit of the target processing stage and each processing stage after the target processing stage is updated to an unverified mark, and it is prompted that the quality inspection processing is skipped in the target processing stage.

[0093] Optionally, the device further includes:

[0094] The third calculation identification updating unit is used to update the calculation identification of the calculation unit when the calculation of the calculation unit in the current stage is completed; wherein the calculation identification includes: an identification of calculation failure and an identification of calculation completion;

[0095] The fifth updating unit of the quality inspection mark is used to update the quality inspection mark of the quality inspection unit and the data change mark of the data when the quality inspection of the quality inspection unit of the current stage is completed;

[0096] Among them, the quality inspection mark includes: an incorrect data mark and a passed verification mark; the data change mark includes: a first data change mark and a second data change mark; the second data change mark is used to indicate that the data has completed quality inspection.

[0097] The data quality monitoring device provided in the embodiment of the present invention can execute the data quality monitoring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0098] Figure 8 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0099] like Figure 8As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0100] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0101] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a data quality monitoring method.

[0102] In some embodiments, the data quality monitoring method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data quality monitoring method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the data quality monitoring method in any other appropriate manner (e.g., by means of firmware).

[0103] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0105] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0107] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0108] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0109] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0110] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data quality monitoring method, characterized in that: include: Determine multiple data processing stages according to the data calculation rules, and determine the monitoring scope of the data calculation unit and quality inspection unit in each processing stage; When the target data is a first data change identifier in the current processing stage, determining whether the target data is within the monitoring range of the calculation unit and the quality inspection unit; According to the judgment result, the calculation identification of the calculation unit and the quality inspection identification of the quality inspection unit are updated; The first data change identifier is used to indicate that data has been updated.

2. The method according to claim 1, characterized in that When the target data is a first data change identifier in the current processing stage, determining whether the target data is within the monitoring range of the calculation unit and the quality inspection unit includes: When the target data is a first data change identifier in the current processing stage, it is determined whether the target data is within the monitoring range of the calculation unit and the quality inspection unit according to the production unit identifier of the target data and the current processing stage of the target data.

3. The method according to claim 2, characterized in that According to the judgment result, the calculation identification of the calculation unit and the quality inspection identification of the quality inspection unit are updated, including: When the target data is within the monitoring range of the computing unit, updating the computing identifier of the computing unit in the current processing stage to an identifier that needs to be computed; When the target data is not within the monitoring range of the computing unit, the computing identifier of the computing unit in the current processing stage is kept unchanged.

4. The method according to claim 2, characterized in that: According to the judgment result, the calculation identification of the calculation unit and the quality inspection identification of the quality inspection unit are updated, including: When the target data is within the monitoring range of the quality inspection unit, updating the quality inspection flag of the quality inspection unit at the current processing stage to an unverified flag; When the target data is not within the monitoring range of the quality inspection unit, the quality inspection mark of the quality inspection unit in the current processing stage is kept unchanged.

5. The method according to claim 4, characterized in that After updating the quality inspection mark of the quality inspection unit at the current processing stage to an unverified mark when the target data is within the monitoring range of the quality inspection unit, the method further includes: The quality inspection mark of the quality inspection unit in each processing stage after the current processing stage is updated to an unverified mark.

6. The method according to claim 1, characterized in that The step of updating the calculation identification of the calculation unit and the quality inspection identification of the quality inspection unit according to the judgment result further includes: In response to the user's skip instruction for the quality inspection unit of the target processing stage, the quality inspection mark of the quality inspection unit of the target processing stage and each processing stage after the target processing stage is updated to an unverified mark, and it is prompted that the quality inspection process of the target processing stage is skipped.

7. The method according to claim 1, characterized in that Also includes: When the calculation of the calculation unit in the current stage is completed, the calculation flag of the calculation unit is updated; wherein the calculation flag includes: a flag of calculation failure and a flag of calculation completion; When the quality inspection of the quality inspection unit of the current stage is completed, the quality inspection mark of the quality inspection unit and the data change mark of the data are updated; Among them, the quality inspection mark includes: an incorrect data mark and a passed verification mark; the data change mark includes: a first data change mark and a second data change mark; the second data change mark is used to indicate that the data has completed quality inspection.

8. A data quality monitoring device, characterized in that: include: A monitoring scope determination module is used to determine multiple processing stages of data according to the calculation rules of data, and determine the monitoring scope of the data calculation unit and quality inspection unit in each processing stage; A data monitoring range determination module, used for determining whether the target data is within the monitoring range of the calculation unit and the quality inspection unit when the target data is a first data change identifier in the current processing stage; An identification updating module, used for updating the calculation identification of the calculation unit and the quality inspection identification of the quality inspection unit according to the judgment result; The first data change identifier is used to indicate that data has been updated.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the data quality monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data quality monitoring method according to any one of claims 1 to 7 when executed.

Images