Inspection method, device and computer readable storage medium for inspection object

Abstract

Disclosed is a method for auditing an audit object, comprising the following steps: obtaining an audit task for the audit object; obtaining a plurality of attribute data of the audit task; calculating an execution priority weight of the audit task based on at least part of the plurality of attribute data; and adding the audit task into an audit task sequence according to the execution priority weight, the audit task sequence being arranged according to the execution priority weight of each audit task in the audit task sequence, so that each audit task is executed in the order of the audit task sequence. Also disclosed are corresponding devices and computer-readable storage media, etc.

Description

Technical Field

[0001] This disclosure relates to the field of data, and in particular to methods, equipment and computer-readable storage media for inspecting objects. Background Technology

[0002] In recent years, with the advent of the data era, data information has developed towards massive scale and diversification. Data anomalies can lead to serious losses or accidents. With the rapid development of enterprise businesses and the need for comprehensive digital transformation, conducting proactive and effective data governance has become a common consensus across industries. Among these efforts, data quality management occupies a core position in data governance, serving as a crucial foundation for the realization of business value. Data quality issues are becoming cost factors for enterprises, and may even cause related business disruptions.

[0003] In some enterprise business application systems, data quality rules are used to periodically check the objects being checked in order to check the degradation of data quality or changes in data quality over time. If certain minimum data quality key performance indicators (KPIs) are no longer met, an alert is issued so that appropriate actions can be taken.

[0004] Inefficient verification of the targets being inspected can affect the efficiency of business applications, especially in industries such as finance that are sensitive to data quality, where such impacts could cause incalculable losses. Summary of the Invention

[0005] In view of this, this disclosure provides methods, apparatus and computer-readable storage media for inspecting objects, which can alleviate, reduce or even eliminate at least the above-mentioned problems.

[0006] According to a first aspect of this disclosure, a method for inspecting an object is provided, comprising the following steps: obtaining an inspection task for the object; obtaining multiple attribute data of the inspection task; calculating an execution priority weight of the inspection task based on at least some of the attribute data; and adding the inspection task to an inspection task sequence according to the execution priority weight, wherein the inspection task sequence sorts each inspection task according to the execution priority weight of each inspection task, so that each inspection task is executed in the order of the inspection task sequence.

[0007] In one embodiment, at least some of the attribute data includes a first indicator value and a second indicator value. The first indicator value indicates the frequency with which the inspection object experiences quality problems during a first specified time period within a first time period. The second indicator value indicates the frequency with which the inspection object is called by the inspection object's consumer during a second specified time period within a second time period. The first time period is not less than the first specified time period, the second time period is not less than the second specified time period, and both the first time period and the second time period include the current time.

[0008] In one embodiment, at least some of the attribute data also includes the waiting time for the check task.

[0009] In one embodiment, the execution priority weight is calculated as follows:

[0010] T np =αT nw +βQP f +γUO f ,

[0011] Among them, T np It is the execution priority weight of the inspection task, T nw This is the waiting time for the inspection task, QP. f It is the first indicator value, UO f The second indicator value is α, which is the weight of the waiting time for the inspection task, β is the weight of the first indicator value, and γ is the weight of the second indicator value.

[0012] In one embodiment, if the number of times a quality problem occurs in an inspection object within a certain period of time is higher than a first threshold, or continues to increase, or the rate of increase is higher than a second threshold, then the weight of the first indicator value is increased to affect the ranking of the inspection task in the inspection task sequence, wherein the period of time includes multiple first specified time periods.

[0013] In one embodiment, if the number of times the inspection object is called by the inspection object consumer in a second specified time period within a certain period is higher than the third threshold, or continues to increase, or the rate of increase is higher than the fourth threshold, then the weight of the second indicator value is increased to affect the ranking of the inspection task in the inspection task sequence, wherein the period includes multiple second specified time periods.

[0014] In one embodiment, the first indicator value and the second indicator value are dimensionless values ​​determined relative to the corresponding statistical values ​​over a period of time, which is the time interval from when the object being checked went online to the current time.

[0015] In one embodiment, the first indication value is determined as follows:

[0016] Determine the first count, which is the number of times a quality problem occurs in the first specified time period within the first time period of the inspection object;

[0017] The second number is determined as the average number of quality issues that occurred within the first specified time period from the time the inspected object went online until the current time; and...

[0018] Compare the first count and the second count, and assign a value to the first indicator based on the comparison result.

[0019] In one embodiment, assigning a value to the first indication value further includes: in response to the first number being greater than the second number, determining to assign a value to the first indication value within a first numerical interval; in response to the first number being equal to the second number, determining to assign a value to the first indication value within a second numerical interval; and in response to the first number being less than the second number, determining to assign a value to the first indication value within a third numerical interval, wherein the value within the first numerical interval > the value within the second numerical interval > the value within the third numerical interval.

[0020] In one embodiment, the second indication value is determined as follows: a third number is determined, which is the number of times the object being checked is called by the object consumer during a second specified time period within a second time period; a fourth number is determined, which is the average number of times the object being checked is called by the object consumer during a second specified time period within a time period from the object's online date to the current time; and the third number and the fourth number are compared, and the second indication value is assigned based on the comparison result.

[0021] In one embodiment, assigning a value to the second indicator value further includes: determining to assign a value to the second indicator value within a fourth numerical interval in response to the third number being greater than the fourth number; determining to assign a value to the second indicator value within a fifth numerical interval in response to the third number being equal to the fourth number; and determining to assign a value to the second indicator value within a sixth numerical interval in response to the third number being less than the fourth number, wherein: the value within the fourth numerical interval > the value within the fifth numerical interval > the value within the sixth numerical interval.

[0022] In one embodiment, the assignment further includes: within a defined numerical range, assigning a value based on the importance of the object being checked.

[0023] In one embodiment, multiple attribute data include status information of the check task, which indicates whether the check task meets the execution conditions. The method further includes triggering the following steps only when the check task meets the execution conditions: calculating the execution priority weight of the check task; and adding the check task to the check task sequence according to the execution priority weight.

[0024] In one embodiment, the first specified time period and the second specified time period include: a working day.

[0025] In one embodiment, the inspection task includes multiple inspection subtasks, and the method further includes: allocating a number of threads to the inspection task according to the number of multiple inspection subtasks, so as to execute the multiple inspection subtasks in parallel.

[0026] In one embodiment, the method further includes: allocating a companion thread, the companion thread being used to record the execution result log of the check task, and the companion thread starting synchronously with a number of threads.

[0027] In one embodiment, the multiple attribute data includes: the check task readiness time, which indicates the start time when the check task is ready to be executed, wherein the check task waiting time is calculated by subtracting the check task readiness time from the current time.

[0028] In one embodiment, the method further includes: setting a time-based trigger condition to trigger polling for collecting the execution results of the inspection task; determining whether there is currently any execution result collection being performed in response to the trigger condition being met; and performing the collection of the execution results of the inspection task in response to determining that there is currently no execution result collection being performed.

[0029] According to a second aspect of this disclosure, an inspection apparatus for an inspection object is provided, comprising: one or more processors; and one or more memories configured to store computer-executable instructions thereon, which, when executed in the one or more processors, cause the method according to the above aspects of this disclosure to be implemented.

[0030] According to a third aspect of this disclosure, a computer-readable storage medium is provided, which stores instructions that, when executed on one or more computers, cause the computers to implement the methods described above according to this disclosure.

[0031] Overall, by setting and calculating execution priority weights based on certain attribute data, these attribute data can influence the execution priority of inspection tasks, making the execution order of inspection tasks more scientific and reasonable, and allowing for customization based on specific attribute data. Further advantages of the embodiments of this disclosure will be described in the Detailed Description section. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 The illustration shows an exemplary implementation environment in which the verification of the verification object provided by the embodiments of this disclosure can be applied.

[0034] Figure 2 The illustration shows a general flowchart of a method for checking an object according to an embodiment of the present disclosure.

[0035] Figure 3 A further flowchart of a method for checking an object according to an embodiment of the present disclosure is illustrated.

[0036] Figure 4a A further flowchart of a method for checking an object according to an embodiment of the present disclosure is illustrated.

[0037] Figure 4b A further flowchart of a method for checking an object according to an embodiment of the present disclosure is illustrated.

[0038] Figure 5 A further flowchart of a method for checking an object according to an embodiment of the present disclosure is illustrated.

[0039] Figure 6 The illustration shows a hardware environment diagram related to the inspection of an object according to an embodiment of the present disclosure. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0041] It is understood that the terms "first," "second," etc., as used in this disclosure may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are used only to distinguish one concept from another. For example, without departing from the scope of this disclosure, a first indicator value may be referred to as a second indicator value, or a second indicator value may be referred to as a first indicator value. The terms "first," "second," etc., do not indicate importance or sequence of steps.

[0042] As used in this disclosure, the terms "each", "multiple", etc., "multiple" include two or more, and "each" refers to each of the corresponding multiples.

[0043] As used in this disclosure, the term "check" includes the meaning of inspection, verification, and testing.

[0044] As used in this disclosure, the term "database" refers to a data storage system that provides data storage for the management of various things. In a broader sense, any entity that can provide data information can be considered a database.

[0045] Figure 1 This is a schematic diagram of the implementation environment 100 to which the inspection method for the inspection object provided in the embodiments of this disclosure can be applied. See also: Figure 1 This implementation environment may include a management terminal 110 operated by an administrator 150, an audit target terminal 120, and a data quality management platform 130, and may also include a consumer terminal 160 of the audit target operated by a user 170. The management terminal 110 and the audit target terminal 120 are connected to the data quality management platform 130 via a wireless network or a wired network 140. The consumer terminal 160 of the audit target can be connected to both the audit target terminal 120 and the data quality management platform 130 via a wireless network or a wired network 140.

[0046] The management terminal 110 can be a terminal device such as a smartphone, tablet, laptop, or dedicated computer. The management terminal 110 can also be a server-side device, and it can be an independent physical system, a system cluster or distributed system composed of multiple physical systems, or a cloud system that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0047] The management terminal 110 has an application installed and running that can operate the data quality management platform 130 or send management commands to it. For example, the management terminal 110 is a device used by an administrator, where an administrator account is logged into the running application. Note that "administrator" here should be interpreted broadly; it can be a person or any functional entity capable of operating the application, such as a functional entity that triggers operations on the application based on analysis of human behavior. Those skilled in the art will understand that the number of management terminals 110 in the implementation environment can be more or less. For example, there can be only one management terminal 110, or several, etc. The embodiments of this disclosure do not limit the number or type of management terminals 110.

[0048] Users can configure parameters, data sources, and detailed verification results of the data quality management platform 130 through the management terminal, providing necessary auxiliary support for the daily operation of the data quality management platform 130.

[0049] In one embodiment, parameter configuration includes operation and maintenance management operations such as adding, deleting, freezing, modifying, and versioning parameter types.

[0050] In one embodiment, data source configuration includes configuring data source parameters and configuring the maximum number of connections. Configuring data source parameters includes, for example, providing data source connection parameters, such as the database driver name, connection address, authorized username, and password. Configuring the maximum number of data source connections can prevent interference with the operation of the verification object.

[0051] In one embodiment, the detailed configuration of the verification results supports customized configurations for different verification objects and categories of verification objects. This includes, for example, configuring header attribute aliases for the verification results, data display column widths, data display order, and data visibility. Typically, the header of the verification results contains attribute information about the verification results, and the header information varies considerably for different verification objects and categories.

[0052] The verification target 120 can be a server-side device, which can be an independent physical system, a system cluster or distributed system composed of multiple physical systems, or a cloud system that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. The verification target 120 can also be a terminal device such as a smartphone, tablet, laptop, or dedicated computer.

[0053] The verification object terminal 120 is a functional entity that generates or stores verification objects. The verification object terminal 120 may include a database, enterprise business applications, etc. Those skilled in the art will understand that the number of verification object terminals 120 in the implementation environment 100 may be more or less. For example, there may be only one verification object terminal 120, or several, etc. The embodiments of this disclosure do not limit the number or type of verification object terminals 120.

[0054] The data quality management platform 130 is generally a server-side device. It can be a standalone physical system, a system cluster or distributed system composed of multiple physical systems, or a cloud system providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. Of course, the possibility that the data quality management platform 130 is also a terminal device cannot be completely ruled out.

[0055] In one embodiment, the implementation environment 100 further includes a consumer terminal 160 of the object being checked, which is operated by a user 170. The consumer terminal 160 can be a terminal device such as a smartphone, tablet, laptop, or dedicated computer. The consumer terminal 160 can also be a server-side device, and it can be an independent physical system, a system cluster or distributed system composed of multiple physical systems, or a cloud system providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0056] Consumer terminal 160 has an application installed and running that can access the verification target terminal 120. For example, consumer terminal 160 is a device used by a user, where a user account is logged into the running application. Those skilled in the art will understand that the number of consumer terminals 160 for the verification target in implementation environment 100 can be more or less. For example, there can be only one consumer terminal 160 for the verification target, or several, etc. Embodiments of this disclosure do not limit the number or type of consumer terminals 160 for the verification target.

[0057] In one embodiment, the management terminal 110 may be located in the same device as the data quality management platform 130. In another embodiment, the management terminal 110, the data quality management platform 130, and the audit target terminal 120 may all be located in the same device. The management terminal 110 and the data quality management platform 130, as well as the audit target terminal 120 and the data quality management platform 130, may be directly or indirectly connected via wired or wireless communication, and this disclosure does not impose any limitations.

[0058] Cloud computing, mentioned above, is a computing model that distributes computing tasks across a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, the resources in the "cloud" are infinitely scalable, readily available, on-demand, expandable, and pay-as-you-go. As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed within this pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices. Logically, a PaaS (Platform as a Service) layer can be deployed on top of the IaaS layer, and a SaaS (Software as a Service) layer can be deployed on top of the PaaS layer. Alternatively, SaaS can be directly deployed on top of IaaS. PaaS is a platform for running software, such as databases and web containers. SaaS refers to various types of business software, such as web portals and bulk SMS senders. Generally, SaaS and PaaS are upper layers compared to IaaS. As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS) is established. Infrastructure as a Service (IaaS) platforms deploy various types of virtual resources in a resource pool for external customers to choose from. The cloud computing resource pool primarily includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0059] Network 140 may include, but is not limited to, wide area networks, local area networks, wired networks, wireless networks, or any combination thereof.

[0060] In some embodiments of this disclosure, the verification method for the verification object can be implemented jointly by the management terminal 110 and the data quality management platform 130. In other embodiments of this disclosure, the verification method for the verification object can be implemented solely by the data quality management platform 130.

[0061] Figure 2 The illustration shows a general flowchart of a method for checking an object according to an embodiment of the present disclosure. This method can be applied to a data system, such as the one described above. Figure 1The described implementation environment includes a management terminal 110, an audit target terminal 120, and a data quality management platform 130, and may also include an audit target consumer terminal 160. The following section will combine... Figure 2 This is a general flowchart describing the inspection method used for the inspection objects in this implementation environment.

[0062] The overall flowchart may include four stages: defining the relevant data for the inspection task 210, scheduling and executing the inspection task 220, collecting the inspection results 230, and analyzing the inspection results 240.

[0063] First, in step 210, define the data related to the inspection task.

[0064] In one embodiment, defining the data related to the inspection task includes classifying the inspection objects. This classification is based on industry data, business rule characteristics, etc. Table 1 below shows several major categories and some examples of specific questions under each category. Of course, the categories of inspection objects and the specific questions under each category in the table below are only examples and are not intended to limit this disclosure in any way.

[0065] Table 1. Categories of Inspection Targets

[0066] .

[0067] In one embodiment, defining the data related to the inspection task includes dividing the quality of the inspection object into dimensions. For example, these dimensions may include one or more of the following: accuracy, relevance, timeliness, consistency, availability, and uniqueness.

[0068] Among these, accuracy refers to the correctness of data values ​​or calculations; relevance refers to the index relationships or functional relationships in retrieval; consistency refers to the consistency of data involved in data synchronization or year-on-year or month-on-month data calculations, including but not limited to the consistency of data encoding, naming, and meaning; timeliness refers to the timeliness of periodic data updates or data transmission; availability refers to real-time availability, including whether the data can be downloaded or accessed; and uniqueness refers to the uniqueness of data in data access and display, data calculation and value retrieval, and master data management.

[0069] In one embodiment, the determination of the aforementioned measurement dimensions can be based on the collection, classification, abstraction, and generalization of the business rules targeted by the check, and adapted to the application scenario. For example, for search business, the relevance of search results is more important; if combined with a download scenario, the availability of search results also needs to be considered.

[0070] In a further example, defining the data related to the inspection task also includes fine-grained subdivision of the aforementioned dimensions and defining corresponding measurement rules. For example, for accuracy, it can be further subdivided into smaller granularities: completely accurate, basically accurate, generally accurate, and inaccurate. The measurement rules could be: in response to the inspection object being 100% accurate, the inspection object is determined to be completely accurate; in response to the inspection object being less than 100% but more than 80% accurate, the inspection object is determined to be basically accurate; in response to the inspection object being less than 80% but more than 60% accurate, the inspection object is determined to be generally accurate; and in response to the inspection object being less than 60% accurate, the inspection object is determined to be inaccurate. In this article, "more than" includes the number itself, that is, for example, 80% or more includes 80% itself.

[0071] In one embodiment, the determination of the above-mentioned measurement rules can be authorized to the administrator. The administrator can define measurement rules for various factors such as different business operations, different categories of audit objects, and different application scenarios.

[0072] In one embodiment, defining the data related to the inspection task includes defining the inspection method for the inspection object.

[0073] The inspection object referred to in this article is a carrier that describes the data quality problems of a business entity based on measurement rules. It is generally a business unit with inspectable granularity. It can include one or more information items. For example, the inspection object can be a data source change, including three information items: field type, data content, and data format.

[0074] Different verification methods can be defined for different information items. Verification methods are the core of the verification task. For example, for field type changes, the verification method is to determine whether they are reported; for data content, the verification method is to determine whether it conforms to the data definition; for data format, the verification method is to determine whether it matches the defined data length, and so on.

[0075] Each verification method defines a specific verification execution script and associated attribute information based on its verification object. In one embodiment, the data quality management platform 130 provides a configuration template for the verification execution script to facilitate the definition of specific verification execution scripts. The management terminal 110 can display a visual interface of the verification execution script configuration template, import the template to automatically generate verification methods, and the configuration template can also automatically verify the correctness of the verification methods, identify incorrectly configured verification execution scripts, and provide error information queries.

[0076] The definition and subsequent management of the data related to the inspection tasks mentioned above (such as the categories of inspection objects, the dimensions of the quality of inspection objects, the fine-grained division of the above dimensions, the corresponding measurement rules, inspection methods, etc.) can all be achieved through... Figure 1 The management interface shown is entered by the administrator. Of course, it can also be automatically generated during system initialization, or by any other reasonable method; this disclosure does not limit this.

[0077] In step 220, the verification task is scheduled and executed. This step is a core step performed at the data quality management platform 130. By executing the verification task, corresponding verification results are generated, i.e., problem data is obtained. This problem data can directly reflect data quality issues, especially the quality issues of the data that the consumer end is concerned about.

[0078] Figure 3 The illustration shows a further flowchart of a method for checking an object according to an embodiment of the present disclosure, namely a flowchart of a method 300 for scheduling and executing a check task. This method 300 can be applied to a data system including the above-described combination. Figure 1 The management terminal 110, the verification object terminal 120, and the data quality management platform 130 in the described implementation environment may also include the consumer terminal 160 of the verification object. The following is in conjunction with... Figure 3 This is a flowchart that describes the overall method for performing inspection tasks in a data system.

[0079] In step 2201, the inspection tasks for the inspection object are obtained. In one embodiment, the various inspection tasks are uniformly scheduled and managed. For example, inspection tasks are automatically generated within the job cycle of the business application of the enterprise where the inspection object is located, awaiting scheduling and execution. The specific scheduling method will be described below.

[0080] In step 2202, multiple attribute data for the inspection task are obtained. The multiple attribute data may include one or more of the following:

[0081] T n The identification number of the inspection task, used to uniquely identify the inspection task;

[0082] T nsThe current status information of the inspection task, in one embodiment, indicates whether the inspection task has the conditions for execution (e.g., whether it requires the execution of another inspection task as a prerequisite), so that in a further example, steps 2203 and 2204 are triggered only when the inspection task has the conditions for execution. In another example, the current status includes waiting (e.g., represented by the number "1"), ready (e.g., represented by the number "0"), and completed (e.g., represented by the number "2"), where waiting indicates that the inspection task does not have the conditions for execution, for example, the execution of the inspection task requires the execution of another inspection task as a prerequisite, and the other inspection task has not yet been generated or executed; ready indicates that the inspection task has the conditions for execution; and completed indicates that the inspection task has been completed.

[0083] T nr The preparation time of the inspection task, which indicates the start time when the inspection task is ready to be performed;

[0084] T nw The waiting time for this inspection task can be understood as: subtracting the preparation time T of the inspection task from the current time. nr ;

[0085] QP f A frequency indicator value, indicating the frequency of quality problems occurring in the inspected object within a first specified time period (e.g., a workday) in a first time period. For distinction, it is referred to as the first indicator value in this document.

[0086] UO f This is another frequency indicator, indicating the frequency with which the checked object is called by the checked object's consumer during a second specified time period (e.g., two working days) within the second time period. For distinction, it is referred to as the second indicator in this document. The first specified time period and the second specified time period can be the same or different.

[0087] The first time period mentioned above is not less than the first specified time period, and is generally an integer multiple of the first specified time period. The second time period is not less than the second specified time period, and is generally an integer multiple of the second specified time period. Both the first and second time periods include the current time, reflecting the timeliness requirement. The first and second time periods can be the same or different.

[0088] In one embodiment, the first and second indication values ​​are dimensionless values ​​determined relative to corresponding statistical values ​​over a period of time, which is preferably the time period from when the object being checked went online to the present time.

[0089] See Figure 4aIn a further example, the first indication value is determined as follows: First, in step 22021, a first count and a second count are determined. Here, the first count is the number of times the inspected object has a quality problem within a first specified time period. To reflect timeliness, this first specified time period is a time period including the current time (referred to as the first time period here to distinguish it from a time period mentioned below), such as a working day in the past week, or a working day in the past three working days, or even a working day in the past working day (i.e., the first time period is equal to the first specified time period). The first count can be determined by counting the number of quality problems that occurred within the first time period and then proportionally extrapolating the number of quality problems that occurred within the first specified time period; therefore, this is an average count. Alternatively, a time period within the aforementioned time period can be selected to directly count the number of quality problems that occurred within that time period. This disclosure is not intended to impose any limitations on this. The second number is the average number of quality problems that have occurred in the first specified time period between the time period from the time the object was put online to the current time; then in step 22022, the first number and the second number are compared, and the first indication value is assigned a value based on the comparison result.

[0090] In a further example, assigning a value to the first indicator value further includes: in response to the first number being greater than the second number, determining to assign a value to the first indicator value within a first numerical interval; in response to the first number being equal to the second number, determining to assign a value to the first indicator value within a second numerical interval; and in response to the first number being less than the second number, determining to assign a value to the first indicator value within a third numerical interval, wherein:

[0091] The value in the first numerical interval > the value in the second numerical interval > the value in the third numerical interval.

[0092] For example, the dimensionless value can take the number in the interval [0,10]. When the first number is greater than the corresponding statistical value over a period of time, i.e. the second number, it takes the value in the interval [8,10]. When the first number is equal to the second number, it takes the value in the interval [4,8). When the first number is less than the second number, it takes the value in the interval [0,4).

[0093] For example, the specific value in each interval can also be determined based on the importance of the audit object. For instance, in the interval [8,10], the value is 10 for an audit object of important business. In a further example, the specific value can be an integer, so the above interval [0,10] is actually divided into three intervals: [0,3], [4,7], and [8,10].

[0094] Similarly, the second indicator value mentioned above is also assigned in a similar manner. That is, the corresponding statistical value over a period of time is used as a reference to determine the dimensionless value. In particular, the average number of times the object being called by the consumer end of the object during the second specified time period (e.g., a working day) from the time the object went online to the current time period is used as a reference.

[0095] Specifically, see Figure 4b The second indicator value is determined as follows: First, in step 22023, the third and fourth counts are determined. The third count is the number of times the object being checked is called by the object's consumer end within a second specified time period. To reflect timeliness, this second specified time period is a time period including the current time (referred to as the second time period here to distinguish it from the above), such as a working day in the past week, or a working day in the past three working days, or even a working day in the past working day (i.e., the second time period is equal to the second specified time period). The second time period can be equal to or unequal to the first time period. The third count can be determined by counting the number of times the object being checked is called by the consumer end within the second time period, and then proportionally extrapolating the number of times the object being checked is called by the consumer end within the second specified time period; therefore, this is an average count. Alternatively, a time period within the second time period can be selected to directly count the number of times the object being checked is called by the consumer end within that time period. This disclosure does not intend to impose any limitations on this. The fourth count is the average number of times the object being checked has been called by the object's consumer during the second specified time period between the object's online launch and the current time. Then, in step 22024, the third count and the fourth count are compared, and the second indicator value is assigned based on the comparison result.

[0096] It is understood that steps 22021-22022 and steps 22023-22024 can be executed simultaneously or sequentially, and this disclosure is not intended to limit this.

[0097] In a further example, assigning the second indicator value further includes: determining that the second indicator value is assigned within a fourth numerical interval in response to the third number being greater than the fourth number; determining that the second indicator value is assigned within a fifth numerical interval in response to the third number being equal to the fourth number; and determining that the second indicator value is assigned within a sixth numerical interval in response to the third number being less than the fourth number, wherein:

[0098] The value in the fourth numerical interval > the value in the fifth numerical interval > the value in the sixth numerical interval.

[0099] For example, the dimensionless value can take the number in the interval [0,10]. When the third number is greater than the corresponding statistical value over a period of time, i.e. the fourth number, it takes the value in the interval [8,10]. When the third number is equal to the fourth number, it takes the value in the interval [4,8). When the third number is less than the fourth number, it takes the value in the interval [0,4).

[0100] For example, the specific value in each interval can also be determined based on the importance of the audit object. For instance, in the interval [8,10], the value is 10 for an audit object of important business. In a further example, the specific value can be an integer, so the above interval [0,10] is actually divided into three intervals: [0,3], [4,7], and [8,10].

[0101] It should be understood that the division of the interval and subintervals of the dimensionless values ​​of the second indicator value can be the same as or different from the division of the interval and subintervals of the dimensionless values ​​of the first indicator value described above.

[0102] The first and second indicator values ​​of the inspected object are influenced by multiple complex factors such as data capture, entry, cleaning, database storage, dynamic calculation, business changes, code iteration, and even data transmission, and generally do not conform to a normal distribution. This disclosure uses the corresponding statistical values ​​over a period of time as a reference to determine the dimensionless values, especially the average number of quality problems occurring within a specified time period (e.g., one working day) since the inspected object went online, which is more valuable for reference. The inventors of this application have considered that the above-mentioned corresponding statistical values ​​gradually converge as quality problems are detected and resolved, and therefore have reference value.

[0103] In step 2203, the execution priority weight of the inspection task is calculated based on at least some of the attribute data from multiple attribute data.

[0104] By sorting the execution priority weights of each inspection task, a sequence of inspection tasks is obtained, allowing the data quality management platform to execute each task in the order of the sequence. In one embodiment, a higher execution priority weight indicates a higher priority for execution, and the corresponding inspection task will be ranked earlier in the inspection task sequence. In another embodiment, the execution priority weight is based on the waiting time T of the inspection task. nw The aforementioned first indicator value QP f And the second indication value UO mentioned above f It is calculated from three factors:

[0105] T np =αT nw +βQP f +γUO f (1)

[0106] Among them, T np The weights represent execution priority weights, where α is the weight of the waiting time for the inspection task, β is the weight of the first indicator value, and γ is the weight of the second indicator value.

[0107] Typically, in formula (1), α is set to be much larger than β and γ, so that the waiting time of the inspection task plays a major role, thereby ensuring that the execution of the inspection task basically meets the first-in, first-out principle. However, in some cases, when the first indicator value indicates that the inspection object has a high frequency of quality problems in the first specified time period (e.g., one working day) within a certain period of time (e.g., within the last week, month, or two months), such as the first number always being higher than a preset threshold, or continuously increasing, or increasing at a rate greater than a preset threshold, the weight β of the first indicator value is increased. Similarly, when the second indicator value indicates that the inspection object is frequently called by the inspection object consumer in the second specified time period (e.g., one working day) within a certain period of time (e.g., within the last week, month, or two months), such as the third number always being higher than a preset threshold, or continuously increasing, or increasing at a rate greater than a preset threshold, the weight γ of the second indicator value is increased. The increase of the weights β and γ is sufficient to affect the change in the order of the corresponding inspection task in the inspection task sequence.

[0108] Alternatively, instead of increasing the weights β and γ, one can choose to decrease the weight α, which will also affect the change in the order of the corresponding inspection tasks in the inspection task sequence.

[0109] In step 2204, the inspection task is added to the inspection task sequence according to the execution priority weight calculated in step 2203. The inspection task sequence sorts the inspection tasks according to their execution priority weights so that the inspection tasks are executed in the order of the inspection task sequence.

[0110] Each time a new inspection task is obtained, its execution priority weight is calculated. Then, based on the execution priority weights of the other inspection tasks in the sequence, the position of the new inspection task in the sequence is determined. In other words, if the new inspection task has the highest execution priority weight, it is placed at the beginning of the sequence; if the new inspection task has the lowest execution priority weight, it is placed at the end of the sequence.

[0111] According to embodiments of this disclosure, by setting and calculating execution priority weights based on certain data attributes, these data attributes can influence the execution priority of inspection tasks, making the execution order of inspection tasks more scientific and reasonable, and customizable based on specific factors. Furthermore, by dynamically adjusting the weights of these data attributes, the scheduling and execution of inspection tasks becomes more intelligent and flexible.

[0112] In one embodiment, a verification task may include multiple verification subtasks, such as multiple verification methods. Accordingly, method 300 further includes: allocating a corresponding number of threads to the verification task based on the number of its multiple verification subtasks, so as to execute the multiple verification subtasks in parallel. In this way, verification efficiency can be improved task-by-task. It maintains the execution order of each verification task while introducing parallel execution to improve verification efficiency.

[0113] In one embodiment, method 300 further includes allocating a companion thread for recording the execution result log of the check task. The companion thread starts synchronously with the execution thread of the check task. When the check task includes multiple check subtasks and a corresponding number of threads are allocated to execute these subtasks, the companion thread and these corresponding number of threads start synchronously. By introducing a companion task that executes in parallel, efficiency is further improved.

[0114] Back Figure 2 In step 230, the inspection results are collected. The result data obtained from executing the inspection method will be generated into corresponding inspection result files, such as DAT files, XML files, etc. The inspection result files are stored in a predetermined location, such as a folder named after the inspection date of the current inspection task plus the application where the inspection object is located.

[0115] In one embodiment, the Quartz job scheduling framework is used to perform polling collection of check results. Quartz is an open-source job scheduling framework written entirely in Java and designed for use in J2SE and J2EE applications. It offers great flexibility without sacrificing simplicity. It can be used to create simple or complex schedules for executing a job. Quartz implements functionality in three layers: (1) Job: the specific work to be performed, which in this disclosure is the collection of check results; (2) Trigger: triggers the execution of the task by setting specific time conditions, including simple triggers and complex triggers; (3) Scheduler: the actual executor of the task, responsible for gluing the task and trigger, and executing the task based on the trigger.

[0116] The trigger's time condition might be simple, such as being set to every fixed time interval. However, due to variations in the number of files for each check result or other interfering factors, the data collection for each check task may differ in timing. This could result in the previous check result collection not yet finishing when the next check result collection task starts. For example, the trigger might be set to poll every second; however, when check result B is polled, the collection of check result A might not have finished.

[0117] In one embodiment, the collection of inspection results adopts a single-job execution mode. That is, if the previous collection job has not yet finished when the current collection job starts, the current collection job will automatically end, and the next collection job will not start until the previous collection job is completed. In this way, the resource contention problem between different batches of jobs can be avoided. For example, when inspection result B is polled, if the collection of inspection result A has not yet finished, then the collection of inspection result B will not be performed, and the collection of inspection result A will be waited for.

[0118] Figure 5 The flowchart for the example above is described. In step 2301, a time-based trigger condition is set to trigger polling for collecting the execution results of the check task. In step 2302, it is determined whether the trigger condition is met. If the condition is not met, the process continues to wait; if the condition is met, in step 2303, it is determined whether there is currently any execution result collection being performed. If it is determined that there is no execution result collection being performed, then in step 2304, the execution result collection for the check task is performed. If it is determined that there is an execution result collection being performed, no new collection is performed; instead, the process waits.

[0119] In one embodiment, the data acquisition task collects the inspection result files into a database. During this process, a data pre-summarization operation is performed on these inspection results, and the detailed data and summary data of the inspection results are stored in different files, such as a detailed table and a summary table of inspection results. If an error or exception occurs during this process, a database rollback operation is performed to avoid incomplete data, and the files with errors or exceptions are moved to a dedicated directory.

[0120] In one embodiment, all inspection result files are deleted after being collected into the database. For example, after all inspection result files have been collected, it is determined whether there are any uncollected inspection result files in the predetermined location (e.g., a folder named after the inspection date of the current inspection task plus the application where the inspection object is located). If not, the current folder is deleted; if there are, for example, files that were not collected because the ready files have not arrived, all files in the current folder are retained, waiting for the next collection time.

[0121] Then, in step 240, the inspection results can be analyzed using the database that has collected the inspection results.

[0122] In one embodiment, the database can provide data retrieval, such as retrieval based on a single condition or a combination of conditions, to filter the inspection results. Alternatively or optionally, the summary or detailed data can be customized in terms of configuration or sorting, for example, by customizing the data presentation according to the business unit where the inspection task is located, the inspection category, etc. Alternatively or optionally, based on predetermined conditions, such as the inspection cycle and inspection method, the database can display the changing trends of data quality issues and the results of quality issue remediation in the inspection results. Alternatively or optionally, based on the inspection results, a data quality analysis report can be provided, which supports text or graphical presentation and export. Alternatively or optionally, based on the inspection results, customized key monitoring inspection methods can be implemented to meet the different needs of data consumer users, facilitating centralized analysis and management.

[0123] See Figure 6 In an embodiment of the present invention, the inspection method for the inspection object can be executed on the data quality management platform 130. Although Figure 6 The data quality management platform 130 is illustrated as a single device 600, but in reality, it can be multiple devices, each including a processor 604, which includes hardware components 610. The processor 604 may include, for example, one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. As used herein, the term "processor" may refer to any of the above-described structures or any other structures suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for checking the audited object, or incorporated into combined hardware and / or software modules. Furthermore, the techniques may be fully implemented in one or more circuit or logic elements. The methods in this disclosure may be implemented in various components, modules, or units, but do not necessarily require implementation through different hardware units. Rather, as described above, various components, modules, or units may be combined or provided by a collection of interoperable hardware units (including one or more processors as described above) combined with suitable software and / or firmware.

[0124] In one or more examples, the above combination Figures 1-5The described content can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality can be stored or transmitted as one or more instructions or code on or via computer-readable medium 606 and executed by a hardware-based processor. Computer-readable medium 606 may comprise a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium comprising any medium that facilitates the transmission of a computer program (including one or more of the aforementioned instructions or code) from one place to another, for example, according to a communication protocol. In this way, computer-readable medium 606 may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. Data storage medium may be any available medium that can be read by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. A computer program product may comprise computer-readable medium 606 and one or more instructions or code stored thereon.

[0125] For example, and not as a limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc memory, disk memory or other magnetic memory, flash memory, or any other memory 612 that can be used to store desired program code in the form of instructions or data structures and is readable by a computer. Furthermore, any connection is appropriately referred to as computer-readable medium 606. For example, if instructions are transmitted from a website, billing system, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, these are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs use lasers to reproduce data optically. The above combination should also be included within the scope of computer-readable medium 606.

[0126] Device 600 may also include I / O interfaces for transmitting data, and other functions 614. Device 600 may also be included in different terminals, such as computer 616, mobile device 618, and other terminals 620, etc. Each of these configurations includes devices that can have generally different constructions and capabilities, and thus can be used for verification of the object to be verified according to one or more configurations in different device categories. Furthermore, the technology of the present invention can also be implemented, in whole or in part, on a distributed system, such as through a platform 624 as described below, in the “cloud” 622.

[0127] Cloud 622 includes and / or represents platform 624 for resource 626. Platform 624 abstracts the underlying functionality of the hardware (e.g., billing system) and software resources of cloud 622. Resource 626 may include applications and / or data that can be used when performing computer processing on a data system remote from computing device 600. Resource 626 may also include services provided via the Internet and / or via subscriber networks such as cellular or Wi-Fi networks.

[0128] Platform 624 can abstract resources and functions to connect computing devices to other computing devices. Platform 624 can also be used to abstract resource hierarchy to provide a tiered hierarchy of the appropriate level of demand for resource 626 implemented via platform 624. Therefore, in interconnect device embodiments, the implementation of the functions described herein can be distributed throughout device 600. For example, functions can be implemented partly on the computing device and partly through platform 624, which abstracts the functions of cloud 622.

[0129] According to embodiments of this disclosure, by setting and calculating execution priority weights based on several factors, these factors can influence the execution priority of inspection tasks, making the execution order of inspection tasks more scientific and reasonable, and customizable based on specific factors. Furthermore, by dynamically adjusting the weights of these factors, the scheduling and execution of inspection tasks becomes more intelligent and flexible.

[0130] Unless otherwise specified or without preconditions (i.e., the execution of one step depends on the result of the execution of another step), the order in which the method steps are described does not indicate their execution order. The described method steps can be executed in any possible and reasonable order.

[0131] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0132] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for inspecting an object, characterized in that, Includes the following steps: Obtain inspection tasks for the inspection targets; Obtain multiple attribute data of the inspection task; Based on at least some of the attribute data from the plurality of attribute data, the execution priority weight of the inspection task is calculated. The at least some attribute data includes a first indicator value and a second indicator value. The first indicator value indicates the frequency with which the inspection object experiences quality problems within a first specified time period in a first time period. The second indicator value indicates the frequency with which the inspection object is called by the inspection object consumer within a second specified time period in a second time period. The first time period is not less than the first specified time period, and the second time period is not less than the second specified time period. Both the first time period and the second time period include the current time. The first indicator value is determined as follows: a first count is determined, where the first count is the number of times the inspection object experiences quality problems within the first time period. The number of quality problems occurring within the first specified time period; determining a second number, where the second number is the average number of quality problems occurring within the first specified time period during the time interval from the time the inspected object went online to the current time; comparing the first number and the second number, and in response to the first number being greater than the second number, determining to assign a value to the first indicator value within a first numerical interval, in response to the first number being equal to the second number, determining to assign a value to the first indicator value within a second numerical interval, and in response to the first number being less than the second number, determining to assign a value to the first indicator value within a third numerical interval, wherein the value within the first numerical interval > the value within the second numerical interval > the value within the third numerical interval; and The inspection task is added to the inspection task sequence according to the execution priority weight. The inspection task sequence sorts the inspection tasks according to the execution priority weight of each inspection task so that the inspection tasks are executed in the order of the inspection task sequence.

2. The method as described in claim 1, characterized in that, The at least some of the attribute data also includes the waiting time for the inspection task.

3. The method as described in claim 2, characterized in that, The execution priority weight is calculated as follows: T np =αT nw +βQP f +γUO f , Among them, T np T is the execution priority weight of the inspection task. nw This is the waiting time for the aforementioned inspection task, QP. f It is the first indication value, UO f The second indication value is α, which is the weight of the waiting time of the inspection task, β is the weight of the first indication value, and γ is the weight of the second indication value.

4. The method as described in claim 3, characterized in that, If the number of times the inspection object has quality problems in the first specified time period is higher than the first threshold, or continues to increase, or the rate of increase is higher than the second threshold, then the weight of the first indicator value is increased to affect the ranking of the inspection task in the inspection task sequence, wherein the time period includes multiple first specified time periods.

5. The method as described in claim 3, characterized in that, If the number of times the inspection object is called by the inspection object consumer in the second specified time period is higher than the third threshold, or continues to increase, or the rate of increase is higher than the fourth threshold, then the weight of the second indicator value is increased to affect the ranking of the inspection task in the inspection task sequence, wherein the time period includes multiple second specified time periods.

6. The method as described in claim 1, characterized in that, The first and second indication values ​​are dimensionless values ​​determined relative to corresponding statistical values ​​over a period of time, which is the time interval from when the object under inspection went online to the present time.

7. The method as described in claim 1, characterized in that, The second indication value is determined as follows: The third number is determined as the number of times the object being checked is invoked by the object's consumer during the second specified time period within the second time period. The fourth number is determined as the average number of times the inspection object is called by the inspection object consumer during the second specified time period within the time period between the inspection object's online date and the current time. as well as The third count and the fourth count are compared, and the second indicator value is assigned based on the result of the comparison.

8. The method as described in claim 7, characterized in that, The step of assigning a value to the second indication value further includes: In response to the third count being greater than the fourth count, a value is assigned to the second indicator within a fourth numerical interval; in response to the third count being equal to the fourth count, a value is assigned to the second indicator within a fifth numerical interval; and in response to the third count being less than the fourth count, a value is assigned to the second indicator within a sixth numerical interval, wherein: The value in the fourth numerical interval > the value in the fifth numerical interval > the value in the sixth numerical interval.

9. The method as described in claim 6 or 8, characterized in that, The assignment further includes: Within a defined numerical range, values ​​are further assigned based on the importance of the object being checked.

10. The method as described in claim 1, characterized in that, The plurality of attribute data includes the status information of the inspection task, which indicates whether the inspection task meets the execution conditions, and the method further includes: The following steps are triggered only when the inspection task meets the execution conditions: The calculation of the execution priority weight of the inspection task; and The inspection task is added to the inspection task sequence according to the execution priority weight.

11. The method as described in claim 1, characterized in that, The first specified time period and the second specified time period include: one working day.

12. The method as described in claim 1, characterized in that, The inspection task includes multiple inspection subtasks, and the method further includes: allocating the number of threads to the inspection task according to the number of the multiple inspection subtasks, so as to execute the multiple inspection subtasks in parallel.

13. The method as described in claim 12, characterized in that, Also includes: A companion thread is allocated, which is used to record the execution result log of the inspection task, and the companion thread starts synchronously with the number of threads.

14. The method as described in claim 2, characterized in that, The multiple attribute data include: The preparation time of the inspection task indicates the start time when the inspection task is ready to be performed. The waiting time for the inspection task is calculated as: the current time minus the preparation time of the inspection task.

15. The method as described in claim 1, characterized in that, Also includes: Set time-based trigger conditions to trigger polling for the collection of execution results of the inspection task; In response to the fulfillment of the triggering condition, it is determined whether there is currently an ongoing process for collecting execution results; In response to determining that there is currently no execution result being collected, the execution result of the check task is collected.

16. An inspection device for an inspection object, characterized in that, include: One or more processors; as well as One or more memories configured to store computer-executable instructions thereon, which, when executed in the one or more processors, cause the method of any one of claims 1-15 to be implemented.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on one or more computers, cause the one or more computers to perform the method as described in any one of claims 1-15.

Images