Data detection method and device and electronic equipment

Abstract

The invention discloses a data detection method and device and electronic equipment. Relates to the field of financial science and technology, and comprises the following steps: determining to-be-detected network assets from a database, and obtaining configuration information of the to-be-detected network assets, the database being used for storing the configuration information of initial network assets; acquiring a data detection rule associated with the to-be-detected network assets from a rule base, and determining a detection mode for detecting the to-be-detected network assets; and detecting the configuration information according to the detection mode and the data detection rule to obtain a detection result. Through the method and the device, the problem of relatively low accuracy of the configuration information of the network assets in the database in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of financial technology, and more specifically, to a data detection method, apparatus, and electronic device. Background Technology

[0002] Currently, when managing the network assets of financial institutions, the configuration management database serves as the core information storage and management system. It is necessary to ensure the accuracy of the asset configuration information stored in it. However, during the data entry process of the configuration management database, data quality problems may occur due to factors such as human error, delays in information updates after system changes, and the highly dynamic nature of the IT (Information Technology) environment itself.

[0003] For example, common data quality issues include incorrect IP (Internet Protocol) address formats and missing service attribution information. The existence of these issues will undoubtedly weaken the reliability and effectiveness of the configuration management database, thereby affecting the overall operational efficiency and decision support capabilities of IT services provided by the network assets of financial institutions.

[0004] There is currently no effective solution to the problem of low accuracy of network asset configuration information in databases in related technologies. Summary of the Invention

[0005] The main objective of this application is to provide a data detection method, apparatus, and electronic device to solve the problem of low accuracy of network asset configuration information in databases in related technologies.

[0006] To achieve the above objectives, according to one aspect of this application, a data detection method is provided. The method includes: identifying a network asset to be detected from a database and obtaining configuration information of the network asset to be detected, wherein the database is used to store the initial configuration information of the network asset; obtaining data detection rules associated with the network asset to be detected from a rule base and determining a detection mode for detecting the network asset to be detected; and detecting the configuration information according to the detection mode and the data detection rules to obtain a detection result.

[0007] Optionally, the configuration information of each initial network asset includes multiple configuration items and configuration data for each configuration item. Determining the network asset to be detected from the database includes: determining the data detection period for each initial network asset stored in the database; determining the initial network asset for which data detection operation needs to be performed based on the data detection period to obtain the network asset to be detected; or, if a rule change operation is detected in the rule base, obtaining the target rule associated with the rule change operation and obtaining the description information of the target rule; determining the configuration item associated with the target rule from the database based on the description information, and determining the target network asset associated with the configuration item as the network asset to be detected, wherein the target network asset is the initial network asset to which the configuration information of the configuration item belongs.

[0008] Optionally, determining the configuration item associated with the target rule from the database based on the description information includes: parsing the description information to obtain multiple keywords; matching each keyword with the configuration items in each configuration information stored in the database, and determining the configuration item that matches any keyword as the configuration item associated with the target rule.

[0009] Optionally, determining the detection mode for the network asset to be detected includes: obtaining historical detection records of the network asset to be detected, and determining the time difference between the last time the network asset was detected using the first detection mode and the current time based on the historical detection records; if the time difference is greater than a time difference threshold, determining the detection mode as the first detection mode; if the time difference is less than or equal to the time difference threshold, determining whether the data detection rules have changed; if the data detection rules have changed, determining the detection mode as the first detection mode; if the data detection rules have not changed, determining the detection mode as the second detection mode, wherein the detection accuracy of the first detection mode is greater than that of the second detection mode.

[0010] Optionally, the configuration information is detected according to the detection mode and data detection rules to obtain the detection results, including: adjusting the data detection rules according to the detection mode to obtain updated data detection rules; obtaining the sub-detection rules corresponding to each configuration item in the configuration information from the updated data detection rules to obtain M sub-detection rules, where M is a positive integer; using each sub-detection rule to detect the configuration data of the configuration item corresponding to the sub-detection rule to obtain M sub-detection results; and combining the M sub-detection results into a detection result.

[0011] Optionally, the data detection rules are adjusted according to the detection mode to obtain the updated data detection rules, including: keeping the data detection rules unchanged when the detection mode is the first detection mode; and obtaining the fuzzy matching rule for each detection rule in the data detection rules when the detection mode is the second detection mode, and using the fuzzy matching rule as the updated data detection rule, wherein the detection accuracy of the first detection mode is greater than that of the second detection mode.

[0012] Optionally, the method further includes: receiving a rule change instruction when the rule base detects a rule change instruction, and determining a rule verification process based on the instruction content of the rule change instruction; verifying the rule change instruction according to the rule verification process, obtaining a verification result, and performing rule change operations on the data in the database according to the rule change instruction if the verification result indicates that the rule change instruction does not have any abnormalities.

[0013] To achieve the above objectives, according to another aspect of this application, a data detection apparatus is provided. The apparatus includes: a first acquisition unit, configured to determine a network asset to be detected from a database and acquire configuration information of the network asset to be detected, wherein the database is used to store initial configuration information of the network asset; a second acquisition unit, configured to acquire data detection rules associated with the network asset to be detected from a rule base and determine a detection mode for detecting the network asset to be detected; and a detection unit, configured to detect the configuration information according to the detection mode and the data detection rules to obtain a detection result.

[0014] To achieve the above objectives, according to another aspect of this application, an electronic device is provided, the electronic device including a memory storing an executable program; and a processor for running the program, wherein the program executes the above-described data detection method when it runs.

[0015] To achieve the above objectives, according to another aspect of this application, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the data detection method described above.

[0016] In this embodiment, the method involves determining the network asset to be detected from a database and obtaining its configuration information. The database stores the initial configuration information of the network assets. Data detection rules associated with the network asset to be detected are obtained from a rule base, and a detection mode for detecting the network asset is determined. The configuration information is then detected according to the detection mode and data detection rules to obtain the detection result. By determining the network asset to be detected in the database and establishing data detection rules for it, the configuration information of the network asset to be detected is detected using these rules and the detection mode. This achieves the goal of improving the accuracy of the network asset to be detected by detecting its configuration information, thereby improving the accuracy of the configuration information of each network asset in the database. This solves the technical problem of low accuracy of network asset configuration information in databases in related technologies. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a computer terminal for implementing a data detection method is shown.

[0019] Figure 2 This is a flowchart of the data detection method provided according to Embodiment 1 of this application;

[0020] Figure 3 This is a schematic diagram of the data detection system provided in Embodiment 1 of this application;

[0021] Figure 4 This is a schematic diagram of the data detection device provided according to Embodiment 2 of this application;

[0022] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] It should be noted that the data detection methods, devices, and electronic equipment specified in this disclosure can be used in the fintech field, or in any field other than fintech. The application fields of the data detection methods, devices, and electronic equipment specified in this disclosure are not limited.

[0027] It should be noted that all information, user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) used in this application are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse use. If the user chooses to refuse, the process will proceed to the expert decision-making process. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface. After receiving consent from the aforementioned user or organization, the relevant information is obtained. Users can view the purpose of data use in real time through the authorization interface and have the right to withdraw authorization or delete data at any time. After the authorization is withdrawn, the system will terminate the relevant data processing within 24 hours.

[0028] The embodiments or examples disclosed herein are not exhaustive, but merely illustrative of some embodiments or examples, and are not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment or example can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment or example can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment or example can be arbitrarily interchanged. Furthermore, optional methods or examples in a particular embodiment or example can be arbitrarily combined; moreover, embodiments or examples can be arbitrarily combined. For example, some or all steps of different embodiments or examples can be arbitrarily combined, and a particular embodiment or example can be arbitrarily combined with optional methods or examples of other embodiments or examples.

[0029] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0030] Network assets refer to all network-related information technology resources in a financial institution, including but not limited to physical network equipment (such as routers, switches, and firewalls), software network components (such as network management software and network monitoring systems), virtual network resources (such as virtual private clouds and virtual network interfaces), as well as network-related services and documents.

[0031] Configuration Management Database: A key component in the IT service management framework, used to store and manage detailed information about all configuration items and their interrelationships in an enterprise's IT infrastructure.

[0032] Configuration item: A configuration item is the basic unit in the configuration management database.

[0033] Example 1

[0034] According to an embodiment of this application, an embodiment of a data detection method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing a data detection method is shown. Figure 1As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, processing devices such as microprocessors or programmable logic devices), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface, a universal serial bus port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0036] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0037] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned data detection method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0039] The display may be, for example, a touchscreen LCD display that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0040] Under the aforementioned operating environment, this application provides the following: Figure 2 The data detection method shown. Figure 2 This is a flowchart of the data detection method provided in Embodiment 1 of this application, as follows: Figure 2 As shown, the method includes:

[0041] Step S201: Determine the network asset to be detected from the database and obtain the configuration information of the network asset to be detected. The database is used to store the initial configuration information of the network asset.

[0042] It should be noted that the execution entity in this embodiment can be a data detection system, which can be connected to a database to detect the configuration data in the configuration information of various network assets in the database, thereby improving the accuracy of the configuration data.

[0043] It should be noted that the network assets to be tested refer to network components or devices in the IT environment of financial institutions that require data quality testing, including but not limited to computer servers, network devices, and virtualization resources. Configuration information refers to the configuration data related to the network assets, covering detailed attributes such as hardware parameters, software versions, network addresses, and service status.

[0044] For example, when inspecting data in the database, the system needs to determine the list of network assets to be inspected from the database. Specifically, the system can locate and extract the set of network assets to be inspected by executing a query statement based on attributes such as the asset's name, type, status, and department. Subsequently, the system retrieves the configuration information of these assets from the database, ensuring that the extracted data comprehensively reflects the configuration status of the network assets. It should be noted that the system needs to synchronize and cache the configuration information of the network assets in real time to improve the timeliness and accuracy of the data during inspection and reduce false alarms or missed detections caused by data delays.

[0045] For example, if it is necessary to focus on checking network asset instances that have changed frequently recently, the system will locate these instances and retrieve all their configuration attribute data, including but not limited to asset ID (Identifier), asset name, IP address, port number, person in charge, change history, etc.

[0046] Step S202: Obtain the data detection rules associated with the network asset to be detected from the rule base, and determine the detection mode for detecting the network asset to be detected.

[0047] It should be noted that the rule base stores data inspection rules, which can be pre-configured by system administrators or data management personnel to guide and standardize data quality inspection activities. Data inspection rules are used to evaluate the data quality standards of the network asset configuration information to be inspected, including dimensions such as format validation, integrity checks, and consistency verification. Inspection modes can characterize the precision mode of data inspection, such as exact match, fuzzy match, and range match, thereby allowing different precision inspection methods to be used according to inspection requirements, thus improving the efficiency of data inspection.

[0048] For example, after obtaining the configuration information, the system also needs to retrieve data detection rules applicable to the network asset to be detected from the rule base and determine the detection mode for the network asset to be detected. The rules stored in the rule base have been reviewed and approved by the rule approval module, and these rules contain the inspection standards for the configuration information. After obtaining the rules, it is also necessary to determine the detection mode for the network asset to be detected, such as exact match, fuzzy match, and range match, so as to select different specific detection rules from the data detection rules according to different detection models, thereby improving the detection efficiency of the data.

[0049] For example, if a server's configuration information requires format and integrity checks, the system will select relevant rules from the rule base, such as IP addresses must conform to IPv4 or IPv6 format, and the responsible person field cannot be empty. Then, based on the characteristics of the selected rules, the system automatically determines the detection mode to use. For example, for IP address format verification, an exact match detection method can be used to check the IP address; for the responsible person field, a fuzzy match detection method can be used, only checking if it is empty.

[0050] Step S203: Detect the configuration information according to the detection mode and data detection rules to obtain the detection results.

[0051] For example, after determining the detection mode and data detection rules, in-depth analysis of the configuration information can be performed according to the selected detection mode and data detection rules. Regular expressions, enumeration lists, numerical ranges, and other rules can be used to compare and verify the extracted configuration information one by one. During this process, the system not only checks whether individual configuration attributes conform to the rules, but also evaluates whether the logical relationships between different attributes are consistent, improving the comprehensiveness and depth of the detection.

[0052] It should be noted that the detection results include both qualified and unqualified data. For unqualified data, the system must also automatically label the reasons for the violation, such as missing fields, incorrect format, or data conflicts, and record the specific instance and its associated configuration attribute. Through this step, the system achieves automated detection and evaluation of network asset data quality, which helps to quickly identify data deviations, reduces the burden of manual inspection, and improves the efficiency of data quality control.

[0053] For example, for the server's IP address attribute, the system uses regular expressions from the rule base for precise matching to ensure it conforms to the IPv4 or IPv6 format specifications. If configuration information that does not conform to the rules is found during the detection, the system will immediately mark it as ineligible data and record the specific attributes of the violation, the reason for the violation, and detailed information about the configuration item.

[0054] The data detection method provided in this application embodiment, by identifying the network asset to be detected from a database and obtaining its configuration information (where the database stores the initial configuration information of the network asset), obtaining data detection rules associated with the network asset to be detected from a rule base and determining a detection mode for detecting the network asset to be detected, and detecting the configuration information according to the detection mode and data detection rules to obtain the detection result, achieves the goal of improving the accuracy of the network asset to be detected by detecting its configuration information. This improves the accuracy of the configuration information of each network asset in the database and solves the technical problem of low accuracy of network asset configuration information in the database in related technologies.

[0055] To determine whether a network to be detected exists in the database and to accurately obtain the network to be detected, optionally, in the data detection method provided in this application embodiment, the configuration information of each initial network asset includes multiple configuration items and configuration data of each configuration item. Determining the network asset to be detected from the database includes: determining the data detection period of each initial network asset stored in the database; determining the initial network asset that needs to perform data detection operation according to the data detection period to obtain the network asset to be detected; or, if a rule change operation is detected in the rule base, obtaining the target rule associated with the rule change operation and obtaining the description information of the target rule; determining the configuration item associated with the target rule from the database according to the description information, and determining the target network asset that has an association with the configuration item as the network asset to be detected, wherein the target network asset is the initial network asset to which the configuration information of the configuration item belongs.

[0056] It's important to note that initial network assets are the fundamental elements of an enterprise's IT architecture, such as servers, network devices, and applications. Configuration information is a data set describing the status and characteristics of network assets, including attributes such as hardware configuration, software version, network address, and service status. Configuration items are the basic units constituting network asset configuration information and can be key information points such as hardware parameters, software settings, service attributes, and affiliated departments. Configuration data is the specific content of the configuration item, such as the server's processor model, software version number, and service IP address. The data detection cycle is the frequency setting for the system to perform periodic data quality checks on network assets, such as daily, weekly, or monthly. Rule change operations refer to the modification of data detection rules in the rule base, including adding, deleting, or updating rules. The target rule is the specific rule in the rule base that is modified or added. Description information is the specific information related to the target rule included in the rule change operation, such as rule name, rule type, and rule expression.

[0057] For example, when determining which assets are network assets to be tested, the data testing cycle of each initial network asset stored in the database can be determined, and the assets that need to be tested at the current time point can be automatically identified according to the preset cycle parameters, thereby obtaining a list of network assets to be tested. This allows data quality testing to be performed regularly and systematically, avoiding over-testing or under-testing, and improving the rationality of testing resource allocation.

[0058] For example, if a server is set to undergo data quality testing once a month, the system will automatically identify the server as a network asset to be tested at the beginning of each month, triggering the subsequent data testing process.

[0059] For example, when determining which assets are to be detected as network assets, this can also be done by identifying rule change operations in the rule base. When a rule change operation is detected in the rule base, the system automatically obtains the changed target rule and its description information. Then, based on the scope of application of the target rule, it locates all relevant configuration items in the database. Next, the system further identifies the network assets to which each configuration item belongs, determines these assets as network assets to be detected, and thus updates the list of network assets to be detected in a targeted manner. This improves the synchronization and consistency between data detection rules and the actual detection process, and strengthens the timeliness of data quality control.

[0060] For example, if the rule base adds a new rule for detecting the format of server IP addresses, the system will immediately update the detection strategy and perform data quality checks on all servers configured with IP address information to verify whether they meet the requirements of the new rule.

[0061] This embodiment determines the network assets to be detected based on the data detection cycle and rule changes in the rule base, thereby improving the accuracy and timeliness of data detection.

[0062] To accurately obtain configuration items, optionally, in the data detection method provided in this application embodiment, determining the configuration item associated with the target rule from the database based on the description information includes: parsing the description information to obtain multiple keywords; matching each keyword with the configuration items in each configuration information stored in the database, and determining the configuration item that matches any keyword as the configuration item associated with the target rule.

[0063] For example, when retrieving configuration items, to improve the comprehensiveness and completeness of the retrieved items, the description information first needs to be deeply analyzed to extract multiple keywords. These keywords are directly related to the attributes and execution conditions of the data detection rules. For instance, if the description information contains "IP address format must be IPv4," then "IP address" will be identified as a keyword. Next, the system matches these keywords with configuration items in the database to locate configuration items directly related to the keywords. During this process, the system may employ techniques such as full-text search, regular expression matching, or attribute value comparison to improve the accuracy and comprehensiveness of the matching.

[0064] Furthermore, after identifying the relevant configuration items, it is necessary to obtain the network assets containing each identified configuration item, and then identify the network assets to be tested that need to be subjected to data quality testing. The execution of the above process needs to be based on the logical relationship between the configuration items and the network assets, so that the testing activities cover all network assets related to the identified configuration items.

[0065] For example, if the system locates the server's "IP address" configuration item through keyword matching, it will further determine the server instance to which the "IP address" attribute belongs, define these server instances as network assets to be detected, and thus complete the confirmation process of the network assets to be detected.

[0066] This embodiment achieves accurate determination of configuration items related to the description information by parsing the description information, thereby improving the accuracy and comprehensiveness of determining the network resources to be detected based on the configuration items.

[0067] To accurately determine the detection mode, optionally, in the data detection method provided in this application embodiment, determining the detection mode for detecting the network asset to be detected includes: obtaining historical detection records of the network asset to be detected, and determining the time difference between the last time the network asset to be detected was detected using the first detection mode and the current time based on the historical detection records; if the time difference is greater than a time difference threshold, determining the detection mode as the first detection mode; if the time difference is less than or equal to the time difference threshold, determining whether the data detection rules have changed; if the data detection rules have changed, determining the detection mode as the first detection mode; if the data detection rules have not changed, determining the detection mode as the second detection mode, wherein the detection accuracy of the first detection mode is greater than that of the second detection mode.

[0068] For example, when determining the detection mode for a network asset to be detected, the system first needs to extract historical detection records of the network asset from the database. These records contain key information such as the timestamps of past data quality checks, the detection mode used, and the detection results. Subsequently, the system calculates the time difference between the last time the network asset was detected using the first detection mode and the current time. Assuming a time difference threshold of 30 days, if the last detection using the first detection mode occurred 40 days ago, the system will determine that the time difference is 40 days, which is greater than the threshold, thus indicating that the detection mode for the network asset to be detected is the first detection mode.

[0069] For example, when the time difference is less than or equal to the time difference threshold, the system will check whether the data detection rules stored in the rule base have changed. If there are new or modified rules in the rule base, it will directly affect the scope and standards of data detection. Therefore, if a rule change is detected, the system will determine the detection mode as the first detection mode regardless of the time difference, so that the new rules can be fully and accurately applied to data quality detection.

[0070] For example, when the data detection rules remain unchanged and the time difference is less than or equal to the time difference threshold, the detection mode will be determined as the second detection mode. Compared to the first detection mode, the second detection mode employs a more efficient detection algorithm and a simpler detection process, aiming to perform rapid routine checks on network assets. It is suitable for situations where the data detection rules are stable and the network asset status does not change significantly. When the rules remain unchanged and the network asset status is relatively stable, the system can adopt a simpler and faster detection process, reducing the consumption of computing resources, while still being able to perform basic data quality monitoring, thus improving the flexibility and execution efficiency of data quality control.

[0071] For example, taking server IP address detection as an example, if the detection rules for server IP addresses have not changed recently, and the last accurate detection occurred 20 days ago (assuming the time difference threshold is 30 days), the system will use the second detection mode for quick routine detection to avoid wasting resources; conversely, if the rules change or the time difference exceeds the threshold, the system will use the first detection mode for deep and accurate detection, so that the new rules can be applied immediately and comprehensive monitoring of data quality can be achieved.

[0072] This embodiment improves the accuracy and efficiency of data quality detection by combining historical detection records of network assets with adjustments to the rule base and implementing an intelligent decision-making detection mode.

[0073] To improve the accuracy of the detection results, optionally, in the data detection method provided in this application embodiment, detecting the configuration information according to the detection mode and data detection rules to obtain the detection results includes: adjusting the data detection rules according to the detection mode to obtain updated data detection rules; obtaining sub-detection rules corresponding to each configuration item in the configuration information from the updated data detection rules to obtain M sub-detection rules, where M is a positive integer; using each sub-detection rule to detect the configuration data of the configuration item corresponding to the sub-detection rule to obtain M sub-detection results; and combining the M sub-detection results into a detection result.

[0074] It should be noted that the detection mode refers to the specific strategy or process adopted by the system when performing data quality detection. Depending on different business needs and network asset characteristics, the detection mode can be further subdivided into precise mode, fast mode, etc. Data detection rules are the standards used to evaluate the quality of configuration information data, including but not limited to constraints on completeness, accuracy, and consistency. Sub-detection rules are some of the rules within the data detection rules, setting specific data quality detection standards for each configuration item in the configuration information.

[0075] For example, when detecting configuration information, the data detection rules are first adjusted according to the selected detection mode to adapt to the detection requirements of different modes. For instance, when starting the first detection mode, the system will automatically load detection rules containing more details and stricter conditions, such as rule type "multi-attribute verification"; while when using the second detection mode, the general rule type "single-attribute verification" is used first to simplify the detection process and improve detection efficiency.

[0076] Furthermore, after obtaining the updated data detection rules, it is necessary to obtain the sub-detection rules for each configuration item from the updated data detection rules, thereby obtaining M sub-detection rules. Then, each sub-detection rule is used to perform data quality detection on its corresponding configuration item, resulting in M ​​sub-detection results.

[0077] Finally, the system summarizes and analyzes the M sub-detection results to comprehensively evaluate the data quality of the entire network asset configuration information and generate the final detection result. This includes calculating the ratio of qualified to unqualified configuration items, analyzing the type distribution and responsibility attribution of unqualified configuration items, providing a basis for the generation and assignment of data governance tasks, and thus obtaining a more comprehensive data quality detection result.

[0078] This embodiment improves the accuracy of detection results obtained based on sub-detection rules by updating the data detection rules and determining sub-detection rules for each configuration item.

[0079] To improve the accuracy of the detection rules, optionally, in the data detection method provided in this application embodiment, adjusting the data detection rules according to the detection mode to obtain the updated data detection rules includes: keeping the data detection rules unchanged when the detection mode is a first detection mode; and obtaining the fuzzy matching rule for each detection rule in the data detection rules when the detection mode is a second detection mode, and using the fuzzy matching rule as the updated data detection rule, wherein the detection accuracy of the first detection mode is greater than that of the second detection mode.

[0080] For example, when the detection mode is the first detection mode, the system will maintain the existing data detection rules unchanged and directly use the original rules stored in the rule base to perform data quality detection. That is, in the first detection mode, the system adopts more stringent data quality standards, including field integrity, format standardization, numerical range restrictions, etc., so that each configuration information can meet the data quality level required for business operation.

[0081] For example, when the detection mode is the second detection mode, the system will adjust the execution logic of each detection rule in the data detection rules, converting it into fuzzy matching rules. Specifically, the system replaces the original exact matching rules with fuzzy rules that allow for a certain degree of deviation. For instance, fields that originally required a perfect match are now allowed to contain certain keywords, or the restriction on the numerical range is relaxed to any value within a certain range. The updated data detection rules will prioritize detection efficiency, identifying the main deviations in the configuration information through rapid screening, thereby improving the efficiency of data quality detection.

[0082] This embodiment improves the flexibility and efficiency of data quality detection operations by adjusting the data detection rules according to the detection mode.

[0083] To improve the accuracy of detection rules, optionally, the data detection method provided in this application embodiment further includes: receiving a rule change instruction when the rule base detects a rule change instruction, and determining a rule verification process based on the instruction content of the rule change instruction; verifying the rule change instruction according to the rule verification process, obtaining a verification result, and performing a rule change operation on the data in the database according to the rule change instruction if the verification result indicates that the rule change instruction is not abnormal.

[0084] It should be noted that rule change instructions are used to instruct the system to modify, add, or delete one or more data quality rules in the rule base. The instruction content details the specific changes included in the rule change instruction, such as the rule name, rule type, rule description, and rule expression. The rule verification process is a pre-defined mechanism used to verify the rationality, compliance, and security of the rule change instructions, ensuring that the changed rules effectively improve data quality without introducing new problems or risks. The verification result is the conclusion reached by the system after evaluating the rule change instruction through the rule verification process, including whether it is abnormal and whether it is feasible.

[0085] For example, when the rule base detects a rule change instruction, it needs to immediately receive and parse the content of the instruction to understand the nature and intent of the change. The system will determine the corresponding rule verification process based on the parsing results of the instruction. For instance, if the instruction requests the addition of a new rule for detecting IP address formats, the system will invoke the verification process associated with IP address detection to verify the rationality of the IP address format rule and its impact on existing data. This ensures that the rule change instruction is effectively reviewed before it takes effect, improving the security of rule change operations.

[0086] For example, after determining the rule verification process, the rule change instructions need to be verified according to the process. First, the completeness of the instruction content is verified, ensuring all necessary information for the rule change is accurately provided. Second, the internal logic of the rule is verified, such as checking the correctness of the regular expression, the rationality of the rule parameters, and conflicts with existing rules. Third, data simulation testing is conducted, using sample data to test the effect of the new rule and assess its potential improvement in data quality and possible impact on business processes. Finally, a security review is performed to ensure that the rule change does not compromise the system's security and stability. This series of verification processes aims to improve the feasibility and security of change instructions, avoiding data errors or other system problems caused by rule changes.

[0087] After verification shows that the rule change instruction is free of any abnormalities or unreasonableness, the system will execute the rule change operation on the database according to the instruction content. Specifically, the system will update the rule information related to the instruction content in the rule base, including rule name, description, type, content, etc. At the same time, the system will record detailed information about this rule change, including the change time, the person making the change, and a comparison of the rules before and after the change, to facilitate subsequent auditing and traceability.

[0088] For example, the rule base receives a rule change instruction requesting the addition of a rule to detect the size of server log files, ensuring that log data does not exceed storage capacity limits. Upon receiving the instruction, the system first establishes a verification process covering log file size detection logic, storage space assessment algorithms, and compatibility with existing server monitoring rules. After multi-level verification, the system confirms that the new rule has clear logic, reasonable parameters, and no conflicts with existing rules. Simulation tests demonstrate that it effectively prevents log data overflow. Finally, the system executes the rule change operation, adding the new rule to the rule base.

[0089] This embodiment improves the security and accuracy of changing rules in the rule base by parsing the rule change instructions and reviewing their content.

[0090] Figure 3 This is a schematic diagram of the data detection system provided in Embodiment 1 of this application, as shown below. Figure 3 As shown, the system includes: a rule configuration module 31, a rule approval module 32, a data quality detection module 33, a data quality result display module 34, a rule base 35, and a configuration management database 36, wherein:

[0091] The rule configuration module 31 is used to manage the rules in the rule base 35. The rule configuration module 31 can perform the following operations:

[0092] Adding Rules: The system provides a rule addition interface, allowing users to input key information such as rule name, rule description, rule type (e.g., completeness, accuracy, consistency), and rule expression. Newly added rules will be securely saved in rule base 35 for later use. When the rule type is selected as "Single Attribute Validation," the rule content includes: regular expression, only numbers, only Chinese characters, only letters, length, IPv4 address combinations separated by a certain symbol, IPv6 address combinations separated by a certain symbol, IPv4 and IPv6 address combinations separated by a certain symbol, port number, enumeration, numerical range, starting with a specific character (string), and ending with a specific character (string). When the rule type is selected as "Multi-Attribute Validation," the rule content can include: Aa in Bb, when A.a1=B.b1, A.a2=B.b2. When the rule type is selected as "Script Validation," the rule content is an operation code.

[0093] Rule deletion: Users can easily select the rule to be deleted through the rule management interface. The system will pop up a confirmation dialog box for the deletion operation. Once confirmed, the selected rule will be completely deleted from rule library 35.

[0094] Rule Editing: In the rule management interface, users can easily select the rule to be modified and enter the editing interface. The editing interface provides comprehensive modification functions, including modifying information such as rule name, rule description, rule type, and rule expression. After modification, the updated rule will be saved to rule base 35.

[0095] Rule search: Rules in the rule base 35 can be searched through the rule configuration module 31. Users can search by rule name, rule type, rule description and other conditions. The system will quickly return a list of rules that meet the conditions, making it convenient for users to find and use them.

[0096] Rule details: Users can view detailed information about each rule in the rule base 35 through the rule configuration module 31. This information includes the rule name, rule description, rule type, rule expression, creation time, and modification time, allowing users to fully understand the specific content and historical changes of the rules.

[0097] Rule List View: The rule list is displayed in a clear table format, including key fields such as rule name, rule type, rule description, creation time, and modification time. In addition, the system supports pagination, with each page displaying a certain number of rules, and users can easily navigate between pages using the pagination buttons.

[0098] Rule sorting function: To meet different user needs, the system provides a flexible rule sorting function. Users can sort by fields such as rule name, creation time, or modification time to find the rules they need more quickly.

[0099] The rule approval module 32 is used to approve the data detection rules entered by the administrator, and if the approval is successful, the data detection rules are stored in the rule base 35.

[0100] The data quality inspection module 33 retrieves rules from the rule base 35 and uses these rules to inspect the data in the configuration management database 36 to determine whether the data conforms to the rules. In data inspection, the system needs to match the collected data with the rules. The matching process uses regular expressions for data content matching and supports multiple matching modes, including exact match, fuzzy match, and range match. The data inspection process includes three stages: data preprocessing, rule matching, and result judgment. The preprocessing stage standardizes the data to ensure consistent data format; the rule matching stage matches the data with the rules to determine whether the data meets preset conditions; and the result judgment stage determines whether the data is compliant based on the matching results. Regarding result output, the system needs to output detailed information about non-compliant data, including data ID, data content, and reasons for non-compliance. The output method uses an API interface and supports multiple formats to meet the needs of different scenarios.

[0101] The data quality result display module 34 acquires the detection results from the data quality detection module 33 and displays the detection results according to the preset display rules.

[0102] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0103] Example 2

[0104] This application also provides a data detection device. It should be noted that the data detection device of this application can be used to execute the data detection method provided in the above embodiments. The data detection device provided in this application is described below.

[0105] According to an embodiment of this application, an apparatus for implementing the above-described data detection method is also provided. Figure 4 This is a schematic diagram of the data detection device provided according to Embodiment 2 of this application, as shown below. Figure 4 As shown, the device includes:

[0106] The first acquisition unit 41 is used to determine the network asset to be detected from the database and acquire the configuration information of the network asset to be detected, wherein the database is used to store the initial configuration information of the network asset.

[0107] The second acquisition unit 42 is used to acquire data detection rules associated with the network asset to be detected from the rule base and determine the detection mode for detecting the network asset to be detected.

[0108] The detection unit 43 is used to detect the configuration information according to the detection mode and data detection rules, and obtain the detection result.

[0109] The data detection apparatus provided in this application embodiment determines the network asset to be detected from the database by a first acquisition unit 41 and acquires the configuration information of the network asset to be detected, wherein the database is used to store the initial configuration information of the network asset; a second acquisition unit 42 acquires the data detection rules associated with the network asset to be detected from the rule base and determines the detection mode for detecting the network asset to be detected; a detection unit 43 detects the configuration information according to the detection mode and the data detection rules to obtain the detection result. By determining the network asset to be detected in the database and determining the data detection rules for detecting the network asset to be detected, the configuration information of the network asset to be detected is detected through the data detection rules and detection mode, thereby achieving the purpose of improving the accuracy of the network asset to be detected by detecting the configuration information of the network asset to be detected, thus realizing the technical effect of improving the accuracy of the configuration information of each network asset in the database, and thus solving the technical problem of low accuracy of network asset configuration information in the database in related technologies.

[0110] Optionally, in the data detection device provided in this application embodiment, the configuration information of each initial network asset includes multiple configuration items and configuration data of each configuration item. The first acquisition unit 41 includes: a first determining module, used to determine the data detection period of each initial network asset stored in the database; a second determining module, used to determine the initial network asset that needs to perform data detection operation according to the data detection period, and obtain the network asset to be detected; or, the first acquisition module, used to acquire the target rule associated with the rule change operation when a rule change operation is detected in the rule base, and acquire the description information of the target rule; a third determining module, used to determine the configuration item associated with the target rule from the database according to the description information, and determine the target network asset that has an association with the configuration item as the network asset to be detected, wherein the target network asset is the initial network asset to which the configuration information of the configuration item belongs.

[0111] Optionally, in the data detection device provided in this application embodiment, the third determining module includes: a parsing submodule, used to parse the description information to obtain multiple keywords; and a matching submodule, used to match each keyword with the configuration items in each configuration information stored in the database, and determine the configuration item that matches any keyword as the configuration item associated with the target rule.

[0112] Optionally, in the data detection apparatus provided in this application embodiment, the second acquisition unit 42 includes: a second acquisition module, used to acquire historical detection records of the network asset to be detected, and determine the time difference between the time when the network asset to be detected was last detected using the first detection mode and the current time based on the historical detection records; a fourth determination module, used to determine the detection mode as the first detection mode when the time difference is greater than a time difference threshold; a judgment module, used to determine whether the data detection rules have changed when the time difference is less than or equal to the time difference threshold; a fifth determination module, used to determine the detection mode as the first detection mode when the data detection rules have changed; and a sixth determination module, used to determine the detection mode as the second detection mode when the data detection rules have not changed, wherein the detection accuracy of the first detection mode is greater than that of the second detection mode.

[0113] Optionally, in the data detection device provided in this application embodiment, the detection unit 43 includes: an adjustment module, used to adjust the data detection rules according to the detection mode to obtain updated data detection rules; a third acquisition module, used to acquire sub-detection rules corresponding to each configuration item in the configuration information from the updated data detection rules to obtain M sub-detection rules, where M is a positive integer; a detection module, used to use each sub-detection rule to detect the configuration data of the configuration item corresponding to the sub-detection rule to obtain M sub-detection results; and to combine the M sub-detection results into a detection result.

[0114] Optionally, in the data detection device provided in this application embodiment, the adjustment module includes: a holding submodule, used to keep the data detection rules unchanged when the detection mode is a first detection mode; and an acquisition submodule, used to acquire the fuzzy matching rule of each detection rule in the data detection rules when the detection mode is a second detection mode, and use the fuzzy matching rule as the updated data detection rule, wherein the detection accuracy of the first detection mode is greater than that of the second detection mode.

[0115] Optionally, in the data detection device provided in the embodiments of this application, the device further includes: a determining unit, configured to receive a rule change instruction when the rule base detects a rule change instruction, and determine a rule verification process according to the instruction content of the rule change instruction; and a verification unit, configured to verify the rule change instruction according to the rule verification process, obtain a verification result, and, if the verification result indicates that the rule change instruction is not abnormal, perform a rule change operation on the data in the database according to the rule change instruction.

[0116] It should be noted that the first acquisition unit 41, the second acquisition unit 42, and the detection unit 43 mentioned above correspond to steps S201 to S203 in Embodiment 1. The instances and application scenarios implemented by each of the above units and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules can also be part of a device and can run in the computer terminal 10 provided in Embodiment 1.

[0117] Example 3

[0118] Embodiments of this application may provide an electronic device. Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 (Only one is shown) processor 1002, memory 1004, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0119] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0120] Those skilled in the art will understand that Figure 5The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0121] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0122] Example 4

[0123] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the data detection method provided in Embodiment 1.

[0124] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0125] Embodiments of this application also provide a computer program product, which, when executed on a data processing device, is a program adapted to perform the steps of a data detection method.

[0126] Embodiments of this application also provide a computer-readable storage medium, which includes a stored executable program, wherein the executable program controls the device where the computer-readable storage medium is located to perform the above-described data detection method when it runs.

[0127] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0128] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0131] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0132] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0133] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data detection method, characterized in that, include: The database is used to identify the network assets to be detected and to obtain the configuration information of the network assets to be detected, wherein the database is used to store the configuration information of the initial network assets; Obtain data detection rules associated with the network asset to be detected from the rule base, and determine the detection mode for detecting the network asset to be detected; The configuration information is detected according to the detection mode and the data detection rules to obtain the detection results.

2. The method according to claim 1, characterized in that, The configuration information for each initial network asset includes multiple configuration items and configuration data for each item. The network assets to be detected, as determined from the database, include: Determine the data detection period for each initial network asset stored in the database; Based on the data detection cycle, the initial network assets requiring data detection operations are determined, thus obtaining the network assets to be detected; or, If a rule change operation is detected in the rule base, the target rule associated with the rule change operation is obtained, and the description information of the target rule is obtained. Based on the description information, the configuration item associated with the target rule is determined from the database, and the target network asset that is associated with the configuration item is determined as the network asset to be detected, wherein the target network asset is the initial network asset to which the configuration information of the configuration item belongs.

3. The method according to claim 2, characterized in that, The configuration items associated with the target rule determined from the database based on the description information include: The description information is parsed to obtain multiple keywords; Each keyword is matched with the configuration items in the configuration information stored in the database, and the configuration item that matches any keyword is determined as the configuration item associated with the target rule.

4. The method according to claim 1, characterized in that, The detection mode for detecting the network assets to be detected includes: Obtain the historical detection records of the network asset to be detected, and determine the time difference between the last time the network asset to be detected was detected using the first detection mode and the current time based on the historical detection records; If the time difference is greater than the time difference threshold, the detection mode is determined to be the first detection mode; If the time difference is less than or equal to the time difference threshold, determine whether the data detection rule has changed; If the data detection rules change, the detection mode is determined to be the first detection mode; If the data detection rules remain unchanged, the detection mode is determined to be the second detection mode, wherein the detection accuracy of the first detection mode is greater than that of the second detection mode.

5. The method according to claim 1, characterized in that, The configuration information is detected according to the detection mode and the data detection rules, and the detection results include: The data detection rules are adjusted according to the detection mode to obtain updated data detection rules; From the updated data detection rules, obtain the sub-detection rules corresponding to each configuration item in the configuration information to obtain M sub-detection rules, where M is a positive integer; The configuration data of the configuration item corresponding to each sub-detection rule is detected using each sub-detection rule to obtain M sub-detection results; The M sub-detection results are combined into the detection result.

6. The method according to claim 5, characterized in that, The data detection rules are adjusted according to the detection mode to obtain the updated data detection rules, which include: When the detection mode is the first detection mode, the data detection rules remain unchanged; When the detection mode is the second detection mode, the fuzzy matching rule of each detection rule in the data detection rules is obtained, and the fuzzy matching rule is used as the updated data detection rule, wherein the detection accuracy of the first detection mode is greater than that of the second detection mode.

7. The method according to claim 2, characterized in that, The method further includes: If the rule base detects a rule change instruction, the rule change instruction is received, and the rule verification process is determined based on the content of the rule change instruction. The rule change instruction is verified according to the rule verification process to obtain a verification result. If the verification result indicates that there is no abnormality in the rule change instruction, the rule change operation is performed on the data in the database according to the rule change instruction.

8. A data detection device, characterized in that, include: The first acquisition unit is used to determine the network asset to be detected from the database and acquire the configuration information of the network asset to be detected, wherein the database is used to store the configuration information of the initial network asset; The second acquisition unit is used to acquire data detection rules associated with the network asset to be detected from the rule base, and to determine the detection mode for detecting the network asset to be detected. The detection unit is used to detect the configuration information according to the detection mode and the data detection rules, and obtain the detection result.

9. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the data detection method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program executes the data detection method according to any one of claims 1 to 7 when it runs.

Images