Checking process determination method and device, equipment and storage medium
By acquiring production problem information and automatically generating investigation processes using knowledge graphs and deep learning models, the problem of time-consuming and labor-intensive investigation of distributed database issues has been solved, and efficient and accurate investigation process determination has been achieved.
Patent Information
- Application Number
- CN202511032407.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-25
- Publication Date
- 2025-11-11
AI Technical Summary
In existing technologies, when problems occur in distributed databases, operations and maintenance personnel need to manually review a large number of log files and monitoring data, relying on experience to troubleshoot, which is time-consuming and easily affected by human factors, making it difficult to quickly and accurately determine the troubleshooting process.
By acquiring production problem information, matching business system information with knowledge graphs, and inputting investigation processes to determine the model, the investigation process is automatically generated. Deep learning or neural network models are used to train the model to improve the efficiency and accuracy of the investigation process.
It enables the rapid and accurate automatic generation of troubleshooting processes, reducing manual analysis time, improving the efficiency and accuracy of problem investigation, and minimizing the impact of human factors.
Smart Images
Figure CN120930997A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, specifically to a method, device, and storage medium for determining a screening process. Background Technology
[0002] Distributed databases, with their advantages of large capacity, easy scalability, and high performance, have greatly facilitated financial services. However, due to the complexity and distributed nature of the system, when production problems arise, such as slow system response, transaction failures, or data inconsistencies, operations and maintenance personnel often need to manually review a large number of log files and monitoring data, relying on their own experience and knowledge to analyze and troubleshoot the problem step by step. This approach is not only time-consuming and labor-intensive but also easily affected by human factors, leading to the omission of key factors and delays in resolving the problem. In other words, existing troubleshooting processes are mostly determined by operations and maintenance personnel based on experience. As the scale of distributed core systems continues to expand and business complexity increases, traditional methods of determining troubleshooting processes are becoming increasingly inadequate to meet the growing demand for problem investigation.
[0003] Currently, there is still a lack of an effective method to quickly and accurately determine the troubleshooting process and help maintenance personnel solve production problems in a timely manner. Summary of the Invention
[0004] In view of this, the purpose of this application is to provide a method, apparatus, equipment and storage medium for determining a troubleshooting process, which can automatically generate troubleshooting process suggestions, improve the efficiency and accuracy of determining the troubleshooting process, reduce the time for manual analysis and judgment, and improve the efficiency and accuracy of problem troubleshooting.
[0005] Firstly, this application provides a method for determining the screening process, including:
[0006] Obtain information on production issues;
[0007] Based on the matching of the production problem information and the knowledge graph, business system information is determined. The business system information includes business process information and system architecture information. The knowledge graph includes the correspondence between the production problem information and the business system information.
[0008] The business system information is input into the investigation process determination model to obtain the target investigation process output by the investigation process determination model.
[0009] Optionally, the investigation process determination model is trained using the following method:
[0010] Acquire training data, which includes historical business system information and historical investigation processes corresponding to the historical business system information; the historical business system information includes historical business process information and historical system architecture information.
[0011] The training data is input into the model to be trained to obtain the first screening process output by the model to be trained.
[0012] The model to be trained is trained using the first investigation process and the historical investigation process until the preset conditions are met, thus obtaining the investigation process determination model.
[0013] Optionally, the method further includes:
[0014] The training data is preprocessed to obtain preprocessed data; the preprocessing is used to filter data in the training data that does not conform to preset filtering rules.
[0015] The step of inputting the training data into the model to be trained includes: inputting preprocessed data into the model to be trained.
[0016] Optionally, there may be multiple target screening processes, which are sorted in order of priority.
[0017] Optionally, the method further includes:
[0018] Obtain feedback information, which is used to instruct the correction of the target investigation process;
[0019] The investigation process determination model is updated based on the feedback information.
[0020] Optionally, the method further includes:
[0021] Obtain updated information, which is used to indicate the need to update the correspondence between the production problem information and the business system information;
[0022] The knowledge graph is updated based on the updated information.
[0023] Optionally, the method further includes:
[0024] The target screening process is illustrated using a flowchart.
[0025] Secondly, this application provides a screening process determination device, the device comprising:
[0026] The acquisition unit is used to acquire production problem information;
[0027] The determining unit is used to determine business system information by matching the production problem information with the knowledge graph. The business system information includes business process information and system architecture information. The knowledge graph includes the correspondence between the production problem information and the business system information.
[0028] The determining unit is further configured to input the business system information into the investigation process determining model to obtain the target investigation process output by the investigation process determining model.
[0029] Thirdly, this application provides a device for determining the screening process, comprising:
[0030] Memory, used to store computer programs;
[0031] A processor for executing a computer program stored in the memory to implement the steps of the investigation process determination method as described in the first aspect.
[0032] Fourthly, this application provides a computer-readable storage medium storing a computer program that is executed by a processor to implement the steps of the investigation process determination method as described in the first aspect.
[0033] As can be seen, the present application discloses a method, apparatus, device, and storage medium for determining a troubleshooting process. In this method, production problem information is acquired; business system information is determined by matching the production problem information with a knowledge graph. The business system information includes business process information and system architecture information; the knowledge graph includes the correspondence between the production problem information and the business system information; the business system information is input into a troubleshooting process determination model to obtain the target troubleshooting process output by the model. This method can automatically generate troubleshooting processes, improving the efficiency and accuracy of determining the troubleshooting process, reducing manual analysis and judgment time, and improving the efficiency and accuracy of problem troubleshooting. Attached Figure Description
[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.
[0035] Figure 1 This is a flowchart illustrating a method for determining a screening process provided in an embodiment of this application;
[0036] Figure 2 This is a flowchart illustrating another method for determining the screening process provided in an embodiment of this application;
[0037] Figure 3 This is a schematic diagram of a screening process determination device provided in an embodiment of this application. Detailed Implementation
[0038] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0039] To facilitate understanding of the technical solution provided in this application, the following description, in conjunction with the accompanying drawings, will explain a method for determining the screening process provided in this application. See also... Figure 1 The figure is a flowchart illustrating a method for determining a screening process according to an embodiment of this application. Figure 1 As shown, the method includes S101-S103.
[0040] S101: Obtain production problem information.
[0041] It is understandable that production problem information can be information about problems that arise during the use of distributed databases, such as slow queries, abnormal cloud platform connection counts, database backup failures, database standby failures, and abnormal database master-slave failover.
[0042] In this application, the specific method of obtaining production problem information is not limited. As one example, this application can obtain production problem information by obtaining problems manually entered by operations and maintenance personnel. As another example, this application can also obtain production problem information by automatically monitoring a distributed database.
[0043] S102: Based on the production problem information and the knowledge graph, the business system information is determined.
[0044] It is understood that a knowledge graph can be pre-established in this embodiment. After obtaining production problem information, the production problem information can be matched with the knowledge graph to determine the business system information. It is understood that business system information includes business process information and system architecture information; the knowledge graph includes the correspondence between production problem information and business system information. Business system information refers to the digital representation describing business objectives, processes, rules, and resource interactions, such as business process inputs and outputs and processing logic. System architecture information refers to the technical implementation scheme supporting business operation, such as the system's component composition, dependencies between components, and network topology.
[0045] This application does not limit the specific method of building a knowledge graph. As an example, this application allows operations and maintenance personnel to pre-obtain business system information and build a knowledge graph based on practical experience. It is understood that the data center operations and maintenance platform has already integrated the business system information of the distributed core system, and operations and maintenance personnel can directly obtain business system information by calling interfaces to quickly build a knowledge graph.
[0046] As one possible implementation, this application can extract keywords from production problem information using a keyword extraction algorithm, and then match the keywords with an indicator graph to more quickly and accurately determine business system information.
[0047] S103: Input the business system information into the investigation process determination model to obtain the target investigation process output by the investigation process determination model.
[0048] In this embodiment of the application, after obtaining the business system information, it is input into the investigation process determination model to obtain the target investigation process output by the investigation process determination model.
[0049] This application does not limit the specific content of the target investigation process, which may include the investigation approach, methods, and tools. For example, it may include the tools used in each investigation step, the specific content, expected results, possible causes, and standard documentation. In this application, the investigation process model is determined to be a trained deep learning model or neural network model.
[0050] As one possible implementation method, there can be multiple target screening processes, which are ordered according to priority.
[0051] Understandably, the troubleshooting process determination model can output multiple target troubleshooting processes and the priority of each target process. This application can determine the priority based on the relevance to business system information; that is, this application actually prioritizes processes based on their relevance to production problem information. By prioritizing multiple possible target troubleshooting processes, the target troubleshooting processes needed by operations and maintenance personnel can be determined more accurately.
[0052] Using the above methods, when production problems are encountered, troubleshooting processes can be automatically generated, improving the efficiency and accuracy of determining the troubleshooting process, reducing the time spent on manual analysis and judgment, and improving the efficiency and accuracy of problem troubleshooting.
[0053] As one possible implementation, the investigation process determines that the model is trained in the following way, with specific training steps including A1-A3:
[0054] A1: Obtain training data, which includes historical business system information and historical investigation processes corresponding to the historical business system information; the historical business system information includes historical business process information and historical system architecture information.
[0055] In this embodiment, training data can be collected through a data acquisition system, and the method of acquiring training data is not limited herein. The training data in this application includes historical business system information and historical investigation processes corresponding to the historical business system information. It is understood that the historical investigation processes can serve as labels identifying the historical business system information during model training.
[0056] A2: Input the training data into the model to be trained to obtain the first screening process output by the model to be trained.
[0057] In this embodiment, the first screening process involves inputting training data into the model to be trained, thereby obtaining the output of the model to be trained. It is understood that the model to be trained can be a deep learning model or a neural network model.
[0058] A3: Use the first investigation process and the historical investigation process to train the model to be trained until the preset conditions are met, and obtain the investigation process determination model.
[0059] After obtaining the first screening process, the model to be trained is trained using both the first and historical screening processes until a preset condition is less than a threshold. It is understood that the model's network structure and parameters will be continuously adjusted during training. The preset condition may include the difference between the output of the first screening process and the historical screening processes being less than a threshold. When the preset condition is met, training ends, and the model's network structure and parameters are fixed. Based on the fixed network structure and parameters, the trained screening process and model can be determined. It is understood that the difference between the first and historical screening processes can be represented by evaluation metrics, such as anomaly similarity and anomaly component correlation. Anomaly similarity indicates the degree of similarity of the anomaly indicators to be screened in the two screening processes, and anomaly component correlation indicates the degree of correlation of the anomaly components to be screened in the two screening processes.
[0060] As one possible implementation, the screening process determination method provided in this application further includes the following steps: preprocessing the training data to obtain preprocessed data; the preprocessing is used to filter data in the training data that does not conform to preset filtering rules.
[0061] In this embodiment, after obtaining the training data, the training data is preprocessed to filter out data that does not conform to preset filtering rules. It is understood that the preset filtering rules can be determined according to actual needs; for example, noise, erroneous data, and redundant information unrelated to problem investigation in the training data can be identified as not conforming to the preset filtering rules.
[0062] Then A2 inputs the training data into the model to be trained, including: inputting preprocessed data into the model to be trained.
[0063] It is understood that this application preprocesses the training data and then inputs the preprocessed data into the model to be trained. This reduces the amount of data that needs to be processed, improves data processing efficiency, and further enhances the efficiency of the screening process in determining the model's training efficiency.
[0064] As one possible implementation, the method for determining the screening process provided in this application further includes the following steps B1-B2:
[0065] B1: Obtain feedback information, which is used to instruct on how to correct the target investigation process.
[0066] In this embodiment of the application, after the target troubleshooting process is determined, maintenance personnel can provide feedback on any problems encountered in the target troubleshooting process to generate feedback information. This feedback information can be used to instruct on how to correct the target troubleshooting process.
[0067] B2: Update the investigation process determination model based on the feedback information.
[0068] In this embodiment, upon receiving feedback information, the target investigation process can be updated to obtain an updated target investigation process. This updated process is then used as training data to further train the investigation process determination model, thereby updating the model. By providing timely feedback during use to update the investigation process determination model, the output accuracy of the model can be improved, further enhancing the accuracy of the investigation process determination.
[0069] As one possible implementation, the method for determining the screening process provided in this application further includes the following steps C1-C2:
[0070] C1: Obtain update information, which is used to indicate the correspondence between the production problem information and the business system information.
[0071] In this embodiment, as the system operates, new problems may arise, or changes may occur in the system architecture and business processes, leading to errors or mismatches in the information within the knowledge graph regarding production issues. This application can obtain updated information, which is used to indicate the updated correspondence between production issue information and business system information.
[0072] C2: Update the knowledge graph based on the updated information.
[0073] Understandably, updating information can include adding new mappings between production problem information and business system information, and / or modifying existing mappings between production problem information and business system information, and / or deleting existing mappings between production problem information and business system information. Timely updates to the knowledge graph ensure that target investigation processes can be automatically generated accurately based on production problem information, thereby improving the efficiency and accuracy of determining investigation processes.
[0074] As one possible implementation, the method for determining the investigation process provided in this application further includes: displaying the target investigation process through a flowchart.
[0075] Understandably, this application uses a visual flowchart to demonstrate the target investigation process, which can intuitively present the sequence of steps and logical relationships, avoid the redundancy of textual descriptions, and help users quickly grasp the steps of the investigation process.
[0076] See Figure 2 , Figure 2 This application provides a flowchart illustrating another troubleshooting process determination method. First, the data center operation and maintenance platform in this application has integrated the business system information of the distributed core system, allowing operation and maintenance personnel to directly obtain this information through interface calls. Furthermore, this application can write algorithms to remove noise and erroneous data, filter out redundant information irrelevant to problem troubleshooting, and preprocess the data to reduce data volume and improve processing efficiency. It also unifies the conversion and processing of data in different formats to achieve data preprocessing. Additionally, this application collects and organizes historical production problem troubleshooting processes. By organizing actual production problems, this application can obtain common historical troubleshooting processes. These include: slow database query troubleshooting process, abnormal cloud platform connection count troubleshooting process, database backup failure troubleshooting process, database standby machine inactivity troubleshooting process, abnormal database primary / standby switchover troubleshooting process, database SQL execution error code troubleshooting process, excessive database log disk usage troubleshooting process, excessive database log generation troubleshooting process, abnormal database and gateway restart troubleshooting process, management node liveness error troubleshooting process, and Filebeat messagememory error troubleshooting process.
[0077] Using the aforementioned data, this application can construct the knowledge graph described above and train a troubleshooting process determination model, which is a deep learning model or a neural network model. After constructing the knowledge graph and the troubleshooting process determination model, production problem information can be obtained by acquiring questions manually entered by operations and maintenance personnel, and / or by automatically monitoring a distributed database.
[0078] After acquiring production issue information, it can be matched with a knowledge graph. If a match is successful, the business system information can be identified. If the match fails, the process ends. After successfully identifying the business system information, the troubleshooting process model can be used for reasoning to obtain multiple target troubleshooting processes. These possible troubleshooting processes will be ranked and presented to operations personnel according to priority. The recommended target troubleshooting processes are presented to operations personnel through flowcharts, including detailed information for each step, expected results, possible causes, and relevant documentation.
[0079] Upon receiving the target troubleshooting process, operations and maintenance personnel can manually assess its rationality. If deemed unreasonable, they can provide feedback. This feedback serves as a guide for correcting the process. Furthermore, after correction, the updated process can be used as training data to further train the process determination model, thereby updating the model. This allows the model to adjust and optimize recommendation results after receiving feedback.
[0080] If the target troubleshooting process is determined to be reasonable, then the problem will be investigated according to the target troubleshooting process. Taking the database slow query troubleshooting process as an example, this application may include:
[0081] Step 1: Use database management software to check component versions, alarms, and performance metrics to determine if there are version bugs, related alarms, or abnormal trends in performance metrics.
[0082] Step 2: Use database management software to check slow queries, real-time sessions, and abnormal sessions to determine whether the slow query problem is affecting individual SQL queries or all SQL queries.
[0083] Step 3: Use database analysis software to check the response time and execution frequency trends of SQL queries to pinpoint the specific slow SQL queries.
[0084] Step 4: Check the operating system monitoring logs to determine if there are any abnormalities in machine performance or network.
[0085] Step 5: Check the MySQL.err, slow.log, and message logs of the database, and search for keywords such as error, fail, and known problem time periods to determine the abnormal information.
[0086] Step 6: Determine whether the investigated phenomenon is consistent with the known causes; otherwise, conduct further analysis.
[0087] Using the above methods, an intelligent recommendation device is built based on a deep learning model or a neural network model. By receiving business system information from the distributed core system and combining it with historical troubleshooting processes and professional knowledge for analysis and reasoning, it can automatically generate troubleshooting processes when encountering production problems, thereby improving the efficiency and accuracy of determining the troubleshooting process, reducing the time spent on manual analysis and judgment, and improving the efficiency and accuracy of problem troubleshooting.
[0088] The following describes a screening process determination device provided in an embodiment of this application. The device described below can be referred to in correspondence with the screening process determination method described above.
[0089] See Figure 3 , Figure 3 This is a schematic diagram of a screening process determination device provided in an embodiment of this application. The device includes an acquisition unit 301 and a determination unit 302.
[0090] Acquisition unit 301 is used to acquire production problem information;
[0091] The determining unit 302 is used to determine business system information based on the matching of the production problem information and the knowledge graph. The business system information includes business process information and system architecture information. The knowledge graph includes the correspondence between the production problem information and the business system information.
[0092] The determining unit 302 is further configured to input the business system information into the investigation process determining model to obtain the target investigation process output by the investigation process determining model.
[0093] As one possible implementation, the device further includes:
[0094] The acquisition unit 301 is further configured to acquire training data, which includes historical business system information and historical investigation processes corresponding to the historical business system information; the historical business system information includes historical business process information and historical system architecture information.
[0095] The input unit is used to input the training data into the model to be trained and obtain the first screening process output by the model to be trained.
[0096] The training unit is used to train the model to be trained using the first investigation process and the historical investigation process until a preset condition is met, thereby obtaining the investigation process determination model.
[0097] As one possible implementation, the device further includes:
[0098] A preprocessing unit is used to preprocess the training data to obtain preprocessed data; the preprocessing is used to filter data in the training data that does not conform to preset filtering rules;
[0099] The input unit is specifically used to input preprocessed data into the model to be trained.
[0100] As one possible implementation, there are multiple target screening processes, which are sorted according to priority.
[0101] As one possible implementation, the device further includes:
[0102] The acquisition unit 301 is also used to acquire feedback information, which is used to instruct the correction of the target investigation process;
[0103] An update unit is used to update the investigation process determination model based on the feedback information.
[0104] As one possible implementation, the device further includes:
[0105] The acquisition unit 301 is further configured to acquire update information, which is used to indicate the need to update the correspondence between the production problem information and the business system information;
[0106] The updating unit is also used to update the knowledge graph based on the update information.
[0107] As one possible implementation, the device further includes:
[0108] The display unit is used to show the target screening process through a flowchart.
[0109] It should be noted that the screening process determination device provided in this application has the technical effects of any of the above embodiments, and will not be described in detail here.
[0110] This application also provides a troubleshooting process determination device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the device may also include various network interfaces, power supplies, and other components.
[0111] It should be noted that the screening process determination device provided in this application has the technical effects of any of the above embodiments, and will not be described in detail here.
[0112] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, can perform the steps provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0113] It should be noted that the computer-readable storage medium provided in this application has the technical effects of any of the above embodiments, and the embodiments of this application will not be described in detail here.
[0114] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0115] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0116] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.
Claims
1. A method for determining an investigation process, characterized in that, include: Obtain information on production issues; Based on the matching of the production problem information and the knowledge graph, business system information is determined. The business system information includes business process information and system architecture information. The knowledge graph includes the correspondence between the production problem information and the business system information. The business system information is input into the investigation process determination model to obtain the target investigation process output by the investigation process determination model.
2. The method according to claim 1, characterized in that, The screening process determination model was trained using the following method: Acquire training data, which includes historical business system information and historical investigation processes corresponding to the historical business system information; the historical business system information includes historical business process information and historical system architecture information. The training data is input into the model to be trained to obtain the first screening process output by the model to be trained. The model to be trained is trained using the first investigation process and the historical investigation process until the preset conditions are met, thus obtaining the investigation process determination model.
3. The method according to claim 2, characterized in that, The method further includes: The training data is preprocessed to obtain preprocessed data; the preprocessing is used to filter data in the training data that does not conform to preset filtering rules. The step of inputting the training data into the model to be trained includes: inputting the preprocessed data into the model to be trained.
4. The method according to claim 1, characterized in that, The target screening process consists of multiple steps, which are ordered according to priority.
5. The method according to claim 1, characterized in that, The method further includes: Obtain feedback information, which is used to instruct the correction of the target investigation process; The investigation process determination model is updated based on the feedback information.
6. The method according to claim 1, characterized in that, The method further includes: Obtain updated information, which is used to indicate the need to update the correspondence between the production problem information and the business system information; The knowledge graph is updated based on the updated information.
7. The method according to claim 1, characterized in that, The method further includes: The target screening process is illustrated using a flowchart.
8. A device for determining a screening process, characterized in that, The device includes: The acquisition unit is used to acquire production problem information; The determining unit is used to determine business system information by matching the production problem information with the knowledge graph. The business system information includes business process information and system architecture information. The knowledge graph includes the correspondence between the production problem information and the business system information. The determining unit is further configured to input the business system information into the investigation process determining model to obtain the target investigation process output by the investigation process determining model.
9. A process for identifying equipment during screening, characterized in that, include: Memory, used to store computer programs; A processor for executing a computer program stored in the memory to implement the steps of the investigation process determination method as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by a processor to implement the steps of the investigation process determination method as described in any one of claims 1 to 7.