Application system fault detection method, device, computer equipment and storage medium

Abstract

The embodiments of the present application provide an application system fault detection method, apparatus, computer equipment and storage medium, the method comprising: receiving a target application system fault diagram and a target application system fault description, and inputting the target application system fault diagram into a pre-trained scenario recognition model to determine the target application system fault scenario; determining a query prompt word for the target application system fault diagram according to the target application system fault scenario; extracting the target application system fault diagram text according to the query prompt word; based on an intelligent agent and a large language model, decomposing the target application system fault text description and the target application system fault diagram text into multiple subtasks, determining the target application system fault factor according to the information returned by the subtasks, and being able to automatically discover the root cause of the fault based on the provided fault diagram and fault description information for faults that will not explicitly report errors, such as video failure to play, video distorted screen, missing content, and text content errors.

Description

Technical Field

[0001] The present application relates to the field of application system operation technology, and in particular, to an application system fault detection method, apparatus, computer equipment and storage medium. Background Art

[0002] With the development of the Internet, the Internet has become the main channel for people to obtain information. The continuous increase in application system functions and the increasing richness of content have improved the user experience, but have also brought many problems. First, the complexity of functions has increased the difficulty of application system design and development, resulting in an increase in the possibility of potential failures. Second, the rapid expansion of information has made capacity management more complex, further increasing the requirements for fault tracing and processing efficiency. Therefore, how to improve the fault tracing capability and processing efficiency has become a key challenge that needs to be solved urgently.

[0003] At present, application system fault detection mainly relies on performance monitoring, status monitoring, application programming interface monitoring, manual page inspection and log analysis to provide detailed error information to operation and maintenance personnel. These methods help to detect application system faults in a timely manner and improve their stability and availability.

[0004] However, the relevant technologies rely on monitoring tools for application systems. These tools can monitor the status of application systems in real time, discover the availability and performance of application systems, and discover failures such as slow page loading and server crashes. However, many tools do not cover all possible failure scenarios. Passive monitoring such as user feedback can help discover failures, but it relies on users' active reports, has a long response time, and it is difficult for users to clearly describe the failures themselves, making it impossible to find the causes of application system failures in a timely manner.

[0005] The relevant technologies still have certain limitations. Especially when dealing with complex and overlapping faults, it is often impossible to effectively identify the root cause of the fault and quickly find a solution to the fault. For example, since faults such as video playback failure, video distorted screen, missing content, and text content errors will not be explicitly reported, it is difficult for operation and maintenance personnel to identify the cause of the fault. These situations usually require communication between multiple departments and manual proofreading by the development team to confirm the fault, which significantly increases the cost from fault discovery to fault handling and prolongs the response and resolution time. Summary of the invention

[0006] The embodiments of the present application provide an application system fault detection method, apparatus, computer equipment and storage medium.

[0007] A first aspect of an embodiment of the present application provides an application system fault detection method, comprising:

[0008] receiving a target application system fault diagram and a target application system fault description, and inputting the target application system fault diagram into a pre-trained scenario recognition model to determine a target application system fault scenario;

[0009] Determine the query prompt words of the target application system fault diagram according to the target application system fault scenario;

[0010] Extract target application system fault diagram text according to query prompt words;

[0011] Based on intelligent agents and large language models, the target application system fault text description and target application system fault graph text are decomposed into multiple subtasks. According to the information returned by the subtasks, the target application system fault factors are determined.

[0012] In an optional embodiment of the present application, the scene recognition model is trained by the following steps:

[0013] Collect historical data of application system failure graphs under each application system failure scenario, wherein the application system failures include video playback errors, sequence errors, data missing, and content errors;

[0014] The application system failure diagram in each application system failure scenario is taken as input, and the application system failure scenario type is taken as output. The neural network model is trained to obtain a scenario recognition model.

[0015] In an optional embodiment of the present application, the step of determining a query prompt word of a target application system fault diagram according to a target application system fault scenario includes:

[0016] Determine the query prompt word corresponding to the target application system failure scenario based on the correspondence between the preset application system failure scenario and the query prompt word;

[0017] The query prompt word is used as the query prompt word of the target application system fault diagram.

[0018] In an optional embodiment of the present application, the step of extracting target application system fault graph text according to the query prompt word includes:

[0019] Inputting the query prompt words and the target application system fault diagram into the preset multi-modal large model;

[0020] The text extracted from the target application system fault diagram according to the query prompt words is output from the preset multimodal large model as the target application system fault diagram text.

[0021] In an optional embodiment of the present application, the target application system fault text description and the target application system fault graph text are decomposed into multiple subtasks based on the agent and the large language model, and the target application system fault factor is determined according to the information returned by the subtasks, including:

[0022] Inputting the target application system fault text description and the target application system fault graph text into the scheduling agent;

[0023] The scheduling agent determines the fault handling state according to the target application system fault text description and the target application system fault graph text, and sends the target application system fault text description and the target application system fault graph text to the task decomposition agent if the fault is not handled;

[0024] The task decomposition agent decomposes the fault into multiple subtasks according to the preset prompt word template based on the functions of the search engine, interface and database in the preset knowledge base, and sends the multiple subtasks to the task execution agent, wherein each subtask corresponds to at least one of the search engine, interface and database in the preset knowledge base;

[0025] The task execution agent generates executable programs for each subtask according to the prompt word template by using the preset large language model;

[0026] The task execution agent executes each executable program using at least one of the search engine, interface and database according to the corresponding relationship between each executable program and the search engine, interface and database, and uses the execution result of each executable program as the failure factor of the target application system.

[0027] In an optional embodiment of the present application, before taking the execution result of each executable program as a target application system failure factor, the method further includes:

[0028] Send the execution result of each executable program to the task decomposition agent;

[0029] The task decomposition agent determines whether the execution result of each executable program meets the preset conditions. If it does not meet the preset conditions, it adjusts the preset prompt word template, and re-decomposes the fault into multiple subtasks according to the adjusted prompt word template to re-execute multiple executable programs until the execution result of each executable program meets the preset conditions.

[0030] In an optional embodiment of the present application, the method further includes:

[0031] The execution result of each executable program is sent to the feedback agent, and the feedback agent sends the execution result of each executable program to the target object corresponding to each subtask.

[0032] A second aspect of an embodiment of the present application provides an application system fault detection device, including:

[0033] An input module is used to receive a target application system fault diagram and a target application system fault description, and input the target application system fault diagram into a pre-trained scenario recognition model to determine a target application system fault scenario;

[0034] A determination module, used to determine a query prompt word of a target application system fault diagram according to a target application system fault scenario;

[0035] An extraction module is used to extract the target application system fault diagram text according to the query prompt word;

[0036] The decomposition module is used to decompose the target application system fault text description and the target application system fault graph text into multiple subtasks based on the intelligent agent and the large language model, and determine the target application system fault factors according to the information returned by the subtasks.

[0037] According to a third aspect of an embodiment of the present application, a computer device is provided, including: a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of any of the above application system fault detection methods are implemented.

[0038] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above application system fault detection methods.

[0039] The above technical solution provided by the embodiment of the present application has at least some or all of the following advantages compared with the prior art:

[0040] The application system fault detection method described in the embodiment of the present application receives a target application system fault diagram and a target application system fault description, and inputs the target application system fault diagram into a pre-trained scenario recognition model to determine the target application system fault scenario; determines the query prompt word of the target application system fault diagram according to the target application system fault scenario; extracts the target application system fault diagram text according to the query prompt word; based on an intelligent agent and a large language model, decomposes the target application system fault text description and the target application system fault diagram text into multiple subtasks, and determines the target application system fault factor based on the information returned by the subtasks. It can automatically discover the root cause of the fault that will not explicitly report errors, such as video playback failure, video distorted screen, content missing, text content error, etc., based on the provided fault diagram and fault description information. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1 A flowchart of an application system fault detection method provided by an embodiment of the present application;

[0043] Figure 2 A flowchart of an application system fault detection method provided by another embodiment of the present application;

[0044] Figure 3 A schematic diagram of the structure of a detection agent provided by an embodiment of the present application;

[0045] Figure 4 A schematic diagram of the structure of an application system fault detection device provided in one embodiment of the present application;

[0046] Figure 5 A schematic diagram of the computer device structure provided for one embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the technical solutions and advantages in the embodiments of the present application more clearly understood, the exemplary embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0048] See also Figure 1 The application system fault detection method provided in the embodiment of the present application includes the following steps 100 to 400:

[0049] Step 100: receiving a target application system fault diagram and a target application system fault description, and inputting the target application system fault diagram into a pre-trained scenario recognition model to determine a target application system fault scenario;

[0050] Step 200: determining a query prompt word for a target application system fault diagram according to a target application system fault scenario;

[0051] Step 300, extracting target application system fault diagram text according to the query prompt word;

[0052] Step 400: Based on the intelligent agent and the large language model, the target application system fault text description and the target application system fault graphic text are decomposed into multiple subtasks, and the target application system fault factors are determined according to the information returned by the subtasks.

[0053] See also Figure 2In step 100, the target application system fault diagram and target application system fault description can be received through the fault submission center. The establishment of the fault submission center and the degree of detail and completeness of the submitted fault determine the difficulty of subsequent task processing. The application system address, title and channel to which it belongs in the fault description should be filled in as completely as possible. At the same time, relevant information can be obtained by clicking the error reporting button on the bottom page. If the fault description does not contain this information, it needs to be included in the submitted picture. At the same time, indicate which content is important on the submission interface.

[0054] In an optional embodiment of the present application, in step 100, the scene recognition model is trained by the following steps:

[0055] Collect historical data of application system failure graphs in each application system failure scenario, where the application system failures include video playback errors, sequence errors, data missing, and content errors, and the application system scenarios include news list pages, news bottom-level playback pages, news graphic pages, sports list pages, etc.;

[0056] The application system failure diagram in each application system failure scenario is taken as input, and the application system failure scenario type is taken as output. The neural network model is trained to obtain a scenario recognition model.

[0057] In an optional embodiment of the present application, in the collection of historical data of application system failure diagrams under each application system failure scenario, the construction of a scene recognition data set includes two parts: a computer terminal and a mobile terminal. According to the different data presentation forms of different terminals, no less than 300 training data are collected for each scene classification, and a training set, a validation set, a test set, and a validation set are constructed in a ratio of 7:1:1:1 for scene recognition model training. The neural network model can be a yolo-v8 model, and the scene recognition model is trained using the classification function of the yolo-v8 model. For these collected data sets, based on the accumulated historical data, scenes that are prone to errors and have a high frequency of errors are summarized, such as video playback errors, sequence errors, missing data, content errors, etc. The scope of the scene is also included as much as possible to include content with differences to improve the accuracy of scene recognition.

[0058] In an optional embodiment of the present application, in step 200, determining the query prompt words of the target application system fault diagram according to the target application system fault scenario includes:

[0059] Determine the query prompt word corresponding to the target application system failure scenario based on the correspondence between the preset application system failure scenario and the query prompt word;

[0060] The query prompt word is used as the query prompt word of the target application system fault diagram.

[0061] In an optional embodiment of the present application, in step 300, extracting target application system fault graph text according to the query prompt word includes:

[0062] Inputting the query prompt words and the target application system fault diagram into the preset multi-modal large model;

[0063] Output the text extracted from the target application system fault diagram according to the query prompt word from the preset multimodal large model as the target application system fault diagram text, wherein the preset multimodal large model can be Qwen2.5 or chatGLM4.0.

[0064] In an optional embodiment of the present application, the correspondence between the preset application system failure scenario and the query prompt word is obtained by the following steps:

[0065] Build different query prompt word templates according to different application system failure scenarios.

[0066] In an optional embodiment of the present application, in step 400, the target application system fault text description and the target application system fault graph text are decomposed into multiple subtasks based on the agent and the large language model, and the target application system fault factor is determined according to the information returned by the subtasks, including:

[0067] Inputting the target application system fault text description and the target application system fault graph text into the scheduling agent;

[0068] The scheduling agent determines the fault handling state according to the target application system fault text description and the target application system fault graph text, and sends the target application system fault text description and the target application system fault graph text to the task decomposition agent if the fault is not handled;

[0069] The task decomposition agent decomposes the fault into multiple subtasks according to the preset prompt word template based on the functions of the search engine, interface and database in the preset knowledge base, and sends the multiple subtasks to the task execution agent, wherein each subtask corresponds to at least one of the search engine, interface and database in the preset knowledge base;

[0070] The task execution agent generates executable programs for each subtask according to the prompt word template by using the preset large language model;

[0071] The task execution agent executes each executable program using at least one of the search engine, interface and database according to the corresponding relationship between each executable program and the search engine, interface and database, and uses the execution result of each executable program as the failure factor of the target application system.

[0072] In an optional embodiment of the present application, the steps of using the search engine, interface and database in the preset knowledge base include:

[0073] For the use of search engines, we built an ES search engine based on our own word segmentation (Elasticsearch, an open source real-time distributed search and analysis engine). Through the title information in the target application system fault map text, we can find the most similar background data, and then through the comparison of release time, scene classification and other information, we can accurately obtain the data identifier of the target application system fault. Search engines are particularly effective for mobile terminals where there are only titles but no identifiers, and they are also compatible with possible errors in content recognition.

[0074] For the use of interfaces, there are seven categories based on the backend data type: video, video collection, pictures and text, picture collection, special topic, audio, and audio collection. They are further divided into single query and list query according to their functions. Each interface needs to explain its function, the tables and query statements involved, the meaning and usage of parameters, and the meaning of the returned fields. Through the API interface (Application Programming Interface, which is a set of pre-defined functions or methods), you can get the single data corresponding to the identifier, the video list under the video collection, the data list in the time period, etc. You can judge whether it is a problem with the page display or the interface.

[0075] For the use of database, we can build a database query knowledge base based on the database query function included in the API. We can remove the query restrictions and query results to find out which conditions are not met or which data anomalies are caused by the failure to update the cache.

[0076] This application builds a knowledge base of how to use the search engine, interface, and database, so that the large language model can know what method to use under what conditions to obtain the correct information.

[0077] In an optional embodiment of the present application, the preset prompt word template is as follows:

[0078] ## Role

[0079] You are very good at breaking down failures into multiple subtasks based on the capabilities of search engines, interfaces, and databases in a preset knowledge base.

[0080] ## Skill

[0081] ### Skill 1: Search Engine

[0082] - Through the title information in the target application system fault image text, the most similar background data can be found. Then, by comparing the release time, scene classification and other information, the data identifier of the target application system fault can be accurately obtained.

[0083] ### Skill 2: Interface

[0084] - Obtain the single data corresponding to the identifier, the video list under the video set, the data list in the time period, etc. through the API interface.

[0085] ### Skill 3: Database

[0086] - Remove the query restriction conditions and query the results to find out which conditions are not met or the data anomaly is caused by the cache not being updated

[0087] ## Reference Case

[0088] enter:

[0089] The image on this page displays an error, and the displayed content is also incorrect. The page belongs to the underlying page of news graphics and text, and the data ID of the page is XXX.

[0090] Output:

[0091] The first subtask: Access the news image and text underlying page API interface, the query ID is XXX, and check whether the queried title content is inconsistent with the displayed content.

[0092] Second subtask: Access the image address obtained by the query and check whether the image address can be accessed normally.

[0093] -------

[0094] Please decompose the fault into multiple subtasks according to the information provided in the following [Fault Content] and the above requirements.

[0095] [Failure details]

[0096] [Subtask Example Table]

[0097] In an optional embodiment of the present application, before taking the execution result of each executable program as a target application system failure factor, the method further includes:

[0098] Send the execution result of each executable program to the task decomposition agent;

[0099] The task decomposition agent determines whether the execution result of each executable program meets the preset conditions. If it does not meet the preset conditions, it adjusts the preset prompt word template, and re-decomposes the fault into multiple subtasks according to the adjusted prompt word template to re-execute multiple executable programs until the execution result of each executable program meets the preset conditions, wherein the preset condition is that the execution result of the subtask is unique.

[0100] In an optional embodiment of the present application, the method further includes:

[0101] The execution result of each executable program is sent to the feedback agent, and the feedback agent sends the execution result of each executable program to the target object corresponding to each subtask.

[0102] In an optional embodiment of the present application, Figure 2 The main function of the detection agent in is to decompose the fault into multiple subtasks, and then comprehensively judge the root cause of the fault based on the information returned by the subtasks. Figure 3 As shown, the detection agent is divided into the following parts according to its functions:

[0103] Dispatch agent: Integrates all information, analyzes the status of fault handling, and sends instructions to different agents. If the processing is completed, the results are returned to the fault submission center.

[0104] Task decomposition agent: The main function is to decompose the submitted fault into appropriate subtasks with single functions according to the prompt word template. For example, for the fault: the picture display of this page is wrong, and the displayed content is also wrong. The page belongs to the bottom page of news pictures and texts, and the data ID of the page is XXX. The task decomposition agent will decompose it into:

[0105] The first subtask: Access the news image and text underlying page API interface, the query ID is XXX, and check whether the queried title content is inconsistent with the displayed content.

[0106] Second subtask: Access the image address obtained by the query and check whether the image address can be accessed normally.

[0107] The task decomposition agent also needs to determine whether the decomposed tasks meet the requirements based on the results returned by the task execution agent. Otherwise, the tasks need to be regenerated and re-executed.

[0108] Task execution agent: The task execution agent calls the set tools according to requirements, fills in the corresponding parameters, executes the task, and returns the returned information to the task decomposition agent.

[0109] The set tools include: interface query tool, which is used to query search engines and API interfaces; database reading tool, which is used to query database information. When executing the first subtask, the image address is obtained through the crawling function, and the API interface in the interface query tool is used to check whether the image address can be accessed normally; when executing the first subtask, the API interface in the interface query tool is used to check whether the query content is different from the displayed content.

[0110] Feedback agent: Feedback the execution results of each executable program to the corresponding department according to the responsibilities and functions of each part in the organizational structure. In an optional embodiment of the present application, the scene recognition model can be a multimodal large model, through which microservices are established and a scene recognition system based on the multimodal large model is formed. The scene recognition system architecture based on the multimodal large model includes hardware, software, database and functional modules. The system hardware is a high-computing-power server. The system database includes a text database and an image database that are uploaded to the system for review. The system functional modules include a system data conversion module, a content review algorithm encapsulation module, a system output module, an unstructured data processing module, a multimodal data integration module, a content analysis module, a content anomaly quantification module and an intelligent control module.

[0111] The data conversion module is used to determine the type of data input by the user into the system, determine the scene category from the input data, and after preprocessing the data, combine it with different questioning methods for each scene, and then transmit it to the multimodal large model for reasoning;

[0112] The unstructured data processing module is used to perform semantic analysis on unstructured data such as news graphics, audio, and video using a multimodal large model, generate a semantic association graph, extract the core context and information relevance in text and images through in-depth analysis, and construct a preliminary review data set;

[0113] The multimodal data integration module is used to integrate the graphic data, audio data and video data in a unified context, convert them into a graphic-text association vector space, and transmit them to the content analysis module for in-depth analysis of content context and emotion;

[0114] The content analysis module is used to preliminarily determine the degree of matching between the text and the image content based on the semantic association graph generated by the unstructured data processing module, generate a context factor Qycs, and analyze the potential sensitive content in each modal data to generate a sensitivity factor Gmyz; then, based on the context factor Qycs and the sensitivity factor Gmyz, perform hierarchical analysis on the potential emotional factors, social sensitive information and potential misleading factors in the image and text content to generate a multimodal association factor Mmgyz;

[0115] The content anomaly quantification module calculates the audit weight value Hqwz based on the situational factor Qycs, the sensitivity factor Gmyz and the multimodal association factor Mmgyz; then obtains the system audit risk factor Rxz by calculation and associates it with the audit weight value Hqwz, thereby generating the final content anomaly index Cxzs;

[0116] Among them, the calculation formula of the audit weight value Hqwz is:

[0117] ;

[0118] Among them, b1, b2 and b3 represent the preset weight coefficients of the situational factor Qycs, the sensitive factor Gmyz and the multimodal association factor Mmgyz respectively, and B represents the audit correction coefficient;

[0119] The intelligent control module is used to set the compliance threshold T and the audit risk threshold R, and compare the audit weight value Hqwz with the compliance threshold T to determine whether the current content meets the audit standards, and compare the audit risk threshold R with the content anomaly index Cxzs to obtain the final audit result, and selectively trigger manual review or automatic release control plans based on the audit results.

[0120] In an optional embodiment of the present application, the unstructured data processing module is first used to receive the unstructured data of news graphics, audio and video to be reviewed, and perform format standardization processing on the unstructured data, including image resolution adjustment, text language standardization processing and audio and video transcoding;

[0121] Then, the pre-trained network of the multimodal large model is called to extract multi-level features of news pictures, audio and video, including visual features, semantic features and emotional features, and the extracted multi-level features are normalized to generate a unimodal feature matrix; based on the unimodal feature matrix, the multi-level features of news pictures, audio and video are mapped in the multimodal association vector space, and a cross-modal semantic association graph is generated through the semantic embedding mechanism of the multimodal large model;

[0122] Secondly, we use the cross-modal semantic association graph to conduct a detailed analysis of the deep-level association between text and images, extract the core contextual elements in the text and images, and simultaneously analyze the emotional state and expression trends in the audio and video, so as to identify potential sensitive information, misleading information and social influencing factors in the news content in real time, and form multi-modal association semantic indicators;

[0123] Finally, a preliminary review dataset is generated.

[0124] In an optional embodiment of the present application, the multimodal data integration module is used to receive the preliminary review data set generated by the unstructured data processing module, and perform uniform alignment processing of the timing and content of the graphic data, audio data and video data therein through a frame synchronization algorithm;

[0125] Then, a multimodal feature mapping algorithm is applied to fuse the context, emotion and semantic features of text, audio and video into a multimodal feature vector, and a correlation matrix is ​​generated based on the correlation between each modality to perform hierarchical mapping of information density, context relevance and semantic consistency.

[0126] Subsequently, the image-text association vector space is constructed based on the association matrix, and the multimodal data is quantified in a unified semantic space to generate an integrated semantic association representation vector. At the same time, the semantic deviation caused by modality differences is corrected through the context adjustment algorithm.

[0127] Finally, the integrated image-text association vector space is transmitted to the content analysis module.

[0128] In an optional embodiment of the present application, the content analysis module includes a preliminary discrimination unit, a multi-layer sensitivity analysis unit and a correlation factor generation unit;

[0129] The preliminary discrimination unit identifies semantic deviation related data and content matching related data by analyzing the semantic similarity and context consistency between the image and the text; extracts the semantic similarity Ycy, context consistency Sjy, context association deviation Qjl and image and text emotion consistency Twq from the semantic deviation related data and the content matching related data and performs dimensionless processing, and then calculates the context factor Qycs through the following formula:

[0130] ;

[0131] The preset situation threshold Q is compared with the situation factor Qycs to evaluate the situational relevance of text and images in the audit scenario. The specific comparison and evaluation contents are as follows:

[0132] If the context factor Qycs ≥ the context threshold Q, it indicates that the contexts of the text and the image match, and the semantics, emotions, and contexts of the two are consistent. The review is passed and marked as "normal context association";

[0133] If the context factor Qycs < context threshold Q, it indicates that the context between the text and the image does not match, and the semantics, emotions and context of the two are inconsistent. The review will fail and further analysis of the specific reasons for the deviation will be conducted, including potential sensitive content.

[0134] In an optional embodiment of the present application, the multi-layer sensitivity analysis unit is used to further perform in-depth analysis on the potential sensitive content of the graphic data, including analyzing the emotion-related data, social sensitivity-related data and potential misleading-related data in the text and image, and constructing a sensitive content data set after summarizing and dimensionlessly processing the emotion-related data, social sensitivity-related data and potential misleading-related data in the analyzed text and image; extracting the sensitive content data set and generating the sensitivity factor Gmyz through the following formula:

[0135] ;

[0136] Where Qmq represents the emotion intensity coefficient in the sensitive content data set, Mgc represents the sensitive word index in the sensitive content data set, and Xwg represents the possibility of information misleading in the sensitive content data set.

[0137] In an optional embodiment of the present application, the correlation factor generation unit is used to quantify the degree of correlation between emotion, context and sensitive content based on the context factor Qycs and the sensitivity factor Gmyz by fusing multi-level information in the image-text correlation vector space, and generate a multimodal correlation factor Mmgyz by calculating through the following formula;

[0138] .

[0139] In an optional embodiment of the present application, the content anomaly quantification module includes an audit calculation unit and an anomaly index acquisition unit;

[0140] The content evaluation unit is used to construct a system audit risk factor Rxz, obtain the semantic deviation degree Ycy and the content consistency index Nry by extracting semantic deviation related data and content matching related data, and obtain the high sensitivity trigger rate Gmg by extracting the sensitive content data set, and calculate the system audit risk factor Rxz in combination with the following formula:

[0141] .

[0142] In an optional embodiment of the present application, the abnormality index obtaining unit is used to calculate and obtain the final content abnormality index Cxzs, and its specific calculation formula is as follows:

[0143]

[0144] In an optional embodiment of the present application, the intelligent control module compares and evaluates the audit weight value Hqwz with the compliance threshold T, and compares and evaluates the audit risk threshold R with the content anomaly index Cxzs, and specifically generates the following evaluation content:

[0145] Compliance standards comparison:

[0146] If the audit weight value Hqwz ≥ the compliance threshold T, it means that the content meets the audit standards and the system marks it as "compliant".

[0147] If the audit weight value Hqwz < the compliance threshold T, it means that the content does not meet the audit standards and is insufficiently compliant, requiring further review or adjustment.

[0148] Abnormal risk assessment:

[0149] If the content anomaly index Cxzs ≥ the audit risk threshold R, it indicates that the content has an abnormal risk, and the system generates an "abnormal content" mark, triggering the manual review process;

[0150] If the content anomaly index Cxzs < the audit risk threshold R, it means that there is no abnormal risk in the content, and the system generates a "normal content" mark, and then enters the automatic publishing process;

[0151] When the evaluation results of the audit weight value Hqwz and the content anomaly index Cxzs are marked as "compliant" and "normal content" at the same time, the content will be automatically marked as "passed" and published directly;

[0152] When the evaluation result of the audit weight value Hqwz and the content anomaly index Cxzs is that the content does not meet the audit standards or has abnormal risks, the content will be automatically marked as "pending review" and the selective review process will be triggered.

[0153] It is used to set the compliance threshold T and the audit risk threshold R, and to determine whether the current content meets the audit standards, obtain the final audit results, and selectively trigger manual review or automatic release control plans based on the audit results.

[0154] In an optional embodiment of the present application, a content audit algorithm encapsulation module is used to encapsulate all algorithms in the system.

[0155] In an optional embodiment of the present application, the system output module is used to extract key information of the results obtained based on the large model and output the final content review category.

[0156] In an optional embodiment of the present application, the first sub-model of the scene recognition model is used to determine the target application system failure scenario according to the target application system failure graph.

[0157] In an optional embodiment of the present application, the second sub-model of the scene recognition model is used to receive and obtain the review result of the comment data of the specified video based on the specified video and its corresponding comment data.

[0158] In an optional embodiment of the present application, the second sub-model of the scene recognition model is trained by the following steps:

[0159] Get the target video and its corresponding comment data;

[0160] Based on the correlation between the target video and its corresponding comment data, the comment data corresponding to the target video is annotated to construct a comment dataset corresponding to the target video;

[0161] The target video and its corresponding comment dataset are used as training sets, and the preset multimodal model is fine-tuned to obtain the second sub-model of the trained scene recognition model.

[0162] In an optional embodiment of the present application, the tagging of the comment data corresponding to the target video based on the correlation between the target video and the comment data corresponding to the target video includes:

[0163] Extract key information and sentiment information from the comment data corresponding to the target video, and preliminarily determine whether the comment data is a negative comment based on the sentiment information;

[0164] If it is preliminarily determined that the comment data is a negative comment, marking the comment data as a negative comment;

[0165] In the case where it is preliminarily determined that the comment data is a positive comment, further determining whether the comment data is a negative comment based on the correlation between the target video and its corresponding comment data;

[0166] If it is further determined that the comment data is a negative comment, marking the comment data as a negative comment;

[0167] When it is further determined that the comment data is a positive comment, the comment data is marked as a positive comment.

[0168] In an optional embodiment of the present application, further determining whether the comment data is a negative comment based on the correlation between the target video and its corresponding comment data includes:

[0169] Extract key information and sentiment information from the target video, and calculate the similarity between the key information and sentiment information in the target video and its corresponding comment data;

[0170] Determine the degree of conflict between the target video and the text content in the corresponding comment data;

[0171] Determine the conflict score between the target video and its corresponding comment data according to the similarity between the key information and emotional information in the target video and its corresponding comment data and the degree of conflict between the text contents;

[0172] The review data whose conflict score exceeds a preset conflict score threshold is determined as a negative review.

[0173] In an optional embodiment of the present application, the similarity between the key information and the sentiment information in the target video and its corresponding comment data is calculated by the following expression:

[0174]

[0175] Among them, A is the first text vector composed of key information and emotional information in the target video, B is the second text vector composed of key information and emotional information in the comment data corresponding to the target video, a i is the i-th vector element in the first text vector, b i is the i-th vector element in the second text vector, and w1 and w2 are both weight factors.

[0176] In an optional embodiment of the present application, constructing a comment dataset corresponding to the target video includes:

[0177] Each comment data corresponding to the target video and the annotation information of each comment data are collected into a comment data set corresponding to the target video.

[0178] In an optional embodiment of the present application, the target video and its corresponding comment data set are used as training sets to fine-tune the preset multimodal model to obtain a trained scene recognition model, including:

[0179] The video frame of the target video and each comment data are taken as input, and the annotation information of the current comment data is taken as output. The network structure used to process the video frames in the preset multimodal model is trained using the Lora fine-tuning method of supervised tasks, and the network structure used to process the comment data is trained using the QLoRa fine-tuning method of supervised tasks to obtain the trained scene recognition model.

[0180] In an optional embodiment of the present application, the inputting of the specified video and its corresponding comment data into the scene recognition model to obtain the review result of the comment data of the specified video includes:

[0181] Input the video frame of the specified video into the network structure used for processing the video frame in the scene recognition model to obtain the video frame features;

[0182] Input the comment data corresponding to the specified video into the network structure used to process the comment data in the scene recognition model to obtain the comment data features;

[0183] The video frame features and the comment data features are input into the fusion network structure in the scene recognition model to obtain the review results of the comment data of the specified video.

[0184] In an optional embodiment of the present application, the third sub-model of the scene recognition model is also used to determine whether the specified file is a target file to be deleted based on the file title, time and text information of the specified file.

[0185] In an optional embodiment of the present application, the third sub-model of the scene recognition model is trained by the following steps:

[0186] The third sub-model of the scene recognition model is trained by taking the file title, time and text information of the historical file as input and whether the file is a file to be deleted as output.

[0187] In an optional embodiment of the present application, the file title, time and text information of the specified file are obtained by the following steps:

[0188] Receive the target file by using the file transfer module, send the received target file to the content distribution network module, and send the information of the target file to the message server;

[0189] The file parsing module is used to obtain the information of the target file on the message server, obtain the target file, parse the target file, and push the parsed file title, time and text information to the index module.

[0190] In an optional embodiment of the present application, a file parsing module is used to obtain information of a target file on a message server, obtain the target file, parse the target file, and push the parsed file title, time, and text information to an indexing module, including:

[0191] Get the file type of the target file;

[0192] According to the file type of the target file, determine whether the target file needs to be preprocessed;

[0193] When the target file needs to be preprocessed, the target file is preprocessed, and then the preprocessed target file is parsed;

[0194] Without preprocessing the target file, the target file is directly parsed;

[0195] Push the parsed file title, time and text information to the index module.

[0196] In an optional embodiment of the present application, determining whether the target file needs to be preprocessed according to the file type of the target file includes:

[0197] When the file type of the target file is a text file, there is no need to preprocess the target file;

[0198] When the file type of the target file is a JSON file or an HTML file, the target file needs to be preprocessed.

[0199] In an optional embodiment of the present application, when the target file needs to be preprocessed, the target file is preprocessed, including:

[0200] When the file type of the target file is a JSON file, delete the JSON tag in the JSON file;

[0201] When the file type of the target file is an HTML file, and the HTML file contains at least one of audio and video, obtain the globally unique identifiers of the audio and video, obtain the identification text of the audio and video in the HTML file through the globally unique identifiers of the audio and video, and then use the identification text as the HTML file to extract the largest body text block of the HTML file; when the HTML file does not contain at least one of audio and video, extract the largest body text block of the HTML file.

[0202] In an optional embodiment of the present application, extracting the largest body text block of the HTML file includes:

[0203] Parse the tags in the HTML file into a DOM tree and obtain the total number of tags in the HTML file and the location information of each tag;

[0204] According to the number of line break tags and paragraph tags in each node of the HTML file, all nodes are sorted to get the top N nodes;

[0205] For each of the top N nodes, perform secondary sorting based on the number of links, the number of Chinese punctuation marks, the total number of words and the number of words with links, the total number of tags in the HTML file, and the position information of each tag, and take the first node as the largest text block;

[0206] Move forward from the node position of the largest text block, extract the title of the HTML file, extract the release time of the HTML file through a regular expression, and if the extraction is successful, use the text of the first node as the text of the HTML file; if the extraction is unsuccessful, use the second node in the secondary sorting as the largest text block, and extract the title and release time of the HTML file until the extraction is successful;

[0207] When the first node is taken as the largest body text block, if the body of the first node cannot be extracted or the number of extracted body words is less than a preset threshold, the html tag in the HTML file is directly deleted, and the HTML file with the html tag deleted is taken as the body of the HTML file.

[0208] In an optional embodiment of the present application, all nodes are sorted according to the number of line break tags and paragraph tags in each node on the DOM tree to obtain the top N nodes, including:

[0209] The first weight value of each node is determined by the following expression:

[0210] W1 = brCount + 2 pCount

[0211] Where W1 is the first weight value of each node, brCount is the number of line break tags of each node, and pCount is the number of paragraph tags of each node.

[0212] All nodes are sorted from largest to smallest according to their first weight values ​​to obtain the top N nodes.

[0213] In an optional embodiment of the present application, for each of the top N nodes, a secondary sorting is performed on all the top N nodes according to the number of links, the number of Chinese punctuation marks, the total number of words and the number of words with links of the current node, the total number of tags of the HTML file, and the position information of each tag, including:

[0214] The second weight value of each node is determined by the following expression:

[0215] W2 = (WN-LWL) × dl × f

[0216] Wherein, W2 is the second weight value of each node, WN is the total number of words of each node, LWL is the number of linked words of each node, dl is the location information of each node, and f is the weight factor of each node.

[0217] According to the second weight values ​​of all the nodes in the top N, sort them again from large to small.

[0218] Among them, the position information of each node is obtained by the following expression:

[0219]

[0220] Among them, dl is the location information of each node, NodeCount and NodePos are the total number of tags in the HTML file and the location information of each tag, respectively.

[0221] The weight factor of each node is obtained by the following expression:

[0222]

[0223] Among them, f is the weight factor of each node, dLinkPower is the number of links of each node, and nCDotNum is the number of Chinese punctuation marks of each node.

[0224] This application feeds back the execution results of each executable program to the corresponding department and submits it to the corresponding function developer. It can perform the detection function in an anthropomorphic and automatic manner. The person asking the question does not need to know which departments the question is related to, thereby reducing communication costs and detection time and improving efficiency.

[0225] In an optional embodiment of the present application, the method further includes:

[0226] The execution result of each executable program is used as the detection result, and the detection result is compared with the manual verification result. The scene recognition model and the prompt words of the detection agent are adjusted according to the comparison result.

[0227] This application establishes a feedback mechanism for detection results and manual verification results, which can improve the accuracy of the scene recognition model, optimize the prompt words of the detection agent, and continuously improve the accuracy of detection.

[0228] This application first collects various page image data to build a scene recognition data set, and uses the classification function of the yolo-v8 model to train the scene recognition model; then predicts the submitted problem image and identifies the scene in which it is located. Secondly, different prompt words are constructed through different scenes, and the large model is used to identify the text content in the image, including link addresses, titles, identifiers, text content, etc.; these contents are submitted to the detection agent. The agent decides to use interface query, database query, search engine interface, etc. to obtain correct information based on the constructed interface, database and other usage method knowledge bases. Finally, the fault information is compared with the correct information, and the results are returned to the fault submission center and the development team where the problem is located to improve the efficiency of fault detection.

[0229] It should be understood that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0230] See also Figure 4 An embodiment of the present application provides an application system fault detection device 400, comprising:

[0231] An input module 410 is used to receive a target application system fault diagram and a target application system fault description, and input the target application system fault diagram into a pre-trained scenario recognition model to determine a target application system fault scenario;

[0232] A determination module 420, configured to determine a query prompt word of a target application system fault diagram according to a target application system fault scenario;

[0233] An extraction module 430 is used to extract target application system fault diagram text according to the query prompt word;

[0234] The decomposition module 440 is used to decompose the target application system fault text description and the target application system fault graphic text into multiple subtasks based on the intelligent agent and the large language model, and determine the target application system fault factor according to the information returned by the subtasks.

[0235] For the specific definition of the above-mentioned device 400, please refer to the definition of the application system fault detection method above, which will not be repeated here. Each module in the above-mentioned device 400 can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0236] In one embodiment, a computer device is provided, the internal structure diagram of which can be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an application system fault detection method as described above is implemented. It includes: a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, any step in the application system fault detection method as described above is implemented.

[0237] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any step in the above application system fault detection method can be implemented.

[0238] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0239] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0240] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0241] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0242] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0243] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for detecting application system failures, characterized in that: include: receiving a target application system fault diagram and a target application system fault description, and inputting the target application system fault diagram into a pre-trained scenario recognition model to determine a target application system fault scenario; Determine the query prompt words of the target application system fault diagram according to the target application system fault scenario; Extract target application system fault diagram text according to query prompt words; Based on the agent and large language model, the target application system fault text description and target application system fault graph text are decomposed into multiple subtasks. According to the information returned by the subtasks, the target application system fault factors are determined. Wherein, based on the agent and the large language model, the target application system fault text description and the target application system fault graph text are decomposed into multiple subtasks, and the target application system fault factors are determined according to the information returned by the subtasks, including: Inputting the target application system fault text description and the target application system fault graph text into the scheduling agent; The scheduling agent determines the fault handling state according to the target application system fault text description and the target application system fault graph text, and sends the target application system fault text description and the target application system fault graph text to the task decomposition agent if the fault is not handled; The task decomposition agent decomposes the fault into multiple subtasks according to the preset prompt word template based on the functions of the search engine, interface and database in the preset knowledge base, and sends the multiple subtasks to the task execution agent, wherein each subtask corresponds to at least one of the search engine, interface and database in the preset knowledge base; The task execution agent generates executable programs for each subtask according to the prompt word template by using the preset large language model; The task execution agent executes each executable program using at least one of the search engine, interface and database according to the corresponding relationship between each executable program and the search engine, interface and database, and uses the execution result of each executable program as the failure factor of the target application system.

2. The method according to claim 1, characterized in that: The scene recognition model is trained by the following steps: Collect historical data of application system failure graphs under each application system failure scenario, wherein the application system failures include video playback errors, sequence errors, data missing, and content errors; The application system failure diagram in each application system failure scenario is taken as input, and the application system failure scenario type is taken as output. The neural network model is trained to obtain a scenario recognition model.

3. The method according to claim 1, characterized in that The step of determining the query prompt words of the target application system fault diagram according to the target application system fault scenario includes: Determine the query prompt word corresponding to the target application system failure scenario based on the correspondence between the preset application system failure scenario and the query prompt word; The query prompt word is used as the query prompt word of the target application system fault diagram.

4. The method according to claim 1, characterized in that: The step of extracting target application system fault graph text according to the query prompt word includes: Inputting the query prompt words and the target application system fault diagram into the preset multi-modal large model; The text extracted from the target application system fault diagram according to the query prompt words is output from the preset multimodal large model as the target application system fault diagram text.

5. The method according to claim 1, characterized in that Before taking the execution result of each executable program as the target application system failure factor, the method further includes: Send the execution result of each executable program to the task decomposition agent; The task decomposition agent determines whether the execution result of each executable program meets the preset conditions. If it does not meet the preset conditions, it adjusts the preset prompt word template, and re-decomposes the fault into multiple subtasks according to the adjusted prompt word template to re-execute multiple executable programs until the execution result of each executable program meets the preset conditions.

6. The method according to claim 1, characterized in that The method further comprises: The execution result of each executable program is sent to the feedback agent, and the feedback agent sends the execution result of each executable program to the target object corresponding to each subtask.

7. An application system fault detection device, characterized in that: include: An input module is used to receive a target application system fault diagram and a target application system fault description, and input the target application system fault diagram into a pre-trained scenario recognition model to determine a target application system fault scenario; A determination module, used to determine a query prompt word of a target application system fault diagram according to a target application system fault scenario; An extraction module is used to extract the target application system fault diagram text according to the query prompt word; The decomposition module is used to decompose the target application system fault text description and target application system fault graph text into multiple subtasks based on the intelligent agent and large language model, and determine the target application system fault factors based on the information returned by the subtasks. Wherein, based on the agent and the large language model, the target application system fault text description and the target application system fault graph text are decomposed into multiple subtasks, and the target application system fault factors are determined according to the information returned by the subtasks, including: Inputting the target application system fault text description and the target application system fault graph text into the scheduling agent; The scheduling agent determines the fault handling state according to the target application system fault text description and the target application system fault graph text, and sends the target application system fault text description and the target application system fault graph text to the task decomposition agent if the fault is not handled; The task decomposition agent decomposes the fault into multiple subtasks according to the preset prompt word template based on the functions of the search engine, interface and database in the preset knowledge base, and sends the multiple subtasks to the task execution agent, wherein each subtask corresponds to at least one of the search engine, interface and database in the preset knowledge base; The task execution agent generates executable programs for each subtask according to the prompt word template by using the preset large language model; The task execution agent executes each executable program using at least one of the search engine, interface and database according to the corresponding relationship between each executable program and the search engine, interface and database, and uses the execution result of each executable program as the failure factor of the target application system.

8. A computer device comprising: A memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the application system fault detection method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the application system fault detection method according to any one of claims 1 to 6 are implemented.

Images