Fault analysis model training method, fault analysis method and device

Abstract

The embodiment of the invention provides a fault analysis model training method and device and a fault analysis method and device.The model training method comprises the steps that an initial fault analysis model, a first sample log and a first labeled fault analysis result are obtained, and the first sample log is input into the initial fault analysis model, enabling the initial fault analysis model to output a first fault analysis result based on the first fault analysis instruction, calculating first loss, and adjusting parameters of the initial fault analysis model to obtain a pre-trained fault analysis model; obtaining a training sample knowledge base, and verifying the accuracy of the pre-training fault analysis model by using a second sample log and a second labeled fault analysis result included in the training sample knowledge base; and when the accuracy is satisfied, adjusting parameters of the first fault analysis instruction and the pre-trained fault analysis model based on the second sample log and the second labeled fault analysis result to obtain a second fault analysis instruction and a fault analysis model. By applying the technical scheme of the invention, the efficiency and accuracy of fault analysis are improved.

Description

Technical Field

[0001] The present invention relates to the field of multimedia processing technology, and in particular to a fault analysis model training method, a fault analysis method and a device. Background Art

[0002] When users encounter playback errors and other problems while using the player to play videos, they usually report the problems they encounter to the player backend. After receiving the fault information reported by the user, the player backend will have professional staff investigate, analyze, locate and solve the problems encountered by the user during video playback, and the user will eventually successfully play the video. The troubleshooting process is: user reports playback failure problem -> customer service staff receives user feedback -> business line staff investigates the problem -> professional staff analyzes, locates and solves the problem -> customer service staff reports the results of the user's problem solving -> the user plays the video normally. This troubleshooting method requires a lot of manual review operations, which is time-consuming and inefficient. It may also fail to accurately restore the error scenario, which in turn leads to inaccurate solutions to the problems. Summary of the invention

[0003] The purpose of the embodiments of the present invention is to provide a fault analysis model training method, a fault analysis method and a device to improve the efficiency and accuracy of fault analysis. The specific technical solution is as follows:

[0004] In a first aspect of the present invention, a fault analysis model training method is first provided, the method comprising:

[0005] Obtaining an initial fault analysis model;

[0006] Obtain a first sample log and a first annotated fault analysis result corresponding to the first sample log; the first sample log is a log corresponding to a video playback error; the first annotated fault analysis result includes at least one of the following: a playback error scenario, an error cause, a solution, and an error scenario category;

[0007] Inputting the first sample log into the initial fault analysis model, so that the initial fault analysis model outputs a first fault analysis result based on a first fault analysis instruction;

[0008] Calculating a first loss between the first fault analysis result and the first labeled fault analysis result, and adjusting parameters of the initial fault analysis model according to the first loss to obtain a pre-trained fault analysis model;

[0009] Acquire a training sample knowledge base; the training sample knowledge base includes a second sample log and a second annotated fault analysis result corresponding to the second sample log;

[0010] Verifying the accuracy of the pre-trained fault analysis model using the second sample log and the second labeled fault analysis result;

[0011] When the accuracy rate is not less than a preset accuracy rate, the parameters of the first fault analysis instruction and the pre-trained fault analysis model are adjusted based on the second sample log and the second labeled fault analysis result to obtain a second fault analysis instruction and a fault analysis model.

[0012] Optionally, the method further comprises:

[0013] When the accuracy is lower than the preset accuracy, the process returns to the step of obtaining the initial fault analysis model.

[0014] Optionally, obtaining an initial fault analysis model includes:

[0015] Obtain a third sample log and a third annotated fault analysis result corresponding to the third sample log;

[0016] Calling a plurality of preset large language models through a preset application programming interface API, inputting the third sample log into each of the large language models respectively, and obtaining a third fault analysis result output by each of the large language models;

[0017] An initial fault analysis model is selected from the plurality of large language models based on a difference between each third fault analysis result and the third annotated fault analysis result.

[0018] Optionally, acquiring a training sample knowledge base includes:

[0019] Obtain sample feedback information when a user makes an error while playing a video within a preset historical time period;

[0020] Determining the type of each sample feedback information according to the feedback content of the sample feedback information;

[0021] Obtain the second sample log corresponding to each sample feedback information under each type;

[0022] Determine a second annotated fault analysis result corresponding to each of the second sample logs;

[0023] A training sample knowledge base is constructed based on each of the second sample logs and each of the second labeled fault analysis results.

[0024] Optionally, the method further comprises:

[0025] The training sample knowledge base is updated using the fourth sample log and the fourth labeled fault analysis result corresponding to the fourth sample log.

[0026] Optionally, the verifying the accuracy of the pre-trained fault analysis model by using the second sample log and the second annotated fault analysis result includes:

[0027] Inputting the second sample log into the pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a second fault analysis result based on the first fault analysis instruction;

[0028] Determine whether the second fault analysis result is consistent with the second marked fault analysis result, and obtain a judgment result;

[0029] Based on the judgment result, the accuracy of the pre-trained fault analysis model is determined.

[0030] Optionally, adjusting the first fault analysis instruction and the parameters of the pre-trained fault analysis model based on the second sample log and the second annotated fault analysis result to obtain a second fault analysis instruction and a fault analysis model includes:

[0031] Obtaining a plurality of fifth sample logs and fifth annotated fault analysis results corresponding to the plurality of fifth sample logs;

[0032] Inputting a plurality of the fifth sample logs into the pre-trained fault analysis model, causing the pre-trained fault analysis model to output a fifth fault analysis result based on the first fault analysis instruction, and calculating a second loss between the fifth fault analysis result and the fifth labeled fault analysis result;

[0033] Using the second sample log and the second annotated fault analysis result as context, querying a plurality of sixth fault analysis results corresponding to the fifth sample logs, and calculating a third loss between the sixth fault analysis result and the fifth annotated fault analysis result;

[0034] Inputting a plurality of the fifth sample logs into the pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a seventh fault analysis result based on the received user interaction information, and obtains an interaction instruction for the seventh fault analysis result;

[0035] Based on the second loss, the third loss and the interaction instruction, the parameters of the pre-trained fault analysis model and the first fault analysis instruction are adjusted to obtain a second fault analysis instruction and a fault analysis model.

[0036] Optionally, adjusting the parameters of the pre-trained fault analysis model and the first fault analysis instruction based on the second loss, the third loss and the interaction instruction to obtain the second fault analysis instruction and the fault analysis model includes:

[0037] When the second loss does not satisfy a preset condition, the third loss does not satisfy a preset condition, and the interaction instruction is not to stop the interaction, returning to the step of obtaining an initial fault analysis model;

[0038] When both the second loss and the third loss meet preset conditions and the interaction instruction is to stop interaction, the current first fault analysis instruction is determined as the second fault analysis instruction and the current parameters of the pre-trained fault analysis model are determined as the parameters of the fault analysis model.

[0039] Optionally, the fifth marked fault analysis result includes at least one of the following: a playback error scenario, an error cause, a solution, an error scenario category, and fault analysis statistical information; the fault analysis statistical information is determined based on the playback error scenario, the error cause, or the error scenario category.

[0040] In a second aspect of the present invention, a fault analysis method is provided, the method comprising:

[0041] Obtaining target feedback information to be analyzed and a target log corresponding to the target feedback information;

[0042] Obtaining a target fault analysis instruction; the target fault analysis instruction is obtained by using any of the above-mentioned fault analysis model training methods;

[0043] The target log is input into a pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a target fault analysis result based on the target fault analysis instruction; wherein the target fault analysis result includes at least one of the following: playback error scenario, error cause, solution and error scenario category; the pre-trained fault analysis model is obtained by adopting any of the above-mentioned fault analysis model training methods.

[0044] Optionally, the target feedback information includes: at least one of an Internet Protocol address IP address of a target user, a user login identifier UID, a device identifier ID and a feedback ID, and a feedback time; and obtaining a target log corresponding to the target feedback information includes:

[0045] Based on the IP address and the feedback time, query the Hive database to obtain the first server log and the first automatic reference notification Pingback log corresponding to the target feedback information;

[0046] Based on the IP address, the UID, the device ID, the feedback ID, and the feedback time, query the Compass log to obtain a first local playback log corresponding to the target feedback information;

[0047] Based on the IP address, the UID, the device ID, the feedback ID, and the feedback time, obtaining a first system log corresponding to the target feedback information;

[0048] Aggregate the first local playback log and the first system log to obtain a first local log;

[0049] The first server log, the first Pingback log, and the first local log are formatted to obtain a target log.

[0050] Optionally, the aggregating the first local playback log and the first system log to obtain a first local log includes:

[0051] Merging the first local playback log with the first system log to obtain a log to be processed;

[0052] The to-be-processed logs are processed according to a preset network request format, and logs of a target type are screened and retained to obtain a first local log.

[0053] Optionally, the target feedback information includes: at least one of the target user's Internet Protocol address IP address, user login identifier UID, device identifier ID and feedback ID, as well as feedback time and feedback content; the target feedback information is multiple, and obtaining the target log corresponding to the target feedback information includes:

[0054] Determining the type of each target feedback information based on the feedback content of each target feedback information;

[0055] Based on the IP address and the feedback time, query the Hive database to obtain the second server log and the second automatic reference notification Pingback log corresponding to each target feedback information under each type;

[0056] Based on the IP address, the UID, the device ID, the feedback ID, and the feedback time, query the Compass log to obtain the second local playback log corresponding to each target feedback information under each type;

[0057] Based on the IP address, the UID, the device ID, the feedback ID, and the feedback time, obtaining a second system log corresponding to each target feedback information under each type;

[0058] Aggregate the second local playback log and the second system log corresponding to each target feedback information under each type to obtain a second local log;

[0059] The second server log, the second Pingback log and the second local log of each target feedback information of each type are formatted in time sequence to obtain a target log.

[0060] Optionally, the target fault analysis result also includes fault analysis statistical information, which is determined based on the playback error scenario, the error cause or the error scenario category, and includes at least one of the following: fault information counted according to feedback content, fault information counted according to error cause, and fault information counted according to a server associated with the target feedback information.

[0061] Optionally, the method further comprises:

[0062] When there is fault information satisfying the alarm condition in the fault analysis statistical information, an alarm is triggered.

[0063] In a third aspect of the present invention, a fault analysis model training device is provided, the device comprising:

[0064] A first acquisition module is used to acquire an initial fault analysis model;

[0065] The second acquisition module is used to obtain a first sample log and a first annotated fault analysis result corresponding to the first sample log; the first sample log is a log corresponding to a video playback error; the first annotated fault analysis result includes at least one of the following: a playback error scenario, an error cause, a solution, and an error scenario category;

[0066] An analysis and prediction module, configured to input the first sample log into the initial fault analysis model, so that the initial fault analysis model outputs a first fault analysis result based on a first fault analysis instruction;

[0067] A first training module, used for calculating a first loss between the first fault analysis result and the first labeled fault analysis result, and adjusting parameters of the initial fault analysis model according to the first loss to obtain a pre-trained fault analysis model;

[0068] A third acquisition module is used to acquire a training sample knowledge base; the training sample knowledge base includes the second sample log and the second annotated fault analysis result corresponding to the second sample log;

[0069] A verification module, used to verify the accuracy of the pre-trained fault analysis model using the second sample log and the second annotated fault analysis result;

[0070] The second training module is used to adjust the parameters of the first fault analysis instruction and the pre-trained fault analysis model based on the second sample log and the second labeled fault analysis result when the accuracy rate is not less than the preset accuracy rate, so as to obtain the second fault analysis instruction and the fault analysis model.

[0071] In a fourth aspect of the present invention, a fault analysis device is provided, the device comprising:

[0072] A fourth acquisition module, used to acquire target feedback information to be analyzed and a target log corresponding to the target feedback information;

[0073] A fifth acquisition module is used to acquire a target fault analysis instruction; the target fault analysis instruction is obtained by using any of the above-mentioned fault analysis model training methods;

[0074] A fault analysis module is used to input the target log into a pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a target fault analysis result based on the target fault analysis instruction; wherein the target fault analysis result includes at least one of the following: playback error scenario, error cause, solution and error scenario category; the pre-trained fault analysis model is trained using any of the above-mentioned fault analysis model training methods.

[0075] In another aspect of the present invention, there is provided an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0076] Memory, used to store computer programs;

[0077] The processor is used to implement any of the above-mentioned fault analysis model training methods or fault analysis methods when executing the program stored in the memory.

[0078] In another aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, any of the above-mentioned fault analysis model training methods or fault analysis methods is implemented.

[0079] In another aspect of the present invention, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned fault analysis model training methods or fault analysis methods.

[0080] The embodiment of the present invention provides a fault analysis model training method, a fault analysis method and a device, which fine-tune the initial fault analysis model through a first sample log, a first labeled fault analysis result corresponding to the first sample log and a first fault analysis instruction to obtain a pre-trained fault analysis model, and then use the second sample log included in the training sample knowledge base and the second labeled fault analysis result corresponding to the second sample log to verify the pre-trained fault analysis model, and when the accuracy of the pre-trained fault analysis model is not less than the preset accuracy, adjust the parameters of the first fault analysis instruction and the pre-trained fault analysis model based on the second sample log and the second labeled fault analysis result to obtain a second fault analysis instruction and a fault analysis model, so that the obtained second fault analysis instruction and the fault analysis model can perform fault analysis more accurately and no longer rely on manual operation, thereby improving the efficiency of fault analysis. In addition, the labeled fault analysis result includes at least one of the following: playback error scenario, error cause, solution and error scenario category, so that the trained fault analysis model can accurately restore the error scenario to provide an accurate solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0082] Figure 1 A schematic diagram of a flow chart of a fault analysis model training method in an embodiment of the present invention;

[0083] Figure 2 A schematic diagram of a process for obtaining an initial fault analysis model in an embodiment of the present invention;

[0084] Figure 3 A schematic diagram of a flow chart of a method for constructing a training sample knowledge base in an embodiment of the present invention;

[0085] Figure 4 A schematic diagram of a flow chart of a model parameter adjustment training method in an embodiment of the present invention;

[0086] Figure 5 A schematic diagram of a manual parameter adjustment method in an embodiment of the present invention;

[0087] Figure 6 A schematic diagram of a fault analysis model training method in an embodiment of the present invention;

[0088] Figure 7 A schematic diagram of a flow chart of a fault analysis method in an embodiment of the present invention;

[0089] Figure 8 A schematic diagram of a flow chart of a log acquisition method in an embodiment of the present invention;

[0090] Fig. 9 Another flow chart of the log acquisition method in an embodiment of the present invention;

[0091] Fig.10 A schematic diagram of the structure of a fault analysis model training device in an embodiment of the present invention;

[0092] Fig.11 A schematic diagram of the structure of a fault analysis device in an embodiment of the present invention;

[0093] Fig.12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0094] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0095] In order to solve the problem that the fault handling method in the related art requires a large number of manual review operations, which is time-consuming and inefficient, and may not accurately restore the error scenario, thereby resulting in the problem that the generated solution to the problem is not accurate enough, the embodiment of the present invention provides a fault analysis model training method, a fault analysis method and a device. The fault analysis model training method and the fault analysis method provided by the embodiment of the present invention can be applied to the scenario where the user encounters a problem in the process of playing a video with a player, and the user gives feedback on the problem encountered, and the customer service system of the player receives the feedback information, which is implemented by the customer service system, which can specifically be a client or server system device, etc.

[0096] The following first introduces in detail a fault analysis model training method provided by an embodiment of the present invention:

[0097] like Figure 1 As shown, Figure 1 This is a flow chart of a method for training a fault analysis model in an embodiment of the present invention. The method includes:

[0098] S101, obtaining an initial fault analysis model.

[0099] S102: Obtain a first sample log and a first annotated fault analysis result corresponding to the first sample log.

[0100] Among them, the first sample log is a log corresponding to an error in playing a video; the first annotated fault analysis result includes at least one of the following: a playback error scenario, an error cause, a solution, and an error scenario category.

[0101] S103: input the first sample log into the initial fault analysis model, so that the initial fault analysis model outputs a first fault analysis result based on the first fault analysis instruction.

[0102] S104, calculating a first loss between the first fault analysis result and the first labeled fault analysis result, and adjusting parameters of the initial fault analysis model according to the first loss to obtain a pre-trained fault analysis model.

[0103] S105: Acquire a training sample knowledge base, where the training sample knowledge base includes a second sample log and a second annotated fault analysis result corresponding to the second sample log.

[0104] S106: Verify the accuracy of the pre-trained fault analysis model using the second sample log and the second annotated fault analysis result.

[0105] S107, when the accuracy is not less than the preset accuracy, adjusting the parameters of the first fault analysis instruction and the pre-trained fault analysis model based on the second sample log and the second annotated fault analysis result to obtain a second fault analysis instruction and a fault analysis model.

[0106] The fault analysis model training method provided by the embodiment of the present invention fine-tunes the initial fault analysis model through the first sample log, the first labeled fault analysis result corresponding to the first sample log, and the first fault analysis instruction to obtain a pre-trained fault analysis model, and then uses the second sample log included in the training sample knowledge base and the second labeled fault analysis result corresponding to the second sample log to verify the pre-trained fault analysis model, and when the accuracy of the pre-trained fault analysis model is not less than the preset accuracy, adjust the parameters of the first fault analysis instruction and the pre-trained fault analysis model based on the second sample log and the second labeled fault analysis result to obtain the second fault analysis instruction and the fault analysis model, so that the obtained second fault analysis instruction and the fault analysis model can perform fault analysis more accurately and no longer rely on manual operation, thereby improving the efficiency of fault analysis. In addition, the labeled fault analysis result includes at least one of the following: playback error scenario, error cause, solution, and error scenario category, so that the trained fault analysis model can accurately restore the error scenario to provide an accurate solution.

[0107] In S101, the initial fault analysis model obtained may be an LLMs (Large Language Models) constructed by initializing model parameters, or may be one selected from multiple open source LLMs. Exemplarily, the initial fault analysis model obtained may be a GPT4-Turbo-128K, GPT4o, or Wenxinyiyan 4.0 model.

[0108] After determining the initial fault analysis model, determine one of the key parameters required for model training - the first fault analysis instruction, which can be a prompt. The prompt can contain task-related information and context to help the model better understand the requirements and generate correct outputs. These prompts can be a question, a description, or a complete sentence, etc., to guide the model to generate relevant answers. By designing effective prompts, the content generated by the model can be controlled to make it more in line with specific needs. Exemplarily, the first fault analysis instruction can be, for example, generating fault analysis results based on feedback logs.

[0109] In S102, a historical time period may be obtained, such as the time in the past week, month, or year when a user encountered a video playback error while using a player to play a video and reported the fault problem, and a log corresponding to the fault problem may be used as a first sample log, which is a log generated when a video playback error occurs, and a fault analysis result for the fault problem may be used as a corresponding first labeled fault analysis result. Exemplarily, the first labeled fault analysis result may include the following information:

[0110] Cause of error (fault location): The DASH (an input / output interface, such as a video cloud on-demand interface) interface has timed out multiple times, resulting in playback failure;

[0111] Error scenario category: network request error;

[0112] Playback error scenario: When trying to play a video, the user encounters a DASH interface connection timeout problem, resulting in the inability to obtain the video playback address. This problem may be caused by unstable network connection or server response timeout. According to the fault location information, the user's network request still fails after trying various fallback strategies, indicating that it may be a problem with the user's network environment or the server.

[0113] Solution: It is recommended to check the user's network connection and try to reconnect. If the problem persists, it may be necessary to contact the service provider for further diagnosis.

[0114] In S103, the first sample log is used as input, and the first fault analysis instruction is used as a pre-training task to instruct the initial fault analysis model to output a first fault analysis result, which is the fault analysis result obtained by the initial fault analysis model performing fault analysis on the first sample log under the instruction of the first fault analysis instruction. Further, in S104, the first loss between the first fault analysis result obtained by the initial fault analysis model performing fault analysis on the first sample log under the instruction of the first fault analysis instruction and the first labeled fault analysis result is calculated, and then the parameters of the initial fault analysis model are adjusted according to the first loss until the first preset end condition is met to obtain the pre-trained fault analysis model. Exemplarily, a single first sample log can be input into the initial fault analysis model, and the initial fault analysis model performs fault analysis on the first sample log under the instruction of the first fault analysis instruction to obtain the first fault analysis result.

[0115] The first preset termination condition may be a preset number of iterations, or the loss reaching a preset loss threshold, etc.

[0116] In S105, the obtained training sample knowledge base including the second sample log and the second labeled fault analysis result corresponding to the second sample log can be a standard knowledge base of the second labeled fault analysis result obtained by accurately analyzing the second sample log based on historical real video playback error cases and reviewed by experts in the field. Based on the knowledge base, the pre-trained fault analysis model is verified and the parameters of the pre-trained fault analysis model are further adjusted to obtain a trained fault analysis model, so that the trained fault analysis model can perform fault analysis more accurately. In an example, the second sample log is different from the first sample log.

[0117] In S106, the second sample log can be used as the input of the pre-trained fault analysis model, and the output of the pre-trained fault analysis model can be repeatedly verified to be consistent with the second labeled fault analysis result corresponding to the second sample log, and the hit rate of the pre-trained fault analysis model can be determined to verify the accuracy of the pre-trained fault analysis model. In S107, when the accuracy is not less than the preset accuracy, the second sample log in the training sample knowledge base and the second labeled fault analysis result corresponding to the second sample log are further used as context to continue training the pre-trained fault analysis model, adjust the first fault analysis instruction and the parameters of the pre-trained fault analysis model, and obtain the second fault analysis instruction and the fault analysis model. Exemplarily, the preset accuracy can be set according to the actual situation, such as 90%, 95% or 98%, etc.

[0118] In a possible implementation, when the accuracy is lower than the preset accuracy, the process may return to the step of obtaining the initial fault analysis model in S101 to retrain the pre-trained fault analysis model.

[0119] When it is verified that the accuracy of the pre-trained fault analysis model is less than the preset accuracy, the pre-trained fault analysis model is retrained so that the accuracy of the trained pre-trained fault analysis model is relatively high, so that the fault analysis model can be trained more quickly in the subsequent process.

[0120] In a possible implementation, Figure 2 As shown, the implementation process of obtaining the initial fault analysis model in S101 may include:

[0121] S201, obtaining a third sample log and a third annotated fault analysis result corresponding to the third sample log.

[0122] The implementation process of step S201 may refer to the implementation process of step S102, and the embodiment of the present invention will not be repeated here. In one example, the third sample log may be the same as or different from the first sample log or the second sample log.

[0123] S202, calling a plurality of preset large language models through a preset API, inputting the third sample log into each of the large language models respectively, and obtaining a third fault analysis result output by each of the large language models.

[0124] The training of the fault analysis model may include front-end and back-end implementations. The front-end and back-end may correspond to different threads. The training of the pre-trained fault analysis model is implemented on the front-end, and the fine-tuning training of the pre-trained fault analysis model is implemented on the back-end to obtain the fault analysis model.

[0125] Exemplarily, the front end is an online convenient debugging platform, on which different pre-set multiple large language models can be selected through a pre-set API (Application Programming Interface). The large language model can be an untrained original open source model or a pre-trained open source model. The open source model can be, for example, GPT4-Turbo-128K, GPT4o, or Wenxin Yiyan 4.0 model.

[0126] Each language model is called respectively, and the third sample log is input into each language model respectively to obtain the third fault analysis result output by each language model to verify the processing capability of each language model.

[0127] S203: Selecting an initial fault analysis model from a plurality of large language models based on the differences between each third fault analysis result and the third annotated fault analysis result.

[0128] The difference between the third fault analysis result and the third labeled fault analysis result output by each large language model is calculated respectively. The difference, such as semantic similarity, is calculated by cosine similarity, Euclidean distance, etc., so as to select the large language model with the smallest difference between the third fault analysis result and the third labeled fault analysis result from multiple large language models as the initial fault analysis model.

[0129] Exemplarily, when there are multiple third sample logs, the weighted sum, mean or sum difference between the third fault analysis results output by each language model and the corresponding third annotated fault analysis results may be compared.

[0130] An initial fault analysis model is selected through the third sample log and the third annotated fault analysis result corresponding to the third sample log, so that the selected initial fault analysis model is more conducive to the training of the fault analysis model, thereby improving the efficiency of the fault analysis model training.

[0131] In a possible implementation, Figure 3 As shown, the implementation process of the above S105 of acquiring the training sample knowledge base may include:

[0132] S301, obtaining sample feedback information when an error occurs when a user plays a video within a preset historical time period.

[0133] The preset historical time period may be half a year, one year, or five years before the current time, etc. Exemplarily, sample feedback information of users playing videos with errors within five years before the current time is obtained, and the sample feedback information may be automatically obtained through the customer service system.

[0134] S302: Determine the type of each sample feedback information according to the feedback content of the sample feedback information.

[0135] The sample feedback information obtained may include: at least one of the user's IP (Internet Protocol Address), UID (User login Identification), device ID (Identification) and feedback ID, as well as feedback time, feedback content, etc.

[0136] Among them, the feedback content can be an error identifier, such as an error code, or a specific fault content when no error code is defined. Furthermore, the type of each sample feedback information is determined according to the feedback content. Exemplarily, the type of sample feedback information may include: the error code of network errors starts with *-1, the error code of video cloud on-demand errors starts with 1-3-A, the error code of signature verification failure is A00001, the error code of member playback authentication error starts with 3-3-Q, the error code of the feature CDN (Content Delivery Network) scheduling error starts with 5-5-3*, the error code of HCDN (Hybrid Content Delivery Network) loading error starts with 7-0, the error code of video cloud on-demand aggregation service business starts with 16-*, playback kernel error, unknown and APP (APPlication) front-end error, etc.

[0137] S303: Obtain the second sample log corresponding to each sample feedback information of each type.

[0138] For each type, obtain the second sample log corresponding to each sample feedback information under the type. Exemplarily, for each sample feedback information in each type, the Hive database can be queried according to the IP address and feedback time included in the sample feedback information to obtain the server log and Pingback (automatic reference notification, or delivery) log corresponding to the sample feedback information under the type; query the Compass (user local) log according to the IP address, UID, device ID, feedback ID and feedback time to obtain the local playback log corresponding to the sample feedback information under the type; and obtain the system log corresponding to the sample feedback information under the type according to the IP address, UID, device ID, feedback ID and feedback time. Then, the server log, Pingback log, local playback log, and system log of each sample feedback information under each type are deduplicated, and the format is unified to obtain the second sample log corresponding to each sample feedback information under each type. Among them, the server log can include: commercial CDN log, self-built CDN log, DCache (Distributed Cache) log, static CDN log, scheduling log, video cloud log, etc.

[0139] S304: Determine a second annotated fault analysis result corresponding to each second sample log.

[0140] For each second sample log, a historical fault analysis result corresponding to the second sample log is obtained to obtain a second annotated fault analysis result, which may be an accurate fault analysis result after manual review and verification by domain experts.

[0141] S305: Building a training sample knowledge base based on each second sample log and each second labeled fault analysis result.

[0142] By utilizing real historical video playback error cases, and determining the second labeled fault analysis results corresponding to each second sample log one by one, the second labeled fault analysis results are accurate fault analysis results after manual review and review by domain experts, so that the training sample knowledge base constructed based on each second sample log and each second labeled fault analysis result is more accurate, and thus the fault analysis model obtained by training with the constructed training sample knowledge base as context can perform fault analysis more accurately, reducing the risk of misleading users due to the inaccurate training sample knowledge base causing the trained fault analysis model to fail to accurately analyze the fault, and further reducing the risk of repeated user complaints and loss of potential customers.

[0143] In a possible implementation manner, the training sample knowledge base may also be updated using the fourth sample log and the fourth labeled fault analysis result corresponding to the fourth sample log.

[0144] The fourth sample log is different from the second sample log, that is, the fourth sample log and the fourth labeled fault analysis result corresponding to the fourth sample log can be used to continuously update the training sample knowledge base, so that the training of the fault analysis model is more accurate and complete, while ensuring the continuity and effectiveness of analyzing faults using the trained fault analysis model.

[0145] In a possible implementation, the implementation process of the above step S106 of verifying the accuracy of the pre-trained fault analysis model using the second sample log and the second annotated fault analysis result may include:

[0146] Step 1: input the second sample log into the pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a second fault analysis result based on the first fault analysis instruction;

[0147] Step 2: determine whether the second fault analysis result is consistent with the second marked fault analysis result, and obtain a judgment result;

[0148] Step 3: Based on the judgment result, determine the accuracy of the pre-trained fault analysis model.

[0149] The second fault analysis result is the result obtained by the pre-trained fault analysis model for fault analysis of the second sample log under the prompt of the first fault analysis instruction. If the second fault analysis result is consistent with the second labeled fault analysis result, it means that the fault analysis result of the pre-trained fault analysis model for the second sample log is accurate. If the second fault analysis result is inconsistent with the second labeled fault analysis result, it means that the fault analysis result of the pre-trained fault analysis model for the second sample log is inaccurate.

[0150] Exemplarily, multiple second sample logs can be input into the pre-trained fault analysis model one by one, so that the pre-trained fault analysis model outputs corresponding second fault analysis results based on the first fault analysis instruction, and then determines whether each second fault analysis result is consistent with the corresponding second labeled fault analysis result through semantic similarity judgment, determines the proportion of consistent judgment results in all judgment results, and obtains the accuracy of the pre-trained fault analysis model.

[0151] The second sample log and the second labeled fault analysis result are used to verify the accuracy of the pre-trained fault analysis model, and then the subsequent parameter adjustment training of the pre-trained fault analysis model is performed when the accuracy meets the conditions, so as to obtain the fault analysis model more quickly.

[0152] In a possible implementation, Figure 4 As shown, the above S107 adjusts the parameters of the first fault analysis instruction and the pre-trained fault analysis model based on the second sample log and the second labeled fault analysis result to obtain the second fault analysis instruction and the fault analysis model. The implementation process may include:

[0153] S401, obtaining a plurality of fifth sample logs and fifth annotated fault analysis results corresponding to the plurality of fifth sample logs.

[0154] The implementation process of step S401 may refer to the implementation process of the above-mentioned step S102, and the embodiment of the present invention will not be repeated here.

[0155] The parameter training of the pre-trained fault analysis model can be implemented by the backend. The backend calls the pre-trained fault analysis model through a preset API, and uses a Python program to adjust the parameters of the pre-trained fault analysis model. The adjusted parameters include but are not limited to: the first fault analysis instruction, the maximum amount of access per time, the output method (such as streaming output), the response method (such as synchronous or asynchronous), the output format, etc. In this process, the second sample log and the second labeled fault analysis result are used as context for training, so the fifth sample log is a log different from the second sample log. Multiple fifth sample logs can be batch sample logs, and the batch fifth sample log corresponds to the fifth labeled fault analysis result, and the fifth labeled fault analysis result contains the labeled fault analysis results corresponding to each fifth sample log.

[0156] The specific parameter adjustment can be implemented by the following steps S402-S404, and steps S402-S404 can be performed in parallel.

[0157] S402: Input a plurality of fifth sample logs into a pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a fifth fault analysis result based on the first fault analysis instruction, and calculates a second loss between the fifth fault analysis result and the fifth labeled fault analysis result.

[0158] The batch of fifth sample logs is input into the pre-trained fault analysis model at one time, so that the pre-trained fault analysis model automatically outputs the fifth fault analysis result based on the first fault analysis instruction, and then the second loss between the fifth fault analysis result and the fifth labeled fault analysis result is calculated.

[0159] When a batch of fifth sample logs are input into a pre-trained fault analysis model and the pre-trained fault analysis model automatically outputs a fifth fault analysis result, the fifth fault analysis result and the fifth labeled fault analysis result include at least one of the following: a playback error scenario, an error cause, a solution, an error scenario category, and fault analysis statistical information; wherein the fault analysis statistical information is determined based on the playback error scenario, the error cause, or the error scenario category.

[0160] Exemplarily, the fault analysis statistical information may include the following information:

[0161] Main error codes:

[0162] 1-1-510-0: 62.34%; TCP (Transmission Control Protocol) handshake failure during the video cloud on-demand interface request triggered a connection timeout (UV (Unique Visitor) accounted for the largest proportion);

[0163] 1-1-513-0: 5.19%; TCP handshake succeeded during the video cloud on-demand interface request, but no data was received from the server (UV accounted for the second largest proportion);

[0164] 1-1-7-0: 3.25%, failed to connect to the server during the video cloud on-demand interface request (UV accounts for the third largest proportion);

[0165] 1-1-511-0: 3.25%, the handshake was successful during the video cloud on-demand interface request, but the download data timed out (UV accounted for the fourth place);

[0166] 11-2-3: 1.95%; local loading data is illegal (UV ranks fifth).

[0167] Main fault location (error cause):

[0168] DASH interface failure: 52.6% (UV ranks first);

[0169] Scheduling failure: 6.94% (UV ranks second);

[0170] CDN failure: 6.36% (UV accounts for the third largest proportion);

[0171] Scheduling interface failure: 2.89% (UV ranks fourth);

[0172] CDN slow speed: 2.31%, (UV ranks fifth). …….

[0173] S403: Using the second sample log and the second annotated fault analysis result as context, querying the sixth fault analysis results corresponding to the plurality of fifth sample logs, and calculating the third loss between the sixth fault analysis result and the fifth annotated fault analysis result.

[0174] Taking the second sample log and the second labeled fault analysis result as the context, query the sixth fault analysis result corresponding to the multiple fifth sample logs, that is, take the training sample knowledge base as the dictionary, obtain the sixth fault analysis result corresponding to the multiple fifth sample logs by querying the dictionary, and then calculate the third loss between the sixth fault analysis result and the fifth labeled fault analysis result.

[0175] S404: Input multiple fifth sample logs into the pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a seventh fault analysis result based on the received user interaction information, and obtains an interaction instruction for the seventh fault analysis result.

[0176] Adjust the parameters of the model by manual fine-tuning. For example, Figure 5 As shown, the number of interactions and interaction questions can be set and input into the pre-trained fault analysis model, and the interaction questions can be input into the pre-trained fault analysis model as context, and multiple fifth sample logs can be input into the pre-trained fault analysis model, so that the pre-trained fault analysis model outputs the seventh fault analysis result based on the received user interaction information (i.e., the interaction question), and further obtains the user's interaction instruction for the seventh fault analysis result output by the pre-trained fault analysis model, and the interaction instruction can be to stop the interaction, or the user continues to input the interaction information, returns to re-input the number of interactions and the interaction question into the pre-trained fault analysis model until the interaction stops.

[0177] S405, based on the second loss, the third loss and the interactive instruction, adjusting the parameters of the pre-trained fault analysis model and the first fault analysis instruction to obtain a second fault analysis instruction and a fault analysis model.

[0178] In a possible implementation, when the second loss does not meet the preset condition, the third loss does not meet the preset condition, and the interaction instruction is not to stop the interaction, the step of obtaining the initial fault analysis model is returned to adjust the parameters of the pre-trained fault analysis model and the first fault analysis instruction; when both the second loss and the third loss meet the preset condition, and the interaction instruction is to stop the interaction, the current first fault analysis instruction is determined as the second fault analysis instruction and the current parameters of the pre-trained fault analysis model are determined as the parameters of the fault analysis model. An exemplary preset condition may be a preset loss threshold.

[0179] In an embodiment of the present invention, the trained fault analysis model is made more accurate by adjusting the parameters of the pre-trained fault analysis model and the first fault analysis instruction through batch input and batch output, using the second sample log and the second labeled fault analysis result as context query, and manual fine-tuning.

[0180] For example, Figure 6 As shown, the embodiment of the present invention realizes the training of the fault analysis model by combining the front-end and the back-end. Different large language models are selected at the front-end through a pre-set API. The large language model may be, for example, GPT4-Turbo-128K, GPT4o, or Wenxinyiyan4.0. After the initial fault analysis model is selected, the model is fine-tuned based on the first fault analysis instruction using the first sample log and its corresponding first annotated fault analysis result, and the model obtained by the model fine-tuning training is verified in combination with the training sample knowledge base. If the verification passes, a pre-trained fault analysis model is obtained.

[0181] In the backend, the pre-trained fault analysis model can also be called through the preset API, and then the parameters of the pre-trained fault analysis model can be adjusted using the Python program. Specifically, the training sample knowledge base is used as the context training model, and the model parameters are adjusted by batch input and batch output, the second sample log and the second labeled fault analysis result in the training sample knowledge base are used as context queries, and manual fine-tuning is used to adjust the model parameters to determine whether the model output is consistent with the labeled value (this process is judged by the calculated loss and the received interactive instructions). If they are consistent, it means that the model training is completed. The model parameters (including fault analysis instructions and the maximum amount of each access, output method, response method, output format, etc.) and the model version are saved here to obtain the trained fault analysis model. If they are inconsistent, it means that the model training has not converged yet. At this time, the front end is returned to re-select the large language model and continue the model training process until the trained fault analysis model is obtained. After obtaining the trained fault analysis model, a manual secondary review can also be performed to determine whether the trained fault analysis model is accurate. If it is inaccurate, the model can be trained again until the final output result of the fault analysis model can accurately match the manual analysis result.

[0182] When a fault analysis model is trained in the embodiment of the present invention, not only can the fault analysis model be used to analyze in real time the fault problems encountered by users when playing videos using a player and reported, so as to locate, analyze and solve the problems reported by users in the first place; fault analysis can also be performed periodically on the problems reported by users, such as monitoring the problems reported by users in the past week, month or year, so as to intelligently analyze the rules of fault location results and periodically discover potential deficiencies of the player, so as to better improve the video player product and enhance the user experience.

[0183] Because the feedback time for users to report video playback errors during the process of using the player to play videos is all day, even if there is 24-hour online customer service, the online customer service personnel are only an intermediate bridge for problem communication rather than solving the problem. The real solution to the user feedback problem is professionals from different business parties, and it is difficult for professionals from different business parties to remain online 24 hours a day. In addition, if the user feedback problem involves multiple businesses, professionals from different business parties are required to investigate and locate it, and the cycle process is long. The embodiment of the present invention uses a fault analysis model to analyze the fault problem that users encounter and report when playing videos in the process of using the player to play videos in real time, without the need for manual online customer service to solve the user feedback problem online 24 hours a day, and the real-time processing of the problem is high.

[0184] The fault analysis method provided by the embodiment of the present invention is introduced in detail below. The fault analysis method can be applied to a customer service system.

[0185] like Figure 7 As shown, Figure 7 1 is a flow chart of a fault analysis method in an embodiment of the present invention. The method comprises:

[0186] S701, obtaining target feedback information to be analyzed and a target log corresponding to the target feedback information.

[0187] The target feedback information to be analyzed may be information about a fault problem encountered by a user when playing a video using a player and reported in real time, or information about a fault problem reported by a user within a preset time period (e.g., one day, one week, or one month). In one example, the target feedback information may be automatically obtained through a customer service system.

[0188] Further, the corresponding target log is obtained according to the target feedback information. Specifically, the target log can be obtained by querying the corresponding log through the identifier contained in the target feedback information. The detailed acquisition process can be found in the following description.

[0189] S702, obtaining a target fault analysis instruction.

[0190] The target fault analysis instruction is obtained by using the above fault analysis model training method, that is, the second fault analysis instruction. The target fault analysis instruction can be, for example, generating a fault analysis result based on a log.

[0191] S703, inputting the target log into a pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a target fault analysis result based on the target fault analysis instruction.

[0192] Among them, the target fault analysis result includes at least one of the following: playback error scenario, error cause, solution and error scenario category; the pre-trained fault analysis model is trained using the above-mentioned fault analysis model training method.

[0193] The fault analysis method provided by the embodiment of the present invention is applied. Since the pre-trained fault analysis model is obtained by fine-tuning the first sample log and its corresponding first labeled fault analysis result, and the first fault analysis instruction, the pre-trained fault analysis model is then verified by using the second sample log and its corresponding second labeled fault analysis result included in the training sample knowledge base, and when the verification is passed, the first fault analysis instruction and the parameters of the pre-trained fault analysis model are adjusted based on the second sample log and the second labeled fault analysis result, so that the trained fault analysis model can be used to perform fault analysis more accurately, and no longer rely on manual operation, thereby improving the efficiency of fault analysis, and significantly reducing the cost of manual service, and improving the long-term return on investment. In addition, it can not only analyze the fault problems reported by users in real time, improve the response rate to user feedback problems, and thus improve user satisfaction and loyalty, but also periodically analyze the fault problems reported by historical users, so as to better improve the video player product and improve the user experience. In addition, the target fault analysis result includes at least one of the following: playback error scene, error cause, solution and error scene category, which realizes the analysis of complex problems, so that the fault analysis can accurately restore the error scene and provide an accurate solution.

[0194] In a possible implementation, the target feedback information may include: at least one of the target user's IP address, UID, device ID, and feedback ID, and feedback time. When the target feedback information is feedback information corresponding to the fault problem fed back by the user in real time, correspondingly, Figure 8 As shown, the implementation process of obtaining the target log corresponding to the target feedback information in the above step S701 includes:

[0195] S801, based on the IP address and feedback time, query the Hive database to obtain the first server log and the first Pingback log corresponding to the target feedback information.

[0196] According to the IP address of the target feedback information, query the log corresponding to the feedback time in the Hive database to obtain the first server log and the first Pingback log corresponding to the target feedback information. Among them, the first server log may include: commercial CDN log, self-built CDN log, D Cache log, static CDN log, scheduling log, video cloud log, etc. Static CDN logs can be static resource information such as pictures. The first Pingback log represents information related to the delivery process of the target feedback information.

[0197] Furthermore, the obtained first server log and first pingback log are deduplicated and dirty data is removed, for example, the contents other than those contained in the first server log are deleted.

[0198] The target feedback information may also include feedback content. The feedback content may be an error identifier, such as an error code, or specific fault content when no error code is defined. Furthermore, the type of the target feedback information may be determined based on the feedback content. Exemplarily, the types of the target feedback information may include: playback error, playback function, audio and video problems, and playback freezes, etc.

[0199] S802, based on the IP address, UID, device ID, feedback ID and feedback time, query the Compass log to obtain a first local playback log corresponding to the target feedback information.

[0200] According to the IP address, UID, device ID and feedback ID of the target feedback information, the log corresponding to the feedback time in the Compass log is queried to obtain the first local playback log corresponding to the target feedback information. The feedback ID is used to uniquely identify the target feedback information.

[0201] S803, based on the IP address, UID, device ID, feedback ID and feedback time, obtain a first system log corresponding to the target feedback information.

[0202] According to the IP address, UID, device ID and feedback ID of the target feedback information, the log corresponding to the feedback time in the system log is queried to obtain the first system log corresponding to the target feedback information.

[0203] S804: Aggregate the first local playback log and the first system log to obtain a first local log.

[0204] In a possible implementation, the first local playback log and the first system log may be merged to obtain a log to be processed, and then the log to be processed may be processed according to a preset network request format, and logs of a target type may be filtered and retained to obtain a first local log.

[0205] The preset network request may be set according to actual network requirements, for example, it may be a QTP (Quic Transfer Protocol) network request, or an HTTP (Hypertext Transfer Protocol) network request, and the like.

[0206] Exemplarily, the first local playback log and the first system log are de-duplicated and merged, and the union is taken to obtain the log to be processed, which is then processed in the format requested by QTP, and logs of types such as PUMA (a player kernel) start-up, freeze, and bit rate switching are filtered and retained to obtain the first local log.

[0207] S805: Format the first server log, the first pingback log, and the first local log to obtain a target log.

[0208] Exemplarily, the obtained first server log, first Pingback log and first local log may contain garbled characters or inconsistent formats, and the first server log, the first Pingback log and the first local log can be formatted according to a preset database format to be processed into a table to obtain the target log.

[0209] In an embodiment of the present invention, the first server log, the first Pingback log and the first local log associated with the target feedback information are obtained respectively, and then aggregated and filtered to obtain the target log. The target log is an associated log of the target feedback information rather than a single log, so that the fault analysis result obtained based on the target log can more accurately locate the problem and accurately feedback the solution.

[0210] For example, for a fault problem reported by a user in real time, the target feedback information corresponding to a user includes, for example: IP address (139.XXX.XX.XXX), UID (464882XXXXXXXXXX), feedback content (error code 1-1-510-0), feedback time (2024-07-16 00:20:20). The fault analysis results obtained using the pre-trained fault analysis model include the following information:

[0211] Cause of error (fault location): When the user tried to play the video, the connection to the DASH interface failed multiple times, resulting in playback interruption;

[0212] Error scenario category: network request error;

[0213] Playback error scenario: The user encountered network connection problems when trying to play the video, especially the connection to the DASH interface. This interface is responsible for providing key data for video playback, but the connection timed out due to a TCP handshake failure (error code 1-1-510-0). Although the user's device tried multiple network fallback strategies, including local DNS resolution, HTTPDNS, proxy servers, etc., it still could not successfully establish a connection. This indicates that the user may be facing network stability issues or limited accessibility of the DASH interface service.

[0214] Solution: It is recommended to check the user's network connection and try to reconnect. If the problem persists, it may be necessary to contact the service provider for further diagnosis.

[0215] In a possible implementation, the above target feedback information is multiple, and the multiple may also be in batches (such as hundreds or thousands). For example, the target feedback information may be log information corresponding to the user feedback fault problem within a preset historical time period (such as one day, the last week, one month or one year). Correspondingly, Fig. 9 As shown, the implementation process of obtaining the target log corresponding to the target feedback information in the above step S701 includes:

[0216] S901: Determine the type of each target feedback information based on the feedback content of each target feedback information.

[0217] The feedback content can be an error code, or specific fault content if no error code is defined. The type of each target feedback information can be determined based on the feedback content. Exemplarily, the type of target feedback information may include: the error code for network errors starts with *-1, the error code for video cloud on-demand errors starts with 1-3-A, the error code for signature verification failure is A00001, the error code for member playback authentication errors starts with 3-3-Q, the error code for feature CDN (Content Delivery Network) scheduling errors starts with 5-5-3*, the error code for HCDN (Hybrid Content Delivery Network) loading errors starts with 7-0, the error code for video cloud on-demand aggregation service starts with 16-*, playback kernel errors, unknown and APP (APPlication) front-end errors, etc.

[0218] S902: Based on the IP address and the feedback time, query the Hive database to obtain the second server log and the second Pingback log corresponding to each target feedback information of each type.

[0219] For each target feedback information of each type, according to the IP address of the target feedback information, query the log corresponding to the feedback time of the target feedback information in the Hive database to obtain the second server log and the second Pingback log corresponding to the target feedback information.

[0220] Furthermore, the first server log and the first Pingback log of each target feedback information of each type obtained are deduplicated and dirty data removed.

[0221] S903: Based on the IP address, UID, device ID, feedback ID, and feedback time, query the Compass log to obtain the second local playback log corresponding to each target feedback information under each type.

[0222] For each target feedback information of each type, query the log corresponding to the feedback time of the target feedback information in the Compass log according to the IP address, UID, device ID and feedback ID of the target feedback information to obtain the second local playback log corresponding to the target feedback information.

[0223] S904, based on the IP address, UID, device ID, feedback ID and feedback time, obtain the second system log corresponding to each target feedback information under each type.

[0224] For each target feedback information of each type, according to the IP address, UID, device ID and feedback ID of the target feedback information, query the log corresponding to the feedback time of the target feedback information in the system log to obtain the second system log corresponding to the target feedback information.

[0225] S905 , aggregate the second local playback log corresponding to each target feedback information under each type with the second system log to obtain a second local log.

[0226] Exemplarily, for each type of target feedback information, the second local playback log of the target feedback information is de-duplicated and merged with the second system log, and the union is taken to obtain the log to be processed, and then the log to be processed is processed according to the format of the QTP request, and the logs of types such as PUMA start-up, freeze, and bit rate switching are filtered and retained to obtain the second local log.

[0227] S906, formatting the second server log, the second Pingback log, and the second local log of each target feedback information of each type in time sequence to obtain a target log.

[0228] In one implementation mode, the second server log, the second Pingback log, and the second local log of each target feedback information of each type may be formatted first, and then sorted in time sequence to obtain the target log.

[0229] For example, the second server log, second Pingback log, and second local log of each target feedback information obtained may contain garbled characters or inconsistent formats, and the second server log, second Pingback log, and second local log of each target feedback information may be formatted according to a preset database format to be processed into a log in a tabular form. Then, for each type, the target feedback information of that type is arranged in time sequence to obtain a time series data set, i.e., the target log.

[0230] In an embodiment of the present invention, for a situation of multiple target feedback information, the target feedback information is first classified, and then the second server log, the second Pingback log, and the second local log associated with each target feedback information of each type are respectively obtained, and the obtained associated logs are aggregated, filtered, and time-series processed to obtain a target log. The target log connects different logs in series, so that the fault analysis results obtained based on the target log can locate the problem more accurately, and accurately feedback the solution, which is convenient for discovering problems that are ignored for a single feedback.

[0231] In a possible implementation, when there are multiple target feedback information, user feedback issues can be processed in batches. Accordingly, the target fault analysis results output by the fault analysis model can also include fault analysis statistical information. The fault analysis statistical information can be determined based on the playback error scenario, error cause or error scenario category, feedback content, etc., including at least one of the following: fault information counted according to feedback content, fault information counted according to error cause, and fault information counted according to the server associated with the target feedback information.

[0232] Exemplarily, the fault analysis statistical information may include the following information:

[0233] 1. Log analysis grouped by feedback content:

[0234] Main error codes:

[0235] 1-1-510-0: accounts for 62.34%; TCP handshake failure triggers connection timeout during the video cloud on-demand interface request (UV accounts for the largest proportion);…

[0236] 2. Log analysis by fault location group:

[0237] Main fault location:

[0238] DASH interface failure: accounting for 52.6%, (UV accounts for the largest proportion);...

[0239] 3. Log analysis grouped by associated servers…

[0240] The embodiments of the present invention can process user feedback problems in batches and intelligently output fault analysis statistical information, thereby improving the efficiency of fault analysis and being able to quickly cope with the processing of large amounts of data.

[0241] In a possible implementation, when there is fault information satisfying an alarm condition in the fault analysis statistical information, an alarm is triggered.

[0242] For example, when the proportion of a certain fault problem in the fault analysis statistical information is greater than a set threshold, an alarm is triggered, and the business party is notified in time to handle it, and the problem is checked and solved as soon as possible, so that the fault problem can be quickly resolved.

[0243] Corresponding to the above method embodiments, the embodiments of the present invention also provide corresponding device embodiments.

[0244] like Fig.10 As shown, Fig.10 Schematic diagram of a structure of a fault analysis model training device in an embodiment of the present invention. The fault analysis model training device in an embodiment of the present invention comprises:

[0245] A first acquisition module 1001 is used to acquire an initial fault analysis model;

[0246] The second acquisition module 1002 is used to obtain a first sample log and a first annotated fault analysis result corresponding to the first sample log; the first sample log is a log corresponding to a video playback error; the first annotated fault analysis result includes at least one of the following: a playback error scenario, an error cause, a solution, and an error scenario category;

[0247] The analysis and prediction module 1003 is used to input the first sample log into the initial fault analysis model, so that the initial fault analysis model outputs a first fault analysis result based on the first fault analysis instruction;

[0248] A first training module 1004 is used to calculate a first loss between the first fault analysis result and the first labeled fault analysis result, and adjust the parameters of the initial fault analysis model according to the first loss to obtain a pre-trained fault analysis model;

[0249] The third acquisition module 1005 is used to acquire a training sample knowledge base; the training sample knowledge base includes the second sample log and the second annotated fault analysis result corresponding to the second sample log;

[0250] A verification module 1006, used to verify the accuracy of the pre-trained fault analysis model using the second sample log and the second annotated fault analysis result;

[0251] The second training module 1007 is used to adjust the parameters of the first fault analysis instruction and the pre-trained fault analysis model based on the second sample log and the second labeled fault analysis result when the accuracy rate is not less than the preset accuracy rate, so as to obtain the second fault analysis instruction and the fault analysis model.

[0252] Optionally, the above device further includes:

[0253] The trigger module is used to trigger the first acquisition module 1001 to execute acquisition of the initial fault analysis model when the accuracy rate is less than a preset accuracy rate.

[0254] Optionally, the first acquisition module 1001 is specifically used to:

[0255] Obtain a third sample log and a third annotated fault analysis result corresponding to the third sample log;

[0256] Calling a plurality of preset large language models through a preset application programming interface API, inputting the third sample log into each of the large language models respectively, and obtaining a third fault analysis result output by each of the large language models;

[0257] Based on the difference between each third fault analysis result and the third annotated fault analysis result, an initial fault analysis model is selected from a plurality of large language models.

[0258] Optionally, the third acquisition module 1005 is specifically used to:

[0259] Obtain sample feedback information when a user makes an error while playing a video within a preset historical time period;

[0260] Determine the type of each sample feedback information according to the feedback content of the sample feedback information;

[0261] Obtain the second sample log corresponding to each sample feedback information under each type;

[0262] Determine a second annotated fault analysis result corresponding to each second sample log;

[0263] Based on each second sample log and each second labeled fault analysis result, a training sample knowledge base is constructed.

[0264] Optionally, the above device further includes:

[0265] The updating module is used to update the training sample knowledge base by using the fourth sample log and the fourth labeled fault analysis result corresponding to the fourth sample log.

[0266] Optionally, the verification module 1006 is specifically used to:

[0267] Inputting the second sample log into the pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a second fault analysis result based on the first fault analysis instruction;

[0268] Determine whether the second fault analysis result is consistent with the second marked fault analysis result to obtain a judgment result;

[0269] Based on the judgment results, the accuracy of the pre-trained fault analysis model is determined.

[0270] Optionally, the second training module 1007 is specifically used to:

[0271] Obtaining a plurality of fifth sample logs and fifth annotated fault analysis results corresponding to the plurality of fifth sample logs;

[0272] Inputting a plurality of fifth sample logs into the pre-trained fault analysis model, causing the pre-trained fault analysis model to output a fifth fault analysis result based on the first fault analysis instruction, and calculating a second loss between the fifth fault analysis result and the fifth labeled fault analysis result;

[0273] Using the second sample log and the second annotated fault analysis result as context, querying the sixth fault analysis results corresponding to the plurality of fifth sample logs, and calculating the third loss between the sixth fault analysis result and the fifth annotated fault analysis result;

[0274] Inputting a plurality of fifth sample logs into a pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a seventh fault analysis result based on the received user interaction information, and obtaining an interaction instruction for the seventh fault analysis result;

[0275] Based on the second loss, the third loss and the interactive instruction, the parameters of the pre-trained fault analysis model and the first fault analysis instruction are adjusted to obtain the second fault analysis instruction and the fault analysis model.

[0276] Optionally, the above-mentioned adjusting the parameters of the pre-trained fault analysis model and the first fault analysis instruction based on the second loss, the third loss and the interactive instruction to obtain the second fault analysis instruction and the fault analysis model includes:

[0277] When the second loss does not meet the preset condition, the third loss does not meet the preset condition, and the interaction instruction is not to stop the interaction, the first acquisition module 1001 is triggered to execute acquisition of the initial fault analysis model;

[0278] When both the second loss and the third loss meet the preset conditions and the interaction instruction is to stop the interaction, the current first fault analysis instruction is determined as the second fault analysis instruction and the current parameters of the pre-trained fault analysis model are determined as the parameters of the fault analysis model.

[0279] Optionally, the fifth annotated fault analysis result includes at least one of the following: playback error scenario, error cause, solution, error scenario category and fault analysis statistical information; the fault analysis statistical information is determined based on the playback error scenario, error cause or error scenario category.

[0280] like Fig.11 As shown, Fig.11 Schematic diagram of a structure of a fault analysis device in an embodiment of the present invention. The fault analysis device in an embodiment of the present invention comprises:

[0281] The fourth acquisition module 1101 is used to acquire the target feedback information to be analyzed and the target log corresponding to the target feedback information;

[0282] The fifth acquisition module 1102 is used to acquire a target fault analysis instruction; the target fault analysis instruction is obtained by using the above-mentioned fault analysis model training method;

[0283] The fault analysis module 1103 is used to input the target log into a pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a target fault analysis result based on the target fault analysis instruction; wherein the target fault analysis result includes at least one of the following: playback error scenario, error cause, solution and error scenario category; the pre-trained fault analysis model is trained using the above-mentioned fault analysis model training method.

[0284] Optionally, the target feedback information includes: at least one of the target user's Internet Protocol address IP address, user login identifier UID, device identifier ID and feedback ID, and feedback time; the fourth acquisition module 1101 is specifically used to:

[0285] Based on the IP address and feedback time, query the Hive database to obtain the first server log and the first automatic reference notification Pingback log corresponding to the target feedback information;

[0286] Based on the IP address, UID, device ID, feedback ID, and feedback time, query the Compass log to obtain the first local playback log corresponding to the target feedback information;

[0287] Based on the IP address, UID, device ID, feedback ID and feedback time, obtain the first system log corresponding to the target feedback information;

[0288] Aggregate the first local playback log and the first system log to obtain a first local log;

[0289] The first server log, the first pingback log, and the first local log are formatted to obtain a target log.

[0290] Optionally, the aggregating the first local playback log and the first system log to obtain the first local log includes:

[0291] Merging the first local playback log with the first system log to obtain a log to be processed;

[0292] The log to be processed is processed according to the preset network request format, and the logs of the target type are filtered and retained to obtain the first local log.

[0293] Optionally, the target feedback information includes: at least one of the target user's Internet Protocol address IP address, user login identifier UID, device identifier ID and feedback ID, as well as feedback time and feedback content; if there are multiple target feedback information, the fourth acquisition module 1101 is specifically used to:

[0294] Determine the type of each target feedback information based on the feedback content of each target feedback information;

[0295] Based on the IP address and feedback time, query the Hive database to obtain the second server log and the second automatic reference notification Pingback log corresponding to each target feedback information under each type;

[0296] Based on the IP address, UID, device ID, feedback ID, and feedback time, query the Compass log to obtain the second local playback log corresponding to each target feedback information under each type;

[0297] Based on the IP address, UID, device ID, feedback ID and feedback time, obtain the second system log corresponding to each target feedback information under each type;

[0298] Aggregate the second local playback log and the second system log corresponding to each target feedback information under each type to obtain a second local log;

[0299] The second server log, the second Pingback log, and the second local log of each target feedback information of each type are formatted in time sequence to obtain a target log.

[0300] Optionally, the above-mentioned target fault analysis results also include fault analysis statistical information, which is determined based on the playback error scenario, error cause or error scenario category, and includes at least one of the following: fault information counted according to feedback content, fault information counted according to error cause, and fault information counted according to the server associated with the target feedback information.

[0301] Optionally, the above device further includes:

[0302] The alarm module is used to trigger an alarm when there is fault information that meets the alarm condition in the fault analysis statistical information.

[0303] The embodiment of the present invention further provides an electronic device, such as Fig.12 As shown, it includes a processor 1201, a communication interface 1202, a memory 1203 and a communication bus 1204, wherein the processor 1201, the communication interface 1202, and the memory 1203 communicate with each other through the communication bus 1204.

[0304] Memory 1203, used for storing computer programs;

[0305] The processor 1201 is used to implement any of the above methods when executing the program stored in the memory 1203 to achieve the same technical effect.

[0306] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0307] The communication interface is used for communication between the above terminal and other devices.

[0308] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0309] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0310] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, any method in the above embodiments is implemented to achieve the same technical effect.

[0311] In another embodiment provided by the present invention, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer is enabled to execute any of the methods in the above embodiments to achieve the same technical effect.

[0312] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0313] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0314] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device / electronic device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0315] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A fault analysis model training method, characterized in that: The method comprises: Obtaining an initial fault analysis model; Obtain a first sample log and a first annotated fault analysis result corresponding to the first sample log; the first sample log is a log corresponding to a video playback error; the first annotated fault analysis result includes at least one of the following: a playback error scenario, an error cause, a solution, and an error scenario category; Inputting the first sample log into the initial fault analysis model, so that the initial fault analysis model outputs a first fault analysis result based on a first fault analysis instruction; Calculating a first loss between the first fault analysis result and the first labeled fault analysis result, and adjusting parameters of the initial fault analysis model according to the first loss to obtain a pre-trained fault analysis model; Acquire a training sample knowledge base; the training sample knowledge base includes a second sample log and a second annotated fault analysis result corresponding to the second sample log; Verifying the accuracy of the pre-trained fault analysis model using the second sample log and the second labeled fault analysis result; When the accuracy rate is not less than a preset accuracy rate, the parameters of the first fault analysis instruction and the pre-trained fault analysis model are adjusted based on the second sample log and the second labeled fault analysis result to obtain a second fault analysis instruction and a fault analysis model.

2. The method according to claim 1, characterized in that The method further comprises: When the accuracy is lower than the preset accuracy, the process returns to the step of obtaining the initial fault analysis model.

3. The method according to claim 1 or 2, characterized in that: The obtaining of the initial fault analysis model comprises: Obtain a third sample log and a third annotated fault analysis result corresponding to the third sample log; Calling a plurality of preset large language models through a preset application programming interface API, inputting the third sample log into each of the large language models respectively, and obtaining a third fault analysis result output by each of the large language models; An initial fault analysis model is selected from the plurality of large language models based on a difference between each third fault analysis result and the third annotated fault analysis result.

4. The method according to claim 1, characterized in that The step of acquiring a training sample knowledge base includes: Obtain sample feedback information when a user makes an error while playing a video within a preset historical time period; Determining the type of each sample feedback information according to the feedback content of the sample feedback information; Obtain the second sample log corresponding to each sample feedback information under each type; Determine a second annotated fault analysis result corresponding to each of the second sample logs; A training sample knowledge base is constructed based on each of the second sample logs and each of the second labeled fault analysis results.

5. The method according to claim 4, characterized in that The method further comprises: The training sample knowledge base is updated using the fourth sample log and the fourth labeled fault analysis result corresponding to the fourth sample log.

6. The method according to claim 1, characterized in that The using the second sample log and the second labeled fault analysis result to verify the accuracy of the pre-trained fault analysis model includes: Inputting the second sample log into the pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a second fault analysis result based on the first fault analysis instruction; Determine whether the second fault analysis result is consistent with the second marked fault analysis result, and obtain a judgment result; Based on the judgment result, the accuracy of the pre-trained fault analysis model is determined.

7. The method according to claim 1, characterized in that The adjusting the first fault analysis instruction and the parameters of the pre-trained fault analysis model based on the second sample log and the second labeled fault analysis result to obtain a second fault analysis instruction and a fault analysis model includes: Obtaining a plurality of fifth sample logs and fifth annotated fault analysis results corresponding to the plurality of fifth sample logs; Inputting a plurality of the fifth sample logs into the pre-trained fault analysis model, causing the pre-trained fault analysis model to output a fifth fault analysis result based on the first fault analysis instruction, and calculating a second loss between the fifth fault analysis result and the fifth labeled fault analysis result; Using the second sample log and the second annotated fault analysis result as context, querying a plurality of sixth fault analysis results corresponding to the fifth sample logs, and calculating a third loss between the sixth fault analysis result and the fifth annotated fault analysis result; Inputting a plurality of the fifth sample logs into the pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a seventh fault analysis result based on the received user interaction information, and obtains an interaction instruction for the seventh fault analysis result; Based on the second loss, the third loss and the interaction instruction, the parameters of the pre-trained fault analysis model and the first fault analysis instruction are adjusted to obtain a second fault analysis instruction and a fault analysis model.

8. The method according to claim 7, characterized in that The adjusting the parameters of the pre-trained fault analysis model and the first fault analysis instruction based on the second loss, the third loss and the interactive instruction to obtain the second fault analysis instruction and the fault analysis model includes: When the second loss does not satisfy a preset condition, the third loss does not satisfy a preset condition, and the interaction instruction is not to stop the interaction, returning to the step of obtaining an initial fault analysis model; When both the second loss and the third loss meet preset conditions and the interaction instruction is to stop interaction, the current first fault analysis instruction is determined as the second fault analysis instruction and the current parameters of the pre-trained fault analysis model are determined as the parameters of the fault analysis model.

9. The method according to claim 7, characterized in that: The fifth annotated fault analysis result includes at least one of the following: a playback error scenario, an error cause, a solution, an error scenario category, and fault analysis statistical information; the fault analysis statistical information is determined based on the playback error scenario, the error cause, or the error scenario category.

10. A fault analysis method, characterized in that: The method comprises: Obtaining target feedback information to be analyzed and a target log corresponding to the target feedback information; Obtaining a target fault analysis instruction; the target fault analysis instruction is obtained by using any method of claims 1-9; The target log is input into a pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a target fault analysis result based on the target fault analysis instruction; wherein the target fault analysis result includes at least one of the following: playback error scenario, error cause, solution and error scenario category; the pre-trained fault analysis model is trained using any one of the methods of claims 1-9.

11. The method according to claim 10, characterized in that The target feedback information includes: at least one of the target user's Internet Protocol address IP address, user login identifier UID, device identifier ID and feedback ID, and feedback time; the step of obtaining the target log corresponding to the target feedback information includes: Based on the IP address and the feedback time, query the Hive database to obtain the first server log and the first automatic reference notification Pingback log corresponding to the target feedback information; Based on the IP address, the UID, the device ID, the feedback ID, and the feedback time, query the Compass log to obtain a first local playback log corresponding to the target feedback information; Based on the IP address, the UID, the device ID, the feedback ID, and the feedback time, obtaining a first system log corresponding to the target feedback information; Aggregate the first local playback log and the first system log to obtain a first local log; The first server log, the first Pingback log, and the first local log are formatted to obtain a target log.

12. The method according to claim 11, characterized in that The aggregating the first local playback log and the first system log to obtain a first local log includes: Merging the first local playback log with the first system log to obtain a log to be processed; The to-be-processed logs are processed according to a preset network request format, and logs of a target type are screened and retained to obtain a first local log.

13. The method according to claim 10, characterized in that The target feedback information includes: at least one of the target user's Internet Protocol address IP address, user login identifier UID, device identifier ID and feedback ID, as well as feedback time and feedback content; the target feedback information is multiple, and obtaining the target log corresponding to the target feedback information includes: Determining the type of each target feedback information based on the feedback content of each target feedback information; Based on the IP address and the feedback time, query the Hive database to obtain the second server log and the second automatic reference notification Pingback log corresponding to each target feedback information under each type; Based on the IP address, the UID, the device ID, the feedback ID, and the feedback time, query the Compass log to obtain the second local playback log corresponding to each target feedback information under each type; Based on the IP address, the UID, the device ID, the feedback ID, and the feedback time, obtaining a second system log corresponding to each target feedback information under each type; Aggregate the second local playback log and the second system log corresponding to each target feedback information under each type to obtain a second local log; The second server log, the second Pingback log and the second local log of each target feedback information of each type are formatted in time sequence to obtain a target log.

14. The method according to claim 13, characterized in that The target fault analysis result also includes fault analysis statistical information, which is determined based on the playback error scenario, the error cause or the error scenario category, and includes at least one of the following: fault information counted according to feedback content, fault information counted according to error cause, and fault information counted according to a server associated with the target feedback information.

15. The method according to claim 14, characterized in that The method further comprises: When there is fault information satisfying the alarm condition in the fault analysis statistical information, an alarm is triggered.

16. A fault analysis model training device, characterized in that: The device comprises: A first acquisition module, used to acquire an initial fault analysis model; The second acquisition module is used to obtain a first sample log and a first annotated fault analysis result corresponding to the first sample log; the first sample log is a log corresponding to a video playback error; the first annotated fault analysis result includes at least one of the following: a playback error scenario, an error cause, a solution, and an error scenario category; An analysis and prediction module, configured to input the first sample log into the initial fault analysis model, so that the initial fault analysis model outputs a first fault analysis result based on a first fault analysis instruction; A first training module, used for calculating a first loss between the first fault analysis result and the first labeled fault analysis result, and adjusting parameters of the initial fault analysis model according to the first loss to obtain a pre-trained fault analysis model; A third acquisition module is used to acquire a training sample knowledge base; the training sample knowledge base includes the second sample log and the second annotated fault analysis result corresponding to the second sample log; A verification module, used to verify the accuracy of the pre-trained fault analysis model using the second sample log and the second annotated fault analysis result; The second training module is used to adjust the parameters of the first fault analysis instruction and the pre-trained fault analysis model based on the second sample log and the second labeled fault analysis result when the accuracy rate is not less than the preset accuracy rate, so as to obtain the second fault analysis instruction and the fault analysis model.

17. A fault analysis device, characterized in that: The device comprises: A fourth acquisition module, used to acquire target feedback information to be analyzed and a target log corresponding to the target feedback information; A fifth acquisition module, used to acquire a target fault analysis instruction; the target fault analysis instruction is obtained by using any method of claims 1-9; A fault analysis module is used to input the target log into a pre-trained fault analysis model, so that the pre-trained fault analysis model outputs a target fault analysis result based on the target fault analysis instruction; wherein the target fault analysis result includes at least one of the following: playback error scenario, error cause, solution and error scenario category; the pre-trained fault analysis model is trained using any one of the methods of claims 1-9.

18. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-15 when executing a program stored in a memory.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

Images