Problem attribution method and device based on large model, electronic equipment and storage medium

Abstract

The invention provides a problem attribution method and device based on a large model, electronic equipment and a storage medium, and relates to the technical field of computers, in particular to the technical fields of artificial intelligence, the large model, application program development, application program maintenance, generative search, intelligent document editing, intelligent assistants, virtual humans and the like. According to the specific implementation scheme, the current problem of a target program is determined; obtaining a first number of reference cases related to the current question; wherein each reference case in the first number of reference cases comprises a reference historical problem of the target program and a reference attribution algorithm for the reference historical problem; determining a target attribution algorithm for the current problem based on the first number of reference cases by using the large model; and utilizing a target attribution algorithm to obtain an attribution result aiming at the current problem.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to technologies such as artificial intelligence, large models, application development, application maintenance, generative search, intelligent document editing, intelligent assistants, virtual humans, etc. Specifically, it relates to a method, apparatus, electronic device, and storage medium for problem attribution based on a large model. Background Art

[0002] In recent years, with the rapid development of computer technologies, various application programs have emerged continuously. In order to ensure the effective and stable operation of these application programs, it has become increasingly crucial to accurately attribute various online problems they encounter. Once the attribution is clear, the application programs can be optimized and improved quickly and effectively based on these attribution results. Summary of the Invention

[0003] The present disclosure provides a method, apparatus, electronic device, and storage medium for problem attribution based on a large model.

[0004] According to a first aspect of the present disclosure, there is provided a method for problem attribution based on a large model, including:

[0005] Determine the current problem of the target program;

[0006] Obtain a first number of reference cases related to the current problem; wherein each of the first number of reference cases includes a reference historical problem of the target program and a reference attribution algorithm for the reference historical problem;

[0007] Use a large model to determine a target attribution algorithm for the current problem based on the first number of reference cases;

[0008] Use the target attribution algorithm to obtain an attribution result for the current problem.

[0009] According to a second aspect of the present disclosure, there is provided a device for problem attribution based on a large model, including:

[0010] A current problem determination unit for determining the current problem of the target program;

[0011] A reference case acquisition unit for obtaining a first number of reference cases related to the current problem; wherein each of the first number of reference cases includes a reference historical problem of the target program and a reference attribution algorithm for the reference historical problem;

[0012] An attribution algorithm acquisition unit for using a large model to determine a target attribution algorithm for the current problem based on the first number of reference cases;

[0013] An attribution result acquisition unit, configured to obtain an attribution result for the current problem by using a target attribution algorithm.

[0014] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0015] At least one processor;

[0016] A memory communicatively connected to the at least one processor;

[0017] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided by the embodiments of the present disclosure.

[0018] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method provided by the embodiments of the present disclosure.

[0019] According to a fifth aspect of the present disclosure, there is provided a computer program product, including a computer program, which implements the method provided by the embodiments of the present disclosure when executed by a processor.

[0020] By using the present disclosure, an attribution result for the current problem can be obtained quickly and accurately.

[0021] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0022] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0023] Figure 1 It is a schematic flowchart of a method for problem attribution based on a large model provided by an embodiment of the present disclosure;

[0024] Figure 2 It is a schematic diagram of the integrity process framework of a method for problem attribution based on a large model provided by an embodiment of the present disclosure;

[0025] Figure 3 It is an auxiliary explanatory diagram of a method for problem attribution based on a large model provided by an embodiment of the present disclosure;

[0026] Figure 4 It is an auxiliary explanatory diagram of a method for problem attribution based on a large model provided by an embodiment of the present disclosure;

[0027] Figure 5A schematic diagram of a problem attribution method based on a large model provided by an embodiment of the present disclosure;

[0028] Figure 6 A schematic structural block diagram of a problem attribution device based on a large model provided by an embodiment of the present disclosure;

[0029] Figure 7 A schematic structural block diagram of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0030] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to assist understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted below.

[0031] Currently, the following method is usually adopted to attribute various online problems encountered by an application (APP):

[0032] During the operation of the APP, its log data is periodically sent to the server. After receiving the log data, the server updates the business reports created for each business scenario of the APP (such as the search scenario, recommendation scenario, etc.). The business reports include various dimension information of the APP (such as the operating system, name, version, etc.) and evaluation metrics (such as the search arrival rate, search speed, etc. in the search scenario), and at the same time, detection and warning of abnormal metrics are configured based on the business reports, so as to achieve automatic warning of online problems. In this way, after receiving the warning, APP operation and maintenance personnel (such as Research and Development Engineers (RD), Quality Assurance Engineers (QA), etc.) can attribute the online problems encountered by the APP by manually checking the business reports to obtain the attribution results for the online problems.

[0033] However, with the upgrade and iteration of the APP, the dimension information and evaluation metrics included in the business reports will be more and more. In this case, relying on the troubleshooting experience of APP operation and maintenance personnel will bring in more human subjectivity. Therefore, it is difficult to quickly and accurately obtain the attribution results for online problems.

[0034] In view of the above problems, embodiments of the present disclosure provide a problem attribution method based on a large model, which can be applied to an electronic device. Among them, the electronic device can be a server, or a workbench, a mainframe computer, or other similar computing devices. Hereinafter, in conjunction with Figure 1 The following schematic flowchart will be used to illustrate a problem attribution method based on a large model provided by an embodiment of the present disclosure. It should be noted that although the logical order is shown in the schematic flowchart, in some cases, the steps shown or described may also be executed in other orders.

[0035] Step S101, determine the current problem of the target program.

[0036] Among them, the target program can be any APP that can run on a terminal device, and the specific type can be social consulting, multimedia application, shopping, tool, etc. Here, the terminal device can be a conventional computer (such as a desktop computer, a laptop computer, etc.), an in-vehicle computer, a tablet computer, a smart phone, a personal digital assistant, or other similar computing devices.

[0037] In addition, in the embodiments of the present disclosure, the current problem can be any online problem that the target program may encounter. Among them, the online problem can be a program stability problem or a business exception problem. For example, when the target program is a social consulting APP, the current problem can be program stability problems such as an abnormal increase in the crash rate and an abnormal increase in the flashback rate, or business exception problems such as an abnormal decrease in the search reach rate, an abnormal decrease in the search speed, and an abnormal increase in the traffic volume; for another example, when the target program is a shopping APP, the current problem can be program stability problems such as an abnormal increase in the crash rate and an abnormal increase in the flashback rate, or business exception problems such as an abnormal increase in the traffic volume and an abnormal decrease in the order conversion rate.

[0038] Step S102, obtain the first number of reference cases related to the current problem.

[0039] Among them, each reference case in the first number of reference cases can include a reference historical problem of the target program and a reference attribution algorithm for the reference historical problem. Here, for each reference case in the first number of reference cases, the reference historical problem included in the reference case can be an online problem related to (such as the same or similar) the current problem that the target program encountered during the historical period, and moreover, the reference attribution algorithm included in the reference case is a problem attribution algorithm that can accurately attribute the reference historical problem included in the reference case.

[0040] In addition, in the embodiments of the present disclosure, for each of the first number of reference cases, the reference attribution algorithm included in the reference case may be one of multiple problem attribution algorithms. Among them, the multiple problem attribution algorithms may include the over-average method, the variance method, the Gini coefficient method, the Jensen-Shannon (JS) divergence method, the fluctuation contribution calculation method, the bucket test method (also known as the A / B Test method), the proportion method, etc.

[0041] Step S103: Use the large model to determine the target attribution algorithm for the current problem based on the first number of reference cases.

[0042] Among them, the large model may be a large language model (LLM), specifically a pre-trained neural network model (for example, an autoregressive generation model with a Transformer architecture), which has general language knowledge, world knowledge, and professional knowledge in various fields (for example, professional knowledge in the field of computer technology). Based on this, in the embodiments of the present disclosure, after obtaining the first number of reference cases related to the current problem, the large model can be directly used to determine the target attribution algorithm for the current problem based on the first number of reference cases. Among them, the target attribution algorithm can be regarded as a problem attribution algorithm that can accurately attribute the current problem.

[0043] Step S104: Use the target attribution algorithm to obtain the attribution result for the current problem.

[0044] Among them, the attribution result can be used to represent the main factor that causes the current problem in the target program; or, it can be used to represent the main factor that causes the current problem in the target program and the contribution degree of this main factor to causing the current problem in the target program.

[0045] By using the problem attribution method based on the large model provided in the embodiments of the present disclosure, after determining the current problem of the target program, the first number of reference cases related to the current problem can be obtained, and the large model can be used to determine the target attribution algorithm for the current problem based on the first number of reference cases, and then the target attribution algorithm can be used to obtain the attribution result for the current problem. In this process, not only will the first number of reference cases be used as a rich basis for algorithm determination to determine the target attribution algorithm for the current problem, but also this step is performed by a large model with general language knowledge, world knowledge, and professional knowledge in various fields. Therefore, the reliability of the target attribution algorithm can be ensured. On this basis, the target attribution algorithm can be directly used to quickly and accurately obtain the attribution result for the current problem.

[0046] Further, in the embodiments of the present disclosure, when the target program runs on a terminal device (here, the terminal device may be a computing device held by different users), it can send its own log data to an electronic device. After receiving the log data, the electronic device will update the overall business data (here, the overall business data may exist in the form of a business report) created for each business scenario of the target program (such as a search scenario, a recommendation scenario, etc.) based on the log data. The overall business data includes various dimension information of the target program (such as the operating system, name, version, etc.) and evaluation metrics (such as the search arrival rate, search speed, etc. in the search scenario), etc. At the same time, it will also configure the detection and warning of abnormal metrics based on the overall business data, and use the evaluation metrics that trigger the detection and warning as the main object of the current problem, and then determine the current problem. Here, the detection and warning of abnormal metrics can be implemented based on a threshold alarm strategy, which is not elaborated in the embodiments of the present disclosure.

[0047] In some alternative embodiments, step S102, that is, "obtain the first number of reference cases related to the current problem" may include:

[0048] Determine multiple case recall methods;

[0049] Take each case recall method among the multiple case recall methods as the target recall method, and recall the second number of primary selected cases related to the current problem from multiple candidate cases according to the target recall method;

[0050] Obtain the first number of reference cases based on the second number of primary selected cases.

[0051] Among them, the multiple case recall methods may include at least one of a similarity recall method and a keyword recall method. Here, the similarity recall method can be implemented based on the word embedding technology (Bi-encoder Graph Embedding, BGE Embedding); the keyword recall method can be implemented based on an information retrieval algorithm (such as the 25th generation best match algorithm (Best Match 25, BM25)); the second number can be set according to actual application requirements. For example, it can be set to 5, which is not specifically limited in the embodiments of the present disclosure; the first number can be set according to actual application requirements. For example, it can be set to 5, which is also not specifically limited in the embodiments of the present disclosure.

[0052] In one example, the similarity recall method among the multiple case recall methods can be taken as the target recall method. In this case, "recall the second number of primary selected cases related to the current problem from multiple candidate cases according to the target recall method" may include:

[0053] Obtain the semantic similarity between the current problem and each of multiple candidate cases;

[0054] Based on the semantic similarity, recall a second number of primary cases related to the current problem from multiple candidate cases.

[0055] Among them, each candidate case in multiple candidate cases may include a candidate historical problem of the target program and a candidate attribution algorithm for the candidate historical problem. Here, for each candidate case in multiple candidate cases, the candidate attribution algorithm included in the candidate case is a problem attribution algorithm that can accurately attribute the candidate historical problem included in the candidate case.

[0056] In a specific example, each candidate case in multiple candidate cases can be used as a target case, calculate the semantic similarity between the current problem and the candidate historical problem included in the target case as the semantic similarity between the current problem and the target case, and recall a second number of candidate cases with the highest semantic similarity to the current problem from multiple candidate cases as primary cases to obtain a second number of primary cases. In a more specific example, "calculate the semantic similarity between the current problem and the candidate historical problem included in the target case" may include: vectorize the current problem to obtain a current problem vector; vectorize the candidate historical problem included in the target case to obtain a candidate historical problem vector; calculate the similarity between the current problem vector and the candidate historical problem vector as the semantic similarity between the current problem and the candidate historical problem included in the target case.

[0057] In the above example, the semantic similarity between the current problem and each candidate case in multiple candidate cases can be obtained, and based on the semantic similarity, a second number of primary cases related to the current problem can be recalled from multiple candidate cases. In this process, no complex calculation process is involved. Therefore, the execution efficiency of the problem attribution method based on the large model can be improved, and the attribution result for the current problem can be obtained more quickly.

[0058] In another example, the keyword recall method in multiple case recall methods can be used as the target recall method. In this case, "recall a second number of primary cases related to the current problem from multiple candidate cases according to the target recall method" may include:

[0059] Determine at least one keyword from the current problem;

[0060] Obtain the overall relevance between at least one keyword and each candidate case in multiple candidate cases;

[0061] Based on the overall relevance, recall a second number of primary cases related to the current problem from multiple candidate cases.

[0062] Among them, each of the multiple candidate cases may include candidate historical problems of the target program and a candidate attribution algorithm for the candidate historical problems. Here, for each of the multiple candidate cases, the candidate attribution algorithm included in the candidate case is a problem attribution algorithm that can accurately attribute the candidate historical problems included in the candidate case.

[0063] In a specific example, after determining at least one keyword of the current problem, the overall relevance of at least one keyword to each of the multiple candidate cases can be obtained in the following manner:

[0064] Taking each keyword in at least one keyword as a target term, obtaining the inverse document frequency of the target term in the multiple candidate cases;

[0065] Taking each of the multiple candidate cases as a target case, obtaining the term frequency of the target term in the target case;

[0066] Based on the inverse document frequency and the term frequency, obtaining the overall relevance of at least one keyword to the target case.

[0067] Among them, the inverse document frequency can be used to represent the universality of the target term in the multiple candidate cases, specifically, it can be used to represent the universality of the target term in the candidate historical problems included in the multiple candidate cases; the term frequency can be used to represent the occurrence frequency of the target term in the target case, specifically, it can be used to represent the occurrence frequency of the target term in the candidate historical problems included in the target case.

[0068] In a more specific example, the inverse document frequency of the target term in the multiple candidate cases can be obtained through the following first calculation logic:

[0069]

[0070] Among them, IDF(i) is used to represent the i-th keyword in at least one keyword, that is, the inverse document frequency of the target term ti in the multiple candidate cases; N is used to represent the total number of candidate cases; n ti is used to represent the number of candidate cases including the target term ti.

[0071] After obtaining the inverse document frequency of the target term in the multiple candidate cases, each of the multiple candidate cases can be taken as a target case, the term frequency of the target term in the target case can be obtained, and based on the inverse document frequency and the term frequency, the overall relevance of at least one keyword to the target case can be obtained. This process can be represented by the following second calculation logic:

[0072]

[0073] Among them, BM25(D, Q) is used to characterize the overall relevance of at least one keyword to the target case; D is used to characterize the target case; Q is used to characterize each keyword in at least one keyword; n is used to characterize the total number of keywords; f i,D is used to characterize the term frequency of the target term ti in the candidate historical questions of the target case D; k is used to characterize the first adjustment parameter, which can be set according to actual application requirements. For example, it can be set within the numerical range [1.2, 2], and the embodiments of the present disclosure do not make specific limitations on this; b is used to characterize the second adjustment parameter, which can be set according to actual application requirements. For example, it can be set within the numerical range [0, 1], and the embodiments of the present disclosure do not make specific limitations on this; |D| is used to characterize the total number of terms in the candidate historical questions in the target case; is used to characterize the average number of terms in the candidate historical questions in multiple candidate cases; IDF(i) is used to characterize the inverse document frequency of the target term ti in multiple candidate cases.

[0074] After obtaining the overall relevance of at least one keyword to each candidate case among multiple candidate cases, the second number of candidate cases with the highest overall relevance to at least one keyword can be recalled from the multiple candidate cases as the primary selected cases, so as to obtain the second number of primary selected cases.

[0075] In the above example, at least one keyword can be determined from the current question, and the overall relevance of at least one keyword to each candidate case among multiple candidate cases can be obtained. Then, based on the overall relevance, the second number of primary selected cases related to the current question can be recalled from the multiple candidate cases. In this process, no complex calculation process is involved. Therefore, the execution efficiency of the problem attribution method based on the large model can be improved, and thus the attribution result for the current question can be obtained more quickly.

[0076] After taking the similarity recall method among multiple case recall methods as the target recall method and recalling the second number of primary selected cases related to the current question from multiple candidate cases according to the target recall method, and at the same time, taking the keyword recall method among multiple case recall methods as the target recall method and recalling the second number of primary selected cases related to the current question from multiple candidate cases according to the target recall method, these primary selected cases can be further fused and screened to obtain the first number of reference cases.

[0077] Exemplarily, the similarity recall method among multiple case recall methods is used as the target recall method, and according to the target recall method, the second number of primary selected cases related to the current problem are recalled from multiple candidate cases, including: primary selected case A1, primary selected case A2, primary selected case A3, primary selected case A4, and primary selected case A5. The semantic similarities (specifically, the normalized semantic similarities) of these primary selected cases with the current problem are respectively:

[0078] Primary selected case A1: 0.95;

[0079] Primary selected case A2: 0.90;

[0080] Primary selected case A3: 0.88;

[0081] Primary selected case A4: 0.80;

[0082] Primary selected case A5: 0.78.

[0083] The keyword recall method among multiple case recall methods is used as the target recall method, and according to the target recall method, the second number of primary selected cases related to the current problem are recalled from multiple candidate cases, including: primary selected case A1, primary selected case A2, primary selected case A3, primary selected case A4, and primary selected case A6. The overall relevance degrees (specifically, the normalized overall relevance degrees) of these primary selected cases with at least one keyword determined from the current problem are respectively:

[0084] Primary selected case A1: 0.90;

[0085] Primary selected case A2: 0.96;

[0086] Primary selected case A3: 0.88;

[0087] Primary selected case A4: 0.80;

[0088] Primary selected case A6: 0.80.

[0089] Then, for the preliminary selected case A1, its comprehensive relevance to the current problem is: 0.95 + 0.90 = 1.85; for the preliminary selected case A2, its comprehensive relevance to the current problem is: 0.90 + 0.96 = 1.86; for the preliminary selected case A3, its comprehensive relevance to the current problem is: 0.88 + 0.88 = 1.76; for the preliminary selected case A4, its comprehensive relevance to the current problem is: 0.80 + 0.80 = 1.60; for the preliminary selected case A5, its comprehensive relevance to the current problem is: 0.78; for the preliminary selected case A6, its comprehensive relevance to the current problem is: 0.80. Therefore, further fusion screening is performed on the preliminary selected cases A1, A2, A3, A4, A5, and A6, and the first number of reference cases obtained may include the preliminary selected cases A1, A2, A3, A4, and A6.

[0090] In the above manner, in the embodiments of the present disclosure, multiple case recall methods can be determined, and each case recall method among the multiple case recall methods is used as the target recall method, so as to recall the second number of preliminary selected cases related to the current problem from multiple candidate cases according to the target recall method, and then based on the second number of preliminary selected cases, the first number of reference cases are obtained. That is to say, in the embodiments of the present disclosure, the first number of reference cases are jointly determined according to different case recall methods. Therefore, it can be ensured that the first number of reference cases are highly relevant to the current problem. In this way, when using the large model to determine the target attribution algorithm for the current problem based on the first number of reference cases, the reliability of the determined target attribution algorithm can be improved.

[0091] In some alternative embodiments, step S103, that is, "using the large model, based on the first number of reference cases, to determine the target attribution algorithm for the current problem" may include:

[0092] Generating initial prompt information based on the first number of reference cases;

[0093] Supplementing the initial prompt information with preset output requirements to obtain target prompt information;

[0094] Using the large model, based on the target prompt information, to determine the target attribution algorithm for the current problem.

[0095] Among them, the initial prompt information may include an algorithm determination instruction and the first number of reference cases. Here, the algorithm determination instruction can be used to instruct the large model to determine the target attribution algorithm for the current problem with the first number of reference cases as the reference information.

[0096] In one example, the preset output requirement may be that when using a large model to determine a target attribution algorithm for the current problem based on the target prompt information, it is necessary to synchronously determine which reference case among the first number of reference cases the target attribution algorithm is specifically determined based on. In this case, "using the preset output requirement to supplement the initial prompt information to obtain the target prompt information" may include: using the preset output requirement to supplement the algorithm determination indication included in the initial prompt information to obtain a supplemented algorithm determination indication, and using the supplemented algorithm determination indication and the first number of reference cases together as the target prompt information. Among them, the supplemented algorithm determination indication can be used to instruct the large model to determine the target attribution algorithm for the current problem with the first number of reference cases as the reference information, and determine which reference case among the first number of reference cases the target attribution algorithm is specifically determined based on.

[0097] In the above manner, in the embodiments of the present disclosure, the initial prompt information can be generated based on the first number of reference cases, and the initial prompt information is supplemented using the preset output requirement to obtain the target prompt information, and then the large model is used to determine the target attribution algorithm for the current problem based on the target prompt information. That is to say, in the embodiments of the present disclosure, the output result of the large model can meet the preset output requirement. For example, when determining the target attribution algorithm for the current problem, it will further determine which reference case among the first number of reference cases the target attribution algorithm is specifically determined based on. In this way, after determining the target attribution algorithm for the current problem, the APP operation and maintenance personnel can combine the output result of the large model, "which reference case among the first number of reference cases the target attribution algorithm is specifically determined based on", to review the target attribution algorithm to further improve the reliability of the determined target attribution algorithm.

[0098] In some alternative embodiments, step S104, that is, "using the target attribution algorithm to obtain the attribution result for the current problem" may include:

[0099] Based on the overall business data related to the current problem, determine the actual generation time of the current problem;

[0100] Based on the actual generation time, select the target business data from the overall business data;

[0101] Use the target attribution algorithm to obtain the attribution result for the current problem based on the target business data.

[0102] Among them, the overall business data includes various dimension information of the target program (such as operating system, name, version, etc.) and evaluation metrics (such as search reach rate, search speed, etc. in the search scenario). Based on this, in the embodiments of the present disclosure, the overall business data related to the current problem can be various dimension information of the target program, and the evaluation metrics related to the current problem at different time points (that is, the main object of the current problem). For example, if the target program is a social consulting APP and the current problem is an abnormal increase in the crash rate, the evaluation metrics related to the current problem can be the crash rates of the target program at different time points; for another example, if the target program is a shopping APP and the current problem is an abnormal decrease in the order conversion rate, the evaluation metrics related to the current problem can be the order conversion rates of the target program at different time points.

[0103] In one example, after obtaining the overall business data related to the current problem, the actual generation time of the current problem can be determined in the following manner:

[0104] Determine multiple change point detection algorithms;

[0105] Take each change point detection algorithm among the multiple change point detection algorithms as the target detection algorithm, and use the target detection algorithm to obtain multiple candidate time points based on the overall business data related to the current problem;

[0106] Based on the multiple candidate time points, determine the actual generation time of the current problem.

[0107] Among them, the multiple change point detection algorithms include at least one of the Bayesian change point detection algorithm and the Pruned Exact Linear Time (PELT) algorithm.

[0108] In one example, the Bayesian change point detection algorithm among the multiple change point detection algorithms can be taken as the target detection algorithm. In this case, the target detection algorithm can be used to determine multiple candidate time points from the time point set and the inflection point probability of each candidate time point among the multiple candidate time points based on the overall business data related to the current problem.

[0109] In another example, the PELT algorithm among the multiple change point detection algorithms can be taken as the target detection algorithm. In this case, the target detection algorithm can be used to determine multiple candidate time points from the time point set and the importance score of each candidate time point among the multiple candidate time points based on the overall business data related to the current problem. In a specific example, for each time point in the time point set, the importance score can be obtained in the following manner:

[0110] First, it can be defined that the set of time points includes N time points, and the overall business data related to the current problem has N evaluation metrics corresponding one-to-one to the N time points. In this case, there is:

[0111] Loss Difference i =Loss without split -Loss with split

[0112] Among them, Loss Difference i is used to represent the loss difference at the i-th time point among the N time points, that is, at the i-th evaluation metric among the N evaluation metrics, when splitting the N evaluation metrics, the loss difference before and after the split; Loss without split is used to represent the overall loss of the N evaluation metrics when not splitting the N evaluation metrics, specifically, it can be the fitting error of the N evaluation metrics (for example, mean square error or absolute error); Loss with split is used to represent the total single-point loss at the i-th time point among the N time points, that is, at the i-th evaluation metric among the N evaluation metrics, after splitting the N evaluation metrics, the sum of the losses of the two parts of the evaluation metrics before and after, specifically, it can be the sum of the fitting errors (for example, mean square error or absolute error) of the two parts of the evaluation metrics before and after. At the same time, there is also:

[0113]

[0114] Among them, RelativeScore i is used to represent the relative loss at the i-th time point among the N time points; Loss with split is used to represent the total single-point loss at the i-th time point among the N time points, that is, at the i-th evaluation metric among the N evaluation metrics, after splitting the N evaluation metrics, the sum of the losses of the two parts of the evaluation metrics before and after, specifically, it can be the sum of the fitting errors (for example, mean square error or absolute error) of the two parts of the evaluation metrics before and after; Average Loss is used to represent the average value of the total single-point losses of the N time points.

[0115] After that, through the following third calculation logic, the importance score of the i-th time point among the N time points can be obtained:

[0116] Importance Score i =α*LossDifference i +β*(1 - Relative Scorei )

[0117] Among them, the Importance Score i is used to represent the importance score of the i-th time point among N time points; α is used to represent the first weight parameter, which can be set according to actual application requirements. For example, it can be set within the numerical range (0, 1), and the embodiments of the present disclosure do not make specific limitations on this; Loss Difference i is used to represent the loss difference of the i-th time point among N time points, that is, at the i-th evaluation index among N evaluation indicators, when splitting N evaluation indicators, the loss difference before and after splitting; β is used to represent the second weight parameter, which can be set according to actual application requirements. For example, it can be set within the numerical range (0, 1) and satisfy the constraint condition of α + β = 1, and the embodiments of the present disclosure do not make specific limitations on this; Relative Score i is used to represent the relative loss of the i-th time point among N time points.

[0118] After taking the Bayesian change point detection algorithm among multiple change point detection algorithms as the target detection algorithm, and using the target detection algorithm, based on the overall business data related to the current problem, determining multiple candidate time points from the time point set, and the inflection point probability of each candidate time point among the multiple candidate time points, at the same time, taking the PELT algorithm among multiple change point detection algorithms as the target detection algorithm, and using the target detection algorithm, based on the overall business data related to the current problem, determining multiple candidate time points from the time point set, and the importance score of each candidate time point among the multiple candidate time points, these candidate time points can be further fused and screened to determine the actual generation time of the current problem.

[0119] Exemplarily, taking the Bayesian change point detection algorithm among multiple change point detection algorithms as the target detection algorithm, and using the target detection algorithm, based on the overall business data related to the current problem, determining multiple candidate time points from the time point set, and the inflection point probability (specifically, the inflection point probability after normalization processing) of each candidate time point among the multiple candidate time points are respectively:

[0120] Candidate time point B1: 0.94;

[0121] Candidate time point B2: 0.94;

[0122] Candidate time point B3: 0.86;

[0123] Candidate time point B4: 0.80;

[0124] Candidate time point B5: 0.78.

[0125] Take the PELT algorithm among multiple change point detection algorithms as the target detection algorithm, and use the target detection algorithm to determine multiple candidate time points from the set of time points based on the overall business data related to the current problem, and the importance scores (specifically, the importance scores after normalization) of each candidate time point among the multiple candidate time points are respectively:

[0126] Candidate time point B1: 0.94;

[0127] Candidate time point B2: 0.98;

[0128] Candidate time point B3: 0.88;

[0129] Candidate time point B4: 0.85;

[0130] Candidate time point B6: 0.80.

[0131] Then, for candidate time point B1, its comprehensive probability score is: 0.94 + 0.94 = 1.88; for candidate time point B2, its comprehensive probability score is: 0.94 + 0.98 = 1.92; for candidate time point B3, its comprehensive probability score is: 0.86 + 0.88 = 1.74; for candidate time point B4, its comprehensive probability score is: 0.80 + 0.85 = 1.65; for candidate time point B5, its comprehensive probability score is: 0.78; for candidate time point B6, its comprehensive probability score is: 0.80. Therefore, further fusion screening is performed on candidate time point B1, candidate time point B2, candidate time point B3, candidate time point B4, candidate time point B5 and candidate time point B6 to determine that the actual generation time of the current problem is candidate time point B2.

[0132] In the above example, multiple change point detection algorithms can be determined, and each change point detection algorithm among the multiple change point detection algorithms is used as the target detection algorithm to use the target detection algorithm to obtain multiple candidate time points based on the overall business data related to the current problem, and then based on the multiple candidate time points, determine the actual generation time of the current problem. That is to say, in the embodiments of the present disclosure, the actual generation time of the current problem is jointly determined according to different change point detection algorithms. Therefore, the reliability of the actual generation time of the current problem can be ensured.

[0133] After determining the actual generation time of the current problem, target business data can be selected from the overall business data based on the actual generation time, and the target attribution algorithm is used to obtain the attribution result for the current problem based on the target business data.

[0134] Exemplarily, the target program is a social consulting APP, the current problem is an abnormal increase in the crash rate, and the target attribution algorithm for the current problem is the bucket test method. In this case, based on the actual generation time, the target business data selected from the overall business data may include the total number of requests received by the target program from the specified historical time to the actual generation time, and the total number of crashes generated. Specifically, it can also be divided according to various dimension information. For example, differentiated by the dimension of the operating system, the target business data selected from the overall business data may include:

[0135] (1) For the target program with the Android operating system, the first number of requests X1 received from the specified historical time to the actual generation time, and the first number of crashes Y1 generated;

[0136] (2) For the target program with the HarmonyOS operating system, the second number of requests X2 received from the specified historical time to the actual generation time, and the second number of crashes Y2 generated.

[0137] Then, using the target attribution algorithm and based on the target business data, the attribution result for the current problem can be obtained:

[0138] For the Android system, there is:

[0139]

[0140] For the HarmonyOS system, there is:

[0141]

[0142] In the case where A / BTest1 > A / BTest2 and the difference value between A / BTest1 and A / BTest2 is greater than the preset difference threshold, the attribution result for the current problem can be obtained as: relative to the HarmonyOS system, there may be a compatibility problem between the target program and the Android system, resulting in the problem of abnormal increase in the crash rate of the target program. Among them, the preset difference threshold can be set according to actual application requirements, and the embodiments of the present disclosure do not make specific limitations on this.

[0143] In the case where A / BTest1 < A / BTest2 and the difference value between A / BTest1 and A / BTest2 is greater than the preset difference threshold, the attribution result for the current problem can be obtained as: relative to the Android system, there may be a compatibility problem between the target program and the HarmonyOS system, resulting in the problem of abnormal increase in the crash rate of the target program.

[0144] In the above manner, in the embodiments of the present disclosure, based on the overall business data related to the current problem, the actual generation time of the current problem can be determined, and based on the actual generation time, the target business data can be selected from the overall business data. Then, using the target attribution algorithm and based on the target business data, the attribution result for the current problem can be obtained. That is to say, in the embodiments of the present disclosure, after determining the target attribution algorithm for the current problem, the specific attribution basis used is the target business data, and the target business data is selected from the overall business data based on the actual generation time of the current problem determined based on the overall business data related to the current problem. Therefore, the reliability of the target business data can be ensured, thereby further improving the accuracy of the attribution result for the current problem.

[0145] Next, in conjunction with Figure 2 , the integrity process framework of a problem attribution method based on a large model provided by the embodiments of the present disclosure will be described.

[0146] First, determine the current problem of the target program.

[0147] Among them, the target program can be any APP that can run on a terminal device, and the specific type can be social consulting, multimedia application, shopping, tool, etc. Here, the terminal device can be a conventional computer (e.g., desktop computer, laptop computer, etc.), in-vehicle computer, tablet computer, smart phone, personal digital assistant or other similar computing devices.

[0148] In addition, in the embodiments of the present disclosure, the current problem can be any online problem that the target program may encounter. Among them, the online problem can be a program stability problem or a business exception problem. For example, when the target program is a social consulting APP, the current problem can be program stability problems such as an abnormal increase in the crash rate or an abnormal increase in the flashback rate, or business exception problems such as an abnormal decrease in the search reach rate, an abnormal decrease in the search speed, or an abnormal increase in the access volume; for another example, when the target program is a shopping APP, the current problem can be program stability problems such as an abnormal increase in the crash rate or an abnormal increase in the flashback rate, or business exception problems such as an abnormal increase in the access volume or an abnormal decrease in the order conversion rate.

[0149] After determining the current problem of the target program, the first number of reference cases related to the current problem can be obtained.

[0150] Among them, each of the first quantity of reference cases may include reference historical problems of the target program and a reference attribution algorithm for the reference historical problems. Here, for each of the first quantity of reference cases, the reference historical problems included in the reference case may be online problems related to the current problem (e.g., the same or similar) encountered by the target program during a historical period, and the reference attribution algorithm included in the reference case is a problem attribution algorithm that can accurately attribute the reference historical problems included in the reference case.

[0151] In addition, in the embodiments of the present disclosure, for each of the first quantity of reference cases, the reference attribution algorithm included in the reference case may be one of a plurality of problem attribution algorithms. Among them, the plurality of problem attribution algorithms may include the super-average method, the variance method, the Gini coefficient method, the JS divergence method, the fluctuation contribution calculation method, the bucket measurement method, the proportion method, etc.

[0152] Specifically, please combine Figure 3 , after determining the current problem of the target program, a plurality of case recall methods may be determined, and each case recall method in the plurality of case recall methods is used as the target recall method to recall a second quantity of primary selected cases related to the current problem from a plurality of candidate cases according to the target recall method, and then based on the second quantity of primary selected cases, the first quantity of reference cases are obtained. Among them, the plurality of case recall methods may include at least one of a similarity recall method and a keyword recall method. Here, the similarity recall method may be implemented based on BGE Embedding; the keyword recall method may be implemented based on BM25; the second quantity may be set according to actual application requirements. For example, it may be set to 5, and the embodiments of the present disclosure do not make specific limitations on this; the first quantity may be set according to actual application requirements. For example, it may be set to 5, and the embodiments of the present disclosure also do not make specific limitations on this.

[0153] After obtaining the first quantity of reference cases related to the current problem, an initial prompt message may be generated based on the first quantity of reference cases, and the initial prompt message is supplemented using a preset output requirement to obtain a target prompt message, and then using a large model, based on the target prompt message, a target attribution algorithm for the current problem is determined, that is, using a large model, based on the target prompt message, a target attribution algorithm for the current problem is determined from a plurality of problem attribution algorithms such as the super-average method, the variance method, the Gini coefficient method, the JS divergence method, the fluctuation contribution calculation method, the bucket measurement method, the proportion method, etc.

[0154] Finally, based on the actual generation time of the current problem, target business data is selected from the overall business data, and then using the target attribution algorithm, based on the target business data, the attribution result for the current problem is obtained. Among them, the actual generation time of the current problem can be determined based on the overall business data related to the current problem. Please combine with Figure 4 , in one example, multiple change point detection algorithms can be determined, and each change point detection algorithm in the multiple change point detection algorithms is used as the target detection algorithm, so as to use the target detection algorithm to obtain multiple candidate time points based on the overall business data related to the current problem, and then based on the multiple candidate time points, determine the actual generation time of the current problem. Among them, the multiple change point detection algorithms include at least one of the Bayesian change point detection algorithm and the PELT algorithm.

[0155] Please refer to Figure 5 , which is a schematic diagram of the scenario of a problem attribution method based on a large model provided by an embodiment of the present disclosure.

[0156] As described above, the problem attribution method based on a large model provided by an embodiment of the present disclosure is applied to an electronic device. Among them, the electronic device can be a server, or a workbench, a computer, or other similar computing devices.

[0157] Among them, the electronic device is used for:

[0158] Determine the current problem of the target program;

[0159] Obtain the first number of reference cases related to the current problem; among them, each reference case in the first number of reference cases includes the reference historical problems of the target program and the reference attribution algorithms for the reference historical problems;

[0160] Use the large model to determine the target attribution algorithm for the current problem based on the first number of reference cases;

[0161] Use the target attribution algorithm to obtain the attribution result for the current problem.

[0162] It should be noted that in the embodiments of the present disclosure, Figure 3 The shown schematic diagram of the scenario is only illustrative and not restrictive. Those skilled in the art can make various obvious changes and / or substitutions based on Figure 3 the example, and the obtained technical solutions still fall within the scope of the disclosure of the embodiments of the present disclosure.

[0163] To better implement the problem attribution method based on a large model, an embodiment of the present disclosure also provides a problem attribution device based on a large model, which can be integrated in an electronic device. Among them, the electronic device can be a server, or a workbench, a mainframe computer, or other similar computing devices. Hereinafter, it will be combined with Figure 6The following is a schematic structural block diagram for explaining a problem attribution device 600 based on a large model provided by the disclosed embodiments.

[0164] In the disclosed embodiments, the problem attribution device 600 based on a large model includes:

[0165] A current problem determination unit 601, configured to determine the current problem of the target program;

[0166] A reference case acquisition unit 602, configured to acquire a first number of reference cases related to the current problem; wherein, each reference case in the first number of reference cases includes a reference historical problem of the target program and a reference attribution algorithm for the reference historical problem;

[0167] An attribution algorithm acquisition unit 603, configured to use a large model to determine a target attribution algorithm for the current problem based on the first number of reference cases;

[0168] An attribution result acquisition unit 604, configured to obtain an attribution result for the current problem by using the target attribution algorithm.

[0169] In some alternative embodiments, the reference case acquisition unit 602 is configured to:

[0170] Determine multiple case recall methods;

[0171] Take each case recall method in the multiple case recall methods as a target recall method, and recall a second number of primary selected cases related to the current problem from multiple candidate cases according to the target recall method;

[0172] Obtain a first number of reference cases based on the second number of primary selected cases.

[0173] In some alternative embodiments, the target recall method is a similarity recall method; the reference case acquisition unit 602 is configured to:

[0174] Obtain the semantic similarity between the current problem and each candidate case in the multiple candidate cases;

[0175] Based on the semantic similarity, recall a second number of primary selected cases related to the current problem from the multiple candidate cases.

[0176] In some alternative embodiments, the target recall method is a keyword recall method; the reference case acquisition unit 602 is configured to:

[0177] Determine at least one keyword from the current problem;

[0178] Obtain the overall relevance between the at least one keyword and each candidate case in the multiple candidate cases;

[0179] Recall the second number of primary selected cases related to the current problem from multiple candidate cases based on the overall relevance.

[0180] In some alternative embodiments, the reference case acquisition unit 602 is configured to:

[0181] Take each keyword in at least one keyword as a target term, and obtain the inverse document frequency of the target term in multiple candidate cases;

[0182] Take each candidate case in multiple candidate cases as a target case, and obtain the term frequency of the target term in the target case;

[0183] Based on the inverse document frequency and the term frequency, obtain the overall relevance between at least one keyword and the target case.

[0184] In some alternative embodiments, the attribution algorithm acquisition unit 603 is configured to:

[0185] Generate initial prompt information based on the first number of reference cases;

[0186] Supplement the initial prompt information using a preset output requirement to obtain target prompt information;

[0187] Use a large model to determine a target attribution algorithm for the current problem based on the target prompt information.

[0188] In some alternative embodiments, the attribution result acquisition unit 604 is configured to:

[0189] Determine the actual generation time of the current problem based on the overall business data related to the current problem;

[0190] Select target business data from the overall business data based on the actual generation time;

[0191] Use the target attribution algorithm to obtain an attribution result for the current problem based on the target business data.

[0192] In some alternative embodiments, the attribution result acquisition unit 604 is configured to:

[0193] Determine multiple change point detection algorithms;

[0194] Take each change point detection algorithm in multiple change point detection algorithms as a target detection algorithm, and use the target detection algorithm to obtain multiple candidate time points based on the overall business data related to the current problem;

[0195] Determine the actual generation time of the current problem based on the multiple candidate time points.

[0196] In some alternative embodiments, the multiple change point detection algorithms include at least one of a Bayesian change point detection algorithm and an exact pruning linear time algorithm.

[0197] For the specific functions and examples of each unit in the large model-based problem attribution device 600 provided by the embodiments of the present disclosure, reference may be made to the relevant descriptions of the corresponding steps in the foregoing embodiments of the large model-based problem attribution method, which will not be elaborated herein.

[0198] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0199] Figure 7 FIG. shows a schematic structural block diagram of an exemplary electronic device 700 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital assistant, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0200] As Figure 7 shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0201] A plurality of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as, for example, a keyboard, a mouse, etc.; an output unit 707, such as, for example, various types of displays, speakers, etc.; a storage unit 708, such as, for example, a disk, an optical disc, etc.; and a communication unit 709, such as, for example, a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0202] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a CPU, a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 executes the various methods and processes described above, such as the large model-based problem attribution method. For example, in some embodiments, the large model-based problem attribution method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the large model-based problem attribution method described above can be executed. Alternatively, in other embodiments, the computing unit 701 can be configured for the large model-based problem attribution method by any other suitable means (e.g., by means of firmware).

[0203] The various embodiments of the systems and techniques described above in this article can be implemented in digital electronic circuit systems, integrated circuit systems, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), System On Chip (SOC) systems, Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor, that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0204] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program codes can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0205] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM), or a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0206] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)); and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other kinds of devices are also used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0207] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN), and the Internet.

[0208] A computer system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0209] Embodiments of the present disclosure also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute a problem attribution method based on a large model.

[0210] Embodiments of the present disclosure also provide a computer program product, including a computer program, which implements a problem attribution method based on a large model when executed by a processor.

[0211] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and no limitation is imposed herein. In addition, in the present disclosure, relational terms such as "first", "second", "third", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In addition, "a plurality" in the present disclosure can be understood to be at least two.

[0212] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A problem attribution method based on a large model, comprising: Identify current issues with the target program; Acquire a first number of reference cases related to the current problem; wherein each reference case in the first number of reference cases includes a reference historical problem of the target program and a reference attribution algorithm for the reference historical problem; Determine a target attribution algorithm for the current problem based on the first number of reference cases using the large model; The target attribution algorithm is used to obtain an attribution result for the current problem.

2. The method according to claim 1, wherein: The obtaining of a first number of reference cases related to the current problem includes: Determine multiple case recall methods; Using each of the plurality of case recall methods as a target recall method, so as to recall a second number of preliminary selected cases related to the current problem from a plurality of candidate cases according to the target recall method; Based on the second number of preliminary cases, the first number of reference cases are obtained.

3. The method according to claim 2, wherein: The target recall method is a similarity recall method; the step of recalling a second number of preliminary cases related to the current problem from a plurality of candidate cases according to the target recall method includes: Obtaining the semantic similarity between the current question and each candidate case in the multiple candidate cases; Based on the semantic similarity, a second number of preliminary selected cases related to the current question are recalled from a plurality of candidate cases.

4. The method according to claim 2, wherein: The target recall method is a keyword recall method; the step of recalling a second number of preliminary cases related to the current problem from a plurality of candidate cases according to the target recall method includes: determining at least one keyword from the current question; Obtaining an overall relevance between the at least one keyword and each candidate case in the plurality of candidate cases; Based on the overall relevance, a second number of preliminary selected cases related to the current problem are recalled from the plurality of candidate cases.

5. The method according to claim 4, wherein: The obtaining of the overall relevance between the at least one keyword and each candidate case in the plurality of candidate cases includes: Taking each keyword of the at least one keyword as a target word, and obtaining the inverse document frequency of the target word in the plurality of candidate cases; Taking each candidate case of the plurality of candidate cases as a target case, and obtaining the vocabulary frequency of the target vocabulary in the target case; Based on the inverse document frequency and the word frequency, an overall relevance of the at least one keyword to the target case is obtained.

6. The method according to claim 1, wherein: The using of the large model to determine a target attribution algorithm for the current problem based on the first number of reference cases includes: Based on the first number of reference cases, generating initial prompt information; Supplementing the initial prompt information with the preset output requirement to obtain target prompt information; Utilizing the large model, based on the target prompt information, a target attribution algorithm for the current problem is determined.

7. The method according to any one of claims 1 to 6, wherein: The using the target attribution algorithm to obtain an attribution result for the current problem includes: Determine the actual occurrence time of the current problem based on the overall business data related to the current problem; Selecting target business data from the overall business data based on the actual generation time; The target attribution algorithm is utilized to obtain an attribution result for the current problem based on the target business data.

8. The method according to claim 7, wherein: The determining, based on the overall business data related to the current problem, the actual generation time of the current problem includes: Identify multiple change point detection algorithms; Using each of the multiple change point detection algorithms as a target detection algorithm, so as to obtain multiple candidate time points based on overall business data related to the current problem using the target detection algorithm; Based on the multiple candidate time points, the actual generation time of the current problem is determined.

9. The method according to claim 8, wherein: The plurality of change point detection algorithms include at least one of a Bayesian change point detection algorithm and an exact pruning linear time algorithm.

10. A problem attribution device based on a large model, comprising: a current problem determination unit, for determining a current problem of a target program; A reference case acquisition unit, configured to acquire a first number of reference cases related to the current problem; wherein each reference case in the first number of reference cases includes a reference history problem of the target program and a reference attribution algorithm for the reference history problem; an attribution algorithm acquisition unit, configured to determine a target attribution algorithm for the current problem based on the first number of reference cases using a large model; The attribution result acquisition unit is used to obtain the attribution result for the current problem by using the target attribution algorithm.

11. An electronic device, comprising: at least one processor; a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

13. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 9.

Images