Dialogue analysis method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to a dialogue analysis method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining dialogue content; respectively inputting the dialogue content into at least two analysis generative models, and determining corresponding initial analysis results; wherein each analysis generative model is obtained by constraining an initial generative model through different model prompt instructions with analysis logic; each model prompt instruction with the analysis logic is determined based on each analysis task corresponding to the dialogue content; the random degree of the analysis generative model is set to be non-random; and based on each initial analysis result, generating a target analysis result of the dialogue content. According to the embodiment of the invention, the accuracy of dialogue analysis and the consistency of multiple dialogue analysis for the same dialogue content can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a conversation analysis method, device, electronic device and storage medium. Background Art

[0002] At least the conversation between two parties can contain a lot of information, such as the intentions of the parties, whether there are problems with the products involved in the conversation, whether the problems of the parties are reasonably solved, etc. This information can provide good guidance for the improvement of products or services.

[0003] At present, a generative model of a certain scale can be used to analyze the content of a conversation to obtain the above information contained in the conversation content. However, there are problems such as insufficient consistency and low accuracy between the multiple analysis results output by the generative model for the same conversation content, resulting in poor accuracy of the above information obtained from the conversation content, thereby reducing the usability of the conversation information. Summary of the invention

[0004] In order to solve the above technical problems, the embodiments of the present disclosure provide a conversation analysis method, device, electronic device and storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a conversation analysis method, the method comprising:

[0006] Get the conversation content;

[0007] Input the conversation content into at least two analysis generation models respectively to determine the corresponding initial analysis results; wherein each of the analysis generation models is obtained by constraining the initial generation model through different model prompt instructions with analysis logic; each of the model prompt instructions with analysis logic is determined based on each analysis task corresponding to the conversation content; and the randomness of the analysis generation models is set to be non-random;

[0008] Based on each of the initial analysis results, a target analysis result of the conversation content is generated.

[0009] In a second aspect, the present disclosure also provides a conversation analysis device, the device comprising:

[0010] A conversation content acquisition module, used to acquire conversation content;

[0011] An initial analysis result determination module is used to input the conversation content into at least two analysis generation models respectively to determine the corresponding initial analysis results; wherein each of the analysis generation models is obtained by constraining the initial generation model through different model prompt instructions with analysis logic; each of the model prompt instructions with analysis logic is determined based on each analysis task corresponding to the conversation content; and the randomness of the analysis generation models is set to be non-random;

[0012] The target analysis result generating module is used to generate the target analysis result of the conversation content based on each of the initial analysis results.

[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0014] processor;

[0015] A memory for storing executable instructions;

[0016] The processor is used to read executable instructions from the memory and execute the executable instructions to implement the conversation analysis method described in any embodiment of the present disclosure.

[0017] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the conversation analysis method described in any embodiment of the present disclosure.

[0018] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, which is used to execute the conversation analysis method described in any embodiment of the present disclosure.

[0019] The conversation analysis method, device, electronic device and storage medium of the embodiments of the present disclosure can construct model prompt instructions with corresponding analysis logic in advance according to each analysis task, and use each model prompt instruction with analysis logic to perform model constraints on the initial pre-trained generative model (i.e., the initial generative model), so that an initial generative model evolves into each analysis generative model that performs conversation analysis from different analysis task perspectives; then, each non-random analysis generative model is used to analyze the conversation content respectively to obtain the corresponding initial analysis results; on the one hand, it can avoid the failure of conversation analysis or the error of conversation analysis results caused by a problem in a certain generative model, thereby improving the stability and accuracy of conversation analysis; on the other hand, it can perform conversation analysis from different analysis task perspectives to improve the comprehensiveness of conversation analysis; finally, the initial analysis results from each analysis task perspective can be combined to obtain the target analysis result, so that multiple conversation analyses for the same conversation content all conform to the same multi-angle analysis logic and comprehensive logic, thereby improving the accuracy and consistency of conversation analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0021] Figure 1 A flowchart of a conversation analysis method provided by an embodiment of the present disclosure;

[0022] Figure 2 A process diagram of a conversation analysis provided by an embodiment of the present disclosure;

[0023] Figure 3 A flowchart of another conversation analysis method provided by an embodiment of the present disclosure;

[0024] Figure 4 A schematic diagram of another process of conversation analysis provided by an embodiment of the present disclosure;

[0025] Figure 5 A structural diagram of a conversation analysis device provided by an embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0029] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0030] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0032] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0033] At present, the main solution for analyzing conversation content is to input the conversation content into a single generative model (such as a language model of a certain scale) and output the analysis results after the model analyzes it. However, because the generative model has learned a lot of knowledge, it is easy for a single generative model to have a large model hallucination, resulting in inconsistencies between multiple analysis results of the same conversation content and the problem of insufficient accuracy of the conversation analysis results.

[0034] Based on the above situation, the embodiment of the present disclosure provides a conversation analysis solution, which utilizes multiple non-random analysis generative models with analysis logic constraints to analyze the conversation content separately, obtains initial analysis results from the perspective of each analysis task, and improves the comprehensiveness of the analysis of the conversation content; then, the initial analysis results are integrated to obtain the final target analysis results, which improves the analysis accuracy of the conversation content and the consistency of the results obtained from multiple conversation analyses.

[0035] The conversation analysis method provided in the embodiment of the present disclosure can be applied to the scenario of conversation content analysis. The method can be performed by a conversation analysis device, which can be implemented by software and / or hardware, and the device can be integrated in an electronic device capable of running a generative model. The electronic device can include but is not limited to a smart phone, a personal digital assistant (PDA), a tablet computer (Tablet Personal Computer, Tablet PC), a laptop computer, a desktop computer, a vehicle terminal (such as a vehicle navigation terminal), a smart TV, a smart home device, etc.

[0036] Figure 1 FIG. 1 is a flow chart of a conversation analysis method provided by an embodiment of the present disclosure. Figure 1 As shown, the conversation analysis method may include the following steps:

[0037] S110: Obtain the conversation content.

[0038] The conversation content is interactive content generated between at least two parties, which can be interactive content between people, human-computer interactive content, automatic interactive content between machines, etc. The conversation content can be in text form, graphic form, or voice form, etc.

[0039] Specifically, the electronic device can obtain the conversation content to be analyzed in different ways.

[0040] In one example, the relevant person can input a complete conversation content into an electronic device where an application or webpage with a conversation analysis function is located. In this way, the electronic device can receive the conversation content.

[0041] In another example, the electronic device may provide an interactive control for a conversation analysis function. When a relevant person triggers the interactive control, the electronic device may determine a complete conversation content associated with the interactive control as the conversation content to be analyzed. For example, the interactive control is associated with the default example conversation content of the function demonstration, and the electronic device obtains the conversation content from the local storage space or the network end, and determines it as the conversation content to be analyzed. For another example, the interactive control is associated with the most recent or most frequently accessed conversation content, and the electronic device obtains the conversation content from the local storage space or the network end, and determines it as the conversation content to be analyzed.

[0042] S120, input the conversation content into at least two analysis generative models respectively to determine the corresponding initial analysis results; each analysis generative model is obtained by constraining the initial generative model through different model prompt instructions with analysis logic; each model prompt instruction with analysis logic is determined based on each analysis task corresponding to the conversation content; the randomness of the analysis generative models is set to non-random.

[0043] Among them, the analysis generative model is a generative model with a dialogue analysis function. The model prompt instruction is a text fragment or instruction used to guide / constrain the generative model to output the model result according to a specific task or topic. The model prompt instruction with analysis logic is a model prompt instruction that at least contains the analysis logic of dialogue analysis. The initial generative model is a pre-trained generative model that has learned a lot of knowledge from a large number of training samples. The analysis task is a dialogue analysis task determined based on business needs. For example, if the business need is to analyze the problems of various business lines related to the application, then the analysis task can be whether there are problems in each business line, what the specific problems are, etc. For another example, if the business need is to analyze students' doubts, then the analysis task can be to analyze various aspects related to students' learning, such as learning habits, wrong history questions, good subjects and learning shortcomings. The randomness of the model is a limiting parameter for the randomness of the model output results. In order to improve the consistency of the model output to a certain extent, the randomness is set to non-random, that is, each output is determined, not randomly changed. The initial analysis result is the result obtained by preliminary analysis of the dialogue content.

[0044] Specifically, a single generative model is prone to model hallucination, which makes the stability, consistency and accuracy of the model output results relatively poor. Therefore, in the embodiment of the present disclosure, according to each analysis task of the dialogue analysis, a corresponding model prompt instruction prompt with analysis logic is pre-constructed, and each model prompt instruction is input into the initial generative model respectively, and at least one of the operation function, operation process and output result of the initial generative model is constrained to limit its free divergence, but to expand around the corresponding analysis task to obtain an analysis generative model with different dialogue analysis functions. At the same time, the model randomness of each analysis generative model is set to non-random, so as to constrain the analysis generative model to output a fixed model result for the same dialogue content as much as possible. In this way, after the electronic device obtains the dialogue content, it can be input into each analysis generative model respectively to obtain the corresponding initial analysis result.

[0045] See also Figure 2 , the electronic device inputs the conversation content into analysis generation model 1, analysis generation model 2, analysis generation model 3, ... respectively. After the operation of each analysis generation model, the initial analysis result 1, initial analysis result 2, initial analysis result 3, ... obtained by performing conversation analysis from the perspective of the corresponding analysis task can be output.

[0046] In some embodiments, the model prompt instruction with analysis logic includes an analysis role prompt and an analysis key point prompt. The analysis role prompt is a prompt text used to specify the identity or role of the analysis generative model, for example, it can be information such as identity, personality, goal, emotion, etc., which can help the model better understand the context and situation of the dialogue content from the perspective of the specified role, limit the scope of model generation, and focus on the corresponding analysis task, thereby generating an initial analysis result that is more in line with the situation. The analysis key point prompt is a prompt text used to specify the core point of the dialogue analysis of the analysis generative model, which can constrain the model to perform dialogue analysis and result generation according to a predetermined logic, so that the model can adapt to the corresponding analysis task perspective and more efficiently analyze and obtain the required analysis results. The prompt designed in this way can make the analysis generative model more adaptable to the corresponding analysis task, improve the accuracy of the dialogue analysis of the analysis generative model and the consistency of the analysis logic, thereby further improving the consistency of the output results of the corresponding analysis generative model.

[0047] In other embodiments, the model prompt instruction with analysis logic includes an analysis role prompt, an analysis function prompt, an analysis key point prompt, and a first output format prompt. The analysis function prompt is a prompt text used to specify the function or capability of the analysis generative model. The analysis function prompt can further clarify the context of the model during the dialogue analysis on the basis of the analysis role prompt, and deepen the model's understanding of the dialogue content and the accuracy of understanding. The first output format prompt is a prompt text used to specify the output format of the analysis generative model, which can improve the standardization of the model output. For example, the first output format prompt can specify which fields are output by the model, what data structure these fields are output in, etc. Such a model prompt instruction can improve the model's focus on the analysis task from many aspects, so that the analysis generative model can analyze the dialogue content more deeply and accurately, and output more standardized and initial analysis results that fit the analysis task.

[0048] S130. Generate a target analysis result of the conversation content based on each initial analysis result.

[0049] Specifically, each initial analysis result is obtained by conducting conversation analysis from the perspective of different analysis tasks, which covers one aspect of conversation analysis. Therefore, in order to improve the accuracy and consistency of conversation analysis, see Figure 3, the initial analysis results can be summarized according to a certain comprehensive logic to obtain a final analysis result, namely the target analysis result. It can be seen that the target analysis result is obtained by considering the dialogue analysis results of many aspects. Compared with the dialogue analysis results of a single generative model, it is more comprehensive, more stable and more accurate. Moreover, the target analysis result is obtained according to the same analysis angle, analysis logic and comprehensive logic each time, so it also has better consistency.

[0050] In some embodiments, S130 includes: based on preset weights corresponding to the initial analysis results, synthesizing the initial analysis results to generate a target analysis result.

[0051] The preset weight is the weight value corresponding to the pre-set analysis task, which represents the proportion of the corresponding analysis task in the conversation analysis. For example, if a certain analysis task is more important in business needs, the preset weight of the analysis task can be set to a relatively large value.

[0052] Specifically, the electronic device can perform weighted aggregation on each initial analysis result, that is, recombining each initial analysis result according to the preset weight corresponding to each initial analysis result to obtain the target analysis result. For example, a corresponding proportion of key information can be extracted from the corresponding initial analysis result according to the preset weight, and then the key information can be organized into a smooth and fluent text as the target analysis result.

[0053] In other embodiments, S130 includes: inputting each initial analysis result into an inference generation model, and outputting a target analysis result.

[0054] Among them, the inference generative model is a generative model with judgment and reasoning functions, which can be obtained by constraining the initial generative model through a model prompt instruction with reasoning logic. The model prompt instruction with reasoning logic is a model prompt instruction prompt that at least contains reasoning logic for judging and reasoning multiple viewpoints. The randomness of the inference generative model is also set to non-random to further improve the consistency of the target analysis results obtained multiple times.

[0055] Specifically, the initial generative model has learned a lot of knowledge, including strong reasoning-related knowledge, so the initial generative model also has strong reasoning capabilities. Therefore, in the disclosed embodiment, a prompt with reasoning logic can be set and input into the initial generative model to constrain it to perform model operations and output results according to the reasoning logic, while efficiently utilizing the reasoning ability of the initial generative model, reducing human errors in the process of synthesizing the initial analysis results, and further improving the rationality and accuracy of the target analysis results.

[0056] In specific implementation, the electronic device inputs each initial analysis result into the inference generation model, and outputs the target analysis result after the model's inference operation. In this way, due to the constraints of the inference logic of the inference generation model, the logic of each initial analysis result is consistent, thereby improving the consistency of the target analysis results output multiple times.

[0057] It should be noted that in these embodiments, since the output results of each analysis generative model will be used as input data for the inference generative model, the model prompt instructions of each analysis generative model can be designed with corresponding first output format prompts according to the input requirements of the inference generative model to ensure the correct connection between each generative model and further improve the generation efficiency and accuracy of the target analysis results.

[0058] In some embodiments, the initial analysis results include analysis conclusions and analysis reasons; the model prompt instructions with reasoning logic include reasoning process prompts, which are generated by reasoning and analyzing each analysis conclusion and each analysis reason through judgment reasoning logic in a group discussion manner.

[0059] Specifically, in order to make full use of the reasoning function of the inference generation model and further improve the accuracy and consistency of the target analysis results, the initial analysis results in the disclosed embodiment can be designed to include not only the analysis conclusions, but also the reasons for obtaining the analysis results, that is, the analysis reasons, so as to provide sufficient reasoning basis. On this basis, the reasoning process prompt can be designed in the prompt with reasoning logic, and the reasoning process prompt can standardize the reasoning flow / ideas of the specific reasoning logic. For example, a reasoning example generated by judgment reasoning in a leaderless group discussion method can be provided as a reasoning process prompt. In this way, the reasoning generation model can learn the judgment reasoning logic and reasoning flow of the leaderless group discussion method, so as to synthesize the initial analysis results accordingly.

[0060] In some embodiments, the model prompt instruction with reasoning logic includes a reference weight corresponding to each analysis task, and the reference weight is used to constrain the proportion of the corresponding initial analysis result in the reasoning process. Each reference weight can be a preset fixed value, such as the preset weight mentioned above; it can also be a dynamically determined value, for example, it can be determined according to the correctness probability of the initial analysis result of each analysis task inferred from the reasoning generative model and the preset probability threshold. For example, when the business demand focuses on analysis task A and the probability threshold is set to 80%, if the probability of the initial analysis result of analysis task A being correct reaches 82% after the above reasoning logic, then the reference weight of analysis task A can be set to 1, that is, the initial analysis result of analysis task A is determined as the target analysis result. In this way, on the basis of using the reasoning model to more reasonably integrate the initial analysis results, the function of dynamically adjusting the emphasis ratio of each analysis task can be further added, thereby dynamically adjusting the emphasis direction of the analysis results for the same conversation content and improving the flexibility of conversation analysis.

[0061] In some embodiments, the model prompt instruction with reasoning logic also includes a reasoning role prompt. The reasoning role prompt is a prompt text used to specify the identity or role of the reasoning generative model. For example, it can be information such as identity, personality, goal, emotion, etc. It can help the model better understand the context and situation of each initial analysis result from the perspective of the specified role, limit the scope of model generation, and focus on judging the reasoning process, thereby generating a target analysis result that is more in line with the situation. In this way, through reasoning role prompts and reasoning process prompts, the reasoning generative model can be constrained to a large extent to output a more reasonable target analysis result according to a certain reasoning process.

[0062] In other embodiments, the model prompt instruction with reasoning logic also includes reasoning role prompts, reasoning function prompts, and second output format prompts. The reasoning function prompt is a prompt text used to specify the function or capability of the reasoning generative model. It can further clarify the context of the model when synthesizing the initial analysis results, and deepen the model's understanding of each initial analysis result and the accuracy of reasoning. The second output format prompt is a prompt text used to specify the output format of the reasoning generative model, which can improve the standardization of the model output. For example, the second output format prompt can specify which fields the model outputs, what data structure these fields are output in, etc. In view of the fact that the target analysis result is the final output result, it may include an analysis conclusion. Such a model prompt instruction can improve the model's understanding of each initial analysis result and the rationality of reasoning from many aspects, so that the reasoning generative model can judge and reason each initial analysis result more deeply and accurately, and output a more standardized, more accurate and more consistent target analysis result.

[0063] The conversation analysis method provided by the above-mentioned embodiments of the present disclosure can construct model prompt instructions with corresponding analysis logic in advance according to each analysis task, and use each model prompt instruction with analysis logic to perform model constraints on the initial pre-trained generative model (i.e., the initial generative model), so that an initial generative model evolves into each analysis generative model that performs conversation analysis from different analysis task perspectives; then each non-random analysis generative model is used to analyze the conversation content respectively to obtain the corresponding initial analysis results; on the one hand, it can avoid the situation where the conversation analysis fails or the conversation analysis result is wrong when a certain generative model has a problem, thereby improving the stability and accuracy of the conversation analysis; on the other hand, it can perform conversation analysis from different analysis task perspectives to improve the comprehensiveness of the conversation analysis; finally, the initial analysis results from each analysis task perspective can be combined to obtain the target analysis result, so that multiple conversation analyses for the same conversation content all conform to the same multi-angle analysis logic and comprehensive logic, thereby improving the accuracy and consistency of the conversation analysis.

[0064] Figure 3 A flow chart of another conversation analysis method provided by an embodiment of the present disclosure is shown. The conversation analysis method can analyze the conversation content generated between the online customer service personnel corresponding to the application and the user, so as to analyze from the perspectives of front-end technology, back-end technology, customer service and product design, obtain the analysis results of the corresponding perspectives, and perform reasoning and synthesis on them to obtain the final target analysis results, so as to provide improvement guidance / instruction for the above aspects.

[0065] See also Figure 3 , the conversation analysis method specifically includes the following steps:

[0066] S310: Obtain the content of a question-and-answer conversation between a user using the application and an online customer service staff.

[0067] Specifically, users may have some questions when using the application. In response to these questions, they can consult the online customer service staff to seek answers. In this process, a question-and-answer conversation will be generated between the user and the online customer service staff. By analyzing the content of the conversation, it can be known whether there are functional problems in technical research and development in the front end and / or back end of the application, whether there are defects in the product design of the application, and whether the customer service’s reply helps the user solve the questions, etc., so as to provide a basis for improvements in the corresponding aspects. Therefore, the electronic device can read the above conversation content from the local storage space or the external storage space.

[0068] S320. Input the conversation content into the generative model for analysis from the product design perspective, the generative model for analysis from the customer service perspective, the generative model for analysis from the front-end technology perspective, and the generative model for analysis from the back-end technology perspective, respectively, to determine the initial analysis results from the product design perspective, the initial analysis results from the customer service perspective, the initial analysis results from the front-end technology perspective, and the initial analysis results from the back-end technology perspective.

[0069] Specifically, the various analysis tasks involved in the above conversation content may include tasks such as analyzing whether the use of a certain function remains unchanged, the cost of understanding is too high, and it is unclear how to use it due to product design problems; tasks to analyze whether the customer service's response has solved the user's questions or calmed the user's emotions; tasks to analyze whether the questions raised by the user are related to the back-end code processing logic, such as tasks to analyze whether a certain function cannot be used, or a function can be used but the result is incorrect; tasks to analyze whether the questions raised by the user are related to front-end research and development, such as tasks to analyze whether the page is displayed normally, whether the page can interact normally, etc. According to these analysis tasks, corresponding model prompt instructions with analysis functions can be designed to obtain generative models for analysis from the perspective of product design, generative models for analysis from the perspective of customer service, generative models for analysis from the perspective of front-end technology, and generative models for analysis from the perspective of back-end technology. Then, refer to Figure 4 By inputting the conversation content into these generative models respectively, we can obtain the initial analysis results from the perspective of product design, the initial analysis results from the perspective of customer service, the initial analysis results from the perspective of front-end technology, and the initial analysis results from the perspective of back-end technology.

[0070] In some embodiments, the analysis role prompts of the generative model that performs analysis from the perspective of product design include role prompts for technical experts in the field of product managers, analysis key point prompts include analysis prompts for usage scenarios of product functions, analysis function prompts include function prompts for analyzing conversation content based on product usage scenarios, and the first output format prompts include format prompts for the analysis conclusion field, the analysis reason field, and the output structure of each field.

[0071] For example, the model prompt instructions for a generative model analyzing from a product design perspective may include the following:

[0072] ##Role

[0073] Field technical expert in natural language processing product manager

[0074] ##ability

[0075] Analyze the conversation content from the perspective of product usage scenarios

[0076] ##detail

[0077] Analysis key points: scenario analysis

[0078] -Bill checking scene

[0079] a. The amount notified to the customer is inconsistent with the amount seen on the page;

[0080] b. Inquire about where the refund or overpayment went;

[0081] c. The problem of not dividing the account into installments.

[0082] - Repeated deduction scenarios

[0083] a. Deducted money twice: The user clearly reported that the money was deducted too much, but the customer service did not give a reasonable explanation;

[0084] b. Money is deducted for no reason: Users are deducted money without their knowledge.

[0085] - Repayment failure scenario

[0086] **************;

[0087] -**Function cannot be used in scenarios

[0088] **************;

[0089] -Use quota scenario

[0090] a. The amount notified to the user is inconsistent with the amount seen on the page;

[0091] b. Temporary credit limit causes display inconsistency;

[0092] c. How to adjust the credit limit.

[0093] ## Output

[0094] {"result":"Analysis conclusion","reason":"Analysis reason"}.

[0095] In some embodiments, the analysis role prompts of the generative model that performs analysis from the customer service perspective include role prompts for technical experts in the field of customer service, analysis key point prompts include analysis prompts for classifying customer service response results, analysis function prompts include function prompts for analyzing conversation content from the customer service perspective, and the first output format prompts include format prompts for the analysis conclusion field, the analysis reason field, and the output structure of each field.

[0096] For example, the model prompt instructions for a generative model analyzing from a customer service perspective may include the following:

[0097] ##Role

[0098] Technical expert in natural language processing customer service

[0099] ##ability

[0100] Analyze the conversation from the customer service perspective

[0101] ##detail

[0102] Key points of analysis:

[0103] a. The basis for judging “invalid feedback” (e.g. too few rounds of dialogue / the number of user replies is lower than a certain threshold)

[0104] Set value, etc.);

[0105] b. The basis for judging “solved” (e.g., users’ feedback such as “OK” or “I understand”)

[0106] certain words, or the user does not give a negative response after the customer service provides a solution);

[0107] c. The basis for judging “users try to solve the problem by themselves” (e.g., users give feedback and try it themselves);

[0108] d. Judgment basis for “no human intervention” (e.g. after the user makes a request, the customer service replies that it is due to human intervention

[0109] unable to intervene and cannot be handled);

[0110] e. The basis for judging "incomplete information" (such as incomplete information provided by the user, or the key

[0111] information, etc.);

[0112] f. The basis for judging "follow-up resolution" (such as querying data based on the information provided by the user and responding to the user's questions);

[0113] g. Judgment basis for "providing specific operation links" (such as customer service replying to the user through ******

[0114] to view or operate);

[0115] h. The basis for judging “unresolved” (e.g. the user clearly indicates that he / she does not agree with the customer’s response);

[0116] ## Output

[0117] {"result":"Analysis conclusion","reason":"Analysis reason"}.

[0118] In some embodiments, the generative model analysis role prompts for analysis from the perspective of front-end technology include role prompts for field technical experts in the front-end research and development direction, analysis key point prompts include analysis prompts for various types of front-end display anomalies, analysis function prompts include function prompts for analyzing conversation content from the perspective of front-end technology, and the first output format prompts include format prompts for the analysis conclusion field, the analysis reason field, and the output structure of each field.

[0119] For example, the model prompt instructions for the generative model analyzed from the perspective of front-end technology may include the following:

[0120] ##Role

[0121] Technical experts in the field of natural language processing front-end

[0122] ##ability

[0123] Analyze the conversation content from the perspective of front-end technology to see if the user experience is hindered by issues related to the front-end page.

[0124] ##detail

[0125] Key points of analysis:

[0126] a. The user page is stuck and cannot be used;

[0127] b. The user page is blank and cannot be opened;

[0128] c. Always loading and network errors;

[0129] d. Display system abnormality;

[0130] e. Display system maintenance;

[0131] f. Other common front-end issues;

[0132] ## Output

[0133] {"result":"Front-end page technical problem","reason":"Analysis reason"}.

[0134] In some embodiments, the generative model analysis role prompts for analysis from the backend technology perspective include role prompts for domain technology experts in the backend R&D direction, analysis key point prompts include analysis prompts for various types of backend anomalies, analysis function prompts include function prompts for analyzing conversation content from the perspective of backend technology, and the first output format prompts include format prompts for the analysis conclusion field, the analysis reason field, and the output structure of each field.

[0135] For example, the model prompt instructions for the generative model analyzed from the backend technology perspective may include the following:

[0136] ##Role

[0137] Technical experts in the field of natural language processing backend

[0138] ##ability

[0139] Analyze the conversation content from the perspective of backend technology to see if there are problems with the database or logic code processing that hinder user usage;

[0140] ##detail

[0141] Key points of analysis:

[0142] a. The user still cannot use it after changing the network, re-downloading, or exiting and re-entering;

[0143] b. User feedback shows that the page is under system maintenance;

[0144] c. It may be caused by database storage management;

[0145] d. Issues related to SMS verification codes;

[0146] e. The notification received by the user is inconsistent with the content seen;

[0147] ## Output

[0148] {"result":"Backend technical issues","reason":"Analysis reasons"}.

[0149] S330, input the initial analysis results from the product design perspective, the initial analysis results from the customer service perspective, the initial analysis results from the front-end technology perspective, and the initial analysis results from the back-end technology perspective into the inference generation model, and output the target analysis results.

[0150] Specifically, see Figure 4 The electronic device can input the initial analysis results from the product design perspective, the initial analysis results from the customer service perspective, the initial analysis results from the front-end technology perspective, and the initial analysis results from the back-end technology perspective obtained above into the reasoning generative model and output the target analysis results.

[0151] According to the description of the above embodiments, the model prompt instruction with reasoning logic can be designed by prompting an example of the reasoning process. For example, the model prompt instruction of the reasoning generation model may include the following content:

[0152] ##Role

[0153] Reasoning and judgment expert

[0154] ##ability

[0155] Find the most logical conclusion from the different analysis processes provided (such as the above analysis conclusions and reasons);

[0156] ##Example

[0157] 1. Input information:

[0158] User: 'The repayment system shows abnormality'. Agent: 'Do you have a screenshot?' Agent: 'Customer service will verify it for you'. User: 'Picture'. User: 'What's going on'. Agent: 'Do you have a screen recording? Can you provide it?'. User: 'How to record the screen?' Agent: 'Dear, does your phone have a screen recording function?'. User: 'Take a look'. User: '? ? ? '. Agent: 'OK, dear'. Agent: 'Please wait a moment'. Agent: 'Dear, is your TikTok the latest version?'. Agent: 'You can check in the App Store to see if it needs to be updated.'. User: 'I'll try it'. Agent: 'OK, dear, sorry to bother you.'. User: 'There is no prompt to update.' Agent: 'Dear, please try other payment methods to see if the same situation occurs.'

[0159] 2. Thinking process:

[0160] In the first step, each expert summarizes his or her own views:

[0161] Expert A: The user emphasized that the repayment display system was abnormal, and result = "front-end page problem";

[0162] Expert B: The user said that he saw a system anomaly. There might be a backend bug. Result = "backend bug problem";

[0163] Expert C: After giving the user suggestions for trying, the user agrees to try, result = "user tries to solve";

[0164] Expert D: Unable to distinguish what scene it is, any scene may exist, result = "Other - Unable to determine what scene";

[0165] Step 2: Group Discussion

[0166] Expert A believes that the user's real intention is to emphasize a page problem. Expert B believes that the user sees page feedback, but the underlying error must be caused by an error in the backend interface, which leads to an error on the frontend page, so it should be a backend bug. Expert C believes that the user has tried to solve it in the end. Expert D has no opinion because he can't judge what the scenario is. Expert B believes that expert A's opinion is not deep enough and does not touch the real underlying cause. Expert B believes that although expert C's opinion is correct, this situation should be checked. Experts A, C, and D believe that expert B's analysis is more in-depth and representative, so expert B's opinion is selected as the final conclusion. "}

[0167] ## Output

[0168] {"result":"Backend technical issue"}.

[0169] It should be noted that the reference weights corresponding to each analysis task and the relevant text content on how to infer based on the reference weights can be added before or during "Step 2: Group Discussion".

[0170] The following is an embodiment of a conversation analysis device provided in an embodiment of the present invention. The device and the conversation analysis methods of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the conversation analysis device, reference can be made to the embodiment of the above-mentioned conversation analysis method.

[0171] Figure 5 FIG. 2 shows a schematic diagram of the structure of a conversation analysis device provided by an embodiment of the present disclosure. Figure 5 As shown, the conversation analysis device 500 may include:

[0172] A conversation content acquisition module 510 is used to acquire the conversation content;

[0173] The initial analysis result determination module 520 is used to input the conversation content into at least two analysis generation models respectively to determine the corresponding initial analysis results; wherein each analysis generation model is obtained by constraining the initial generation model through different model prompt instructions with analysis logic; each model prompt instruction with analysis logic is determined based on each analysis task corresponding to the conversation content; and the randomness of the analysis generation model is set to non-random;

[0174] The target analysis result generating module 530 is used to generate a target analysis result of the conversation content based on each initial analysis result.

[0175] The conversation analysis device provided by the above-mentioned embodiments of the present disclosure can construct model prompt instructions with corresponding analysis logic in advance according to each analysis task, and use each model prompt instruction with analysis logic to perform model constraints on the initial pre-trained generative model (i.e., the initial generative model), so that an initial generative model evolves into each analysis generative model that performs conversation analysis from different analysis task perspectives; then each non-random analysis generative model is used to analyze the conversation content respectively to obtain the corresponding initial analysis results; on the one hand, it can avoid the situation where the conversation analysis fails or the conversation analysis result is wrong when a problem occurs in a certain generative model, thereby improving the stability and accuracy of the conversation analysis; on the other hand, it can perform conversation analysis from different analysis task perspectives to improve the comprehensiveness of the conversation analysis; finally, the initial analysis results from each analysis task perspective can be combined to obtain the target analysis result, so that multiple conversation analyses for the same conversation content all conform to the same multi-angle analysis logic and comprehensive logic, thereby improving the accuracy and consistency of the conversation analysis.

[0176] In some embodiments, the target analysis result generation module 530 is specifically used to:

[0177] Input each initial analysis result into the inference generative model, and output the target analysis result; wherein the inference generative model is obtained by constraining the initial generative model through a model prompt instruction with inference logic; and the randomness of the inference generative model is set to be non-random.

[0178] Furthermore, the initial analysis results include analysis conclusions and analysis reasons;

[0179] The model prompt instructions with reasoning logic include reasoning process prompts; the reasoning process prompts are generated by reasoning and analyzing each analysis conclusion and each analysis reason through judgment reasoning logic in the form of group discussion.

[0180] In some embodiments, the model prompt instruction with reasoning logic includes a reference weight corresponding to each analysis task, and the reference weight is used to constrain the proportion of the corresponding initial analysis result in the reasoning process.

[0181] In some other embodiments, the target analysis result generating module 530 is specifically used to:

[0182] Based on the preset weights corresponding to the initial analysis results, the initial analysis results are combined to generate the target analysis results.

[0183] In some embodiments, the model prompt instruction with analysis logic includes analysis role prompt and analysis key point prompt; or, the model prompt instruction with analysis logic includes analysis role prompt, analysis function prompt, analysis key point prompt and first output format prompt.

[0184] In some embodiments, the model prompt instruction with reasoning logic further includes a reasoning role prompt; or, the model prompt instruction with reasoning logic further includes a reasoning role prompt, a reasoning function prompt, and a second output format prompt.

[0185] In some embodiments, the conversation content is a question and answer content generated between an online customer service staff corresponding to the application and the user;

[0186] The analysis generative models include a generative model for analysis from the perspective of product design, a generative model for analysis from the perspective of customer service, a generative model for analysis from the perspective of front-end technology, and a generative model for analysis from the perspective of back-end technology;

[0187] The analysis role prompts of the generative model for analysis from the perspective of product design include the role prompts of technical experts in the field of product managers, the analysis key point prompts include the analysis prompts of the usage scenarios of product functions, the analysis function prompts include the function prompts of analyzing the content of the dialogue from the perspective of product usage scenarios, and the first output format prompts include the format prompts of the analysis conclusion field, the analysis reason field and the output structure of each field;

[0188] The analysis role prompts of the generative model for analysis from the customer service perspective include the role prompts of the technical experts in the customer service field, the analysis key point prompts include the analysis prompts for the classification of the customer service reply results, the analysis function prompts include the function prompts for analyzing the conversation content from the customer service perspective, and the first output format prompts include the format prompts of the analysis conclusion field, the analysis reason field and the output structure of each field;

[0189] The generative model analysis role prompts for analyzing from the perspective of front-end technology include the role prompts of the technical experts in the field of front-end R&D direction, the analysis key point prompts include the analysis prompts of various front-end display anomalies, the analysis function prompts include the function prompts for analyzing the conversation content from the perspective of front-end technology, and the first output format prompts include the format prompts of the analysis conclusion field, the analysis reason field and the output structure of each field;

[0190] The generative model analysis role prompts for analysis from the perspective of backend technology include role prompts for field technical experts in the backend R&D direction, analysis key point prompts include analysis prompts for various types of backend anomalies, analysis function prompts include function prompts for analyzing conversation content from the perspective of backend technology, and the first output format prompts include format prompts for the analysis conclusion field, analysis reason field, and the output structure of each field.

[0191] The conversation analysis device provided in the embodiment of the present invention can execute the conversation analysis method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0192] It is worth noting that in the embodiment of the above-mentioned conversation analysis device, the various modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the various functional modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present disclosure.

[0193] The disclosed embodiment also provides an electronic device, which may include a processor and a memory, wherein the memory may be used to store executable instructions. The processor may be used to read the executable instructions from the memory and execute the executable instructions to implement the conversation analysis method in any of the above embodiments.

[0194] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0195] like Figure 6 As shown, the electronic device 600 may include a processing device 601 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output interface (I / O interface) 605 is also connected to the bus 604.

[0196] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data.

[0197] It should be noted that Figure 6 The electronic device 600 shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure. Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0198] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the conversation analysis method of any embodiment of the present disclosure are executed.

[0199] The embodiments of the present disclosure also provide a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the conversation analysis method in any embodiment of the present disclosure.

[0200] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0201] In some embodiments, the client and server may communicate using any currently known or future developed network protocol such as HTTP, and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0202] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0203] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the steps of the conversation analysis method described in any embodiment of the present disclosure.

[0204] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0205] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0206] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0207] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0208] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0209] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0210] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A conversation analysis method, characterized in that: include: Get the conversation content; Input the conversation content into at least two analysis generation models respectively to determine the corresponding initial analysis results; wherein each of the analysis generation models is obtained by constraining the initial generation model through different model prompt instructions with analysis logic; each of the model prompt instructions with analysis logic is determined based on each analysis task corresponding to the conversation content; and the randomness of the analysis generation models is set to be non-random; Based on each of the initial analysis results, a target analysis result of the conversation content is generated.

2. The method according to claim 1, characterized in that The generating a target analysis result of the conversation content based on each of the initial analysis results includes: Input each of the initial analysis results into an inference generative model, and output the target analysis result; wherein the inference generative model is obtained by constraining the initial generative model through a model prompt instruction with inference logic; and the randomness of the inference generative model is set to non-random.

3. The method according to claim 2, characterized in that The initial analysis results include analysis conclusions and analysis reasons; The model prompt instruction with reasoning logic includes a reasoning process prompt; the reasoning process prompt is generated by reasoning and analyzing each analysis conclusion and each analysis reason through judgment reasoning logic in a group discussion manner.

4. The method according to claim 2 or 3, characterized in that: The model prompt instruction with reasoning logic includes a reference weight corresponding to each of the analysis tasks, and the reference weight is used to constrain the proportion of the corresponding initial analysis result in the reasoning process.

5. The method according to claim 1, characterized in that The generating a target analysis result of the conversation content based on each of the initial analysis results includes: Based on the preset weights corresponding to the initial analysis results, the target analysis results are generated by integrating the initial analysis results.

6. The method according to claim 1, characterized in that The model prompt instruction with analysis logic includes analysis role prompt and analysis key point prompt; or, the model prompt instruction with analysis logic includes the analysis role prompt, analysis function prompt, the analysis key point prompt and the first output format prompt.

7. The method according to claim 3, characterized in that The model prompt instruction with reasoning logic also includes a reasoning role prompt; or, the model prompt instruction with reasoning logic also includes the reasoning role prompt, the reasoning function prompt and the second output format prompt.

8. The method according to claim 6, characterized in that The conversation content is the question and answer content generated between the online customer service personnel corresponding to the application and the user; Each of the analysis generative models includes a generative model for analysis from a product design perspective, a generative model for analysis from a customer service perspective, a generative model for analysis from a front-end technology perspective, and a generative model for analysis from a back-end technology perspective.

9. A conversation analysis device, characterized in that: include: A conversation content acquisition module, used to acquire conversation content; An initial analysis result determination module is used to input the conversation content into at least two analysis generation models respectively to determine the corresponding initial analysis results; wherein each of the analysis generation models is obtained by constraining the initial generation model through different model prompt instructions with analysis logic; each of the model prompt instructions with analysis logic is determined based on each analysis task corresponding to the conversation content; and the randomness of the analysis generation models is set to be non-random; The target analysis result generating module is used to generate the target analysis result of the conversation content based on each of the initial analysis results.

10. An electronic device, characterized in that: include: processor; A memory for storing executable instructions; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the conversation analysis method described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the conversation analysis method described in any one of claims 1 to 8.

Images