Method and system for analyzing and judging similarity

Abstract

The invention provides a method and a system for analyzing and judging similarity. The method for analyzing and judging similarity comprises the following steps of: acquiring a typical story line included by a robot customer and an occurrence probability of the typical story line; and judging the similarity of the robot customers by using the typical story line and the occurrence probability of the typical story line. According to the technical scheme, a typical story line included by a robot customer and the occurrence probability of the typical story line are obtained; the similarity of the robot customers is judged according to the typical story line and the occurrence probability of the typical story line; the problem that whether robot clients are close or not can only be judged manually in the prior art is solved, manpower needed by research and development of the training system can be saved, and dispute about the number of delivered robots when the system delivers the users can be avoided.

Description

Background Art

[0002] In the field of telephone sales and telephone customer service, agents are the key factor in the operation and development of enterprises, and it is very important to do a good job in agent training. In the early days, agent training was completely manual, and the training content was consolidated and the training effect was tested through training between agents and coaches, and between agents. With the development of computer technology and artificial intelligence, various forms of training systems have emerged to help agents memorize training materials and help companies evaluate the effectiveness of agent training.

[0003] A core function of the training system is to simulate customers and simulate sales or service processes. Specifically, the "customer" asks questions, the agent answers the questions, and the "customer" asks new questions based on the agent's answer. When there are a large number of robot customers in the training system, it is very important to determine whether the robots are similar enough to be considered as one customer, both for designing robot customers and delivering products to users. If you rely on R&D personnel or end users to determine whether robots are similar through repeated experiments, it will require a lot of manpower and time, which is not feasible in actual practice. Summary of the invention

[0004] The present application provides a method and system for analyzing and judging similarity, so as to solve the problem that previously only human judgment could be made on whether robot customers were similar.

[0005] In a first aspect, a method for analyzing and judging similarity is provided, comprising the following steps:

[0006] Obtaining typical story lines included in the robot client and the probability of occurrence of the typical story lines;

[0007] The similarity of robot clients is judged using the typical story lines and the occurrence probabilities of the typical story lines.

[0008] In the above technical scheme, by obtaining the typical story lines contained in the robot client and the probability of occurrence of the typical story lines; using the typical story lines and the probability of occurrence of the typical story lines, the similarity of the robot client is judged; the problem that previously only manual judgment of whether robot clients are similar was possible is solved. On the one hand, it can save the manpower required for the research and development of the training system, and on the other hand, it can avoid disputes about the number of delivered robots when the system is delivered to users.

[0009] In a specific embodiment, it also includes:

[0010] The robot client is constructed by using a topic series method and a large model.

[0011] In a specific implementation scheme, the steps of constructing the robot client using a large model in a topic series manner specifically include:

[0012] Distill all the conversations between agents and customers in real scenarios to get the theme;

[0013] The prompt words are arranged using a large model, and a topic-connected method is adopted to conduct a natural human-computer dialogue under the topic to construct the robot client.

[0014] In a specific implementation scheme, the step of obtaining the typical story lines included in the robot client and the probability of occurrence of the typical story lines specifically includes:

[0015] Vectorizing the topic to obtain a vectorized topic;

[0016] Using the robot client to practice the vectorized topic to obtain a set of story lines;

[0017] Vectorizing the story lines in the story line set to obtain vectorized story lines;

[0018] Merging the vectorized story lines to obtain the typical story line;

[0019] The typical story line is practiced using the robot client to obtain the occurrence probability of the typical story line.

[0020] In a specific implementation scheme, the step of vectorizing the subject to obtain the vectorized subject is specifically as follows:

[0021] The topics are grouped and their similarities are marked, and the vectorized topics are obtained by using a word vector generation technique.

[0022] In a specific implementation scheme, the step of vectorizing the story lines in the story line set to obtain the vectorized story lines is specifically as follows:

[0023] According to the order of the themes included in the story line, the vectors corresponding to the themes are spliced ​​in sequence to obtain a concatenation result as the vector expression of the story line, thereby obtaining the vectorized story line.

[0024] In a specific implementation scheme, the step of merging the vectorized story lines to obtain the typical story line specifically includes:

[0025] Calculating the similarity of the vectorized story lines;

[0026] The same vectorized story lines are grouped according to the similarity, and a representative is selected from each group as the typical story line.

[0027] In a specific implementation scheme, the step of using the robot client to practice the typical story line to obtain the probability of occurrence of the typical story line is specifically as follows:

[0028] The typical story lines are practiced using the robot client, the occurrence frequencies of each of the typical story lines are counted and recorded, and the occurrence probabilities of the typical story lines are obtained.

[0029] In a specific implementation scheme, the step of judging the similarity of robot clients by using the typical story line and the probability of occurrence of the typical story line includes:

[0030] If the typical story lines contained in two robot clients are the same, and the probability of occurrence of the same typical story lines is similar, then it is determined that the two robot clients are similar.

[0031] In the second aspect, a system for analyzing and judging similarity is provided, including:

[0032] A recording module, used to obtain typical story lines included in the robot client and the probability of occurrence of the typical story lines;

[0033] The judgment module is used to judge the similarity of robot clients by using the story lines and the occurrence probabilities of the story lines.

[0034] In the above technical scheme, by obtaining the typical story lines contained in the robot client and the probability of occurrence of the typical story lines; using the typical story lines and the probability of occurrence of the typical story lines, the similarity of the robot client is judged; the problem that previously only manual judgment of whether robot clients are similar was possible is solved. On the one hand, it can save the manpower required for the research and development of the training system, and on the other hand, it can avoid disputes about the number of delivered robots when the system is delivered to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flowchart of a method for analyzing and judging similarity provided in an embodiment of the present application;

[0036] Figure 2 A structural block diagram of a system for analyzing and judging similarity provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of the theme and storyline structure of the method for analyzing and judging similarity provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The present application is further described in detail below through the accompanying drawings and embodiments. Through these descriptions, the characteristics and advantages of the present application will become clearer and more specific.

[0039] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.

[0040] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0041] To facilitate understanding of the method and system for analyzing and judging similarity provided in the embodiment of the present application, its application scenario is first explained. The method and system for analyzing and judging similarity provided in the embodiment of the present application are used to solve the problem that previously only human judgment could be made on whether robot customers were similar. In the field of telephone sales and telephone customer service, agents are key factors in the operation and development of enterprises, and it is very important to do a good job in agent training. Early agent training relied entirely on manual work, and the training content was consolidated and the training effect was tested through training between agents and coaches and between agents. With the development of computer technology and artificial intelligence, various forms of training systems have emerged to help agents memorize training materials and help companies assess the training effect of agents. A core function of the training system is to simulate customers and simulate sales or service processes, specifically "customers" ask questions, agents answer questions, and "customers" ask new questions based on the answers of agents. When there are a large number of robot customers in the training system, it is very important to judge whether the robots are similar to each other so that they can be considered as one customer, which is very important for designing robot customers and delivering products to users. If it depends on R&D personnel or end users to judge whether robots are similar through repeated experiments, it will require a lot of manpower and consume a lot of time, which is not feasible in actual practice. To this end, the present application provides a method and system for analyzing and judging similarity, so as to solve the problem that previously only human judgment can be made on whether robot customers are similar. The following is a detailed description of the method and system with reference to specific figures.

[0042] refer to Figures 1 to 3 , Figure 1 A flowchart of a method for analyzing and judging similarity provided in an embodiment of the present application; Figure 2 A structural block diagram of a system for analyzing and judging similarity provided in an embodiment of the present application; Figure 3 A schematic diagram of the theme and storyline structure of the method for analyzing and judging similarity provided in an embodiment of the present application.

[0043] exist Figure 1 In the embodiment of the present application, a method for analyzing and judging similarity is provided, comprising the following steps:

[0044] Obtaining typical story lines included in the robot client and the probability of occurrence of the typical story lines;

[0045] The similarity of robot clients is judged using the typical story lines and the occurrence probabilities of the typical story lines.

[0046] In the above technical scheme, by obtaining the typical story lines contained in the robot client and the probability of occurrence of the typical story lines; using the typical story lines and the probability of occurrence of the typical story lines, the similarity of the robot client is judged; the problem that previously only manual judgment of whether robot clients are similar was possible is solved. On the one hand, it can save the manpower required for the research and development of the training system, and on the other hand, it can avoid disputes about the number of delivered robots when the system is delivered to users.

[0047] In a specific embodiment, it also includes:

[0048] The robot client is constructed by using a topic series method and a large model.

[0049] In a specific implementation scheme, the steps of constructing the robot client using a large model in a topic series manner specifically include:

[0050] Distill all the conversations between agents and customers in real scenarios to get the theme;

[0051] The prompt words are arranged using a large model, and a topic-connected method is adopted to conduct a natural human-computer dialogue under the topic to construct the robot client.

[0052] In a specific implementation scheme, the step of obtaining the typical story lines included in the robot client and the probability of occurrence of the typical story lines specifically includes:

[0053] Vectorizing the topic to obtain a vectorized topic;

[0054] Using the robot client to practice the vectorized topic to obtain a set of story lines;

[0055] Vectorizing the story lines in the story line set to obtain vectorized story lines;

[0056] Merging the vectorized story lines to obtain the typical story line;

[0057] The typical story line is practiced using the robot client to obtain the occurrence probability of the typical story line.

[0058] In a specific implementation scheme, the step of vectorizing the subject to obtain the vectorized subject is specifically as follows:

[0059] The topics are grouped and their similarities are marked, and the vectorized topics are obtained by using a word vector generation technique.

[0060] In a specific implementation scheme, the step of vectorizing the story lines in the story line set to obtain the vectorized story lines is specifically as follows:

[0061] According to the order of the themes included in the story line, the vectors corresponding to the themes are spliced ​​in sequence to obtain a concatenation result as the vector expression of the story line, thereby obtaining the vectorized story line.

[0062] In a specific implementation scheme, the step of merging the vectorized story lines to obtain the typical story line specifically includes:

[0063] Calculating the similarity of the vectorized story lines;

[0064] The same vectorized story lines are grouped according to the similarity, and a representative is selected from each group as the typical story line.

[0065] In a specific implementation scheme, the step of using the robot client to practice the typical story line to obtain the probability of occurrence of the typical story line is specifically as follows:

[0066] The typical story lines are practiced using the robot client, the occurrence frequencies of each of the typical story lines are counted and recorded, and the occurrence probabilities of the typical story lines are obtained.

[0067] In a specific implementation scheme, the step of judging the similarity of robot clients by using the typical story line and the probability of occurrence of the typical story line includes:

[0068] If the typical story lines contained in two robot clients are the same, and the probability of occurrence of the same typical story lines is similar, then it is determined that the two robot clients are similar.

[0069] Specifically, refer to Figure 3 , the system for analyzing and judging similarity includes:

[0070] Step 1: Use the big model to simulate and build a robot customer. The content that a human or a machine might talk about is divided into topics, and boosting words are written for each topic so that the big model can accurately and smoothly talk to the customer about the topic. The topics are connected according to the real customer behavior, and different robot customers use different topics and connection methods.

[0071] Furthermore, the steps of building a robot client using a large model simulation include:

[0072] Step 11: Gather the conversations between the agent and the customer and divide them into segments according to the specific content discussed;

[0073] Step 12: Summarize the discussion content of each segment and organize it into themes; label each segment according to the discussion content;

[0074] Step 13. If a segment discusses multiple topics, the first option is to continue to divide the segment into multiple segments until each segment has only a unique topic label; the second option is to merge multiple topics into one topic.

[0075] Taking telephone auto insurance as an example, the content of a call between the agent and the customer may be opening remarks, quotation, price objection, and issuance of a policy, while the content of another call may be opening remarks, quotation, insurance type objection, and issuance of a policy. For plan 1, the third call may be opening remarks, quotation, [insurance type objection + price objection], and issuance of a policy. According to the above steps 1, 2, and 3, we first summarize the five themes of opening remarks, quotation, insurance type objection, price objection, and issuance of a policy. Then the segment [insurance type objection + price objection] will be divided into two, namely insurance type objection and price objection. For plan 2, the third call may be opening remarks, quotation, [objection], and issuance of a policy. According to the above steps 11, 12, and 13, we first summarize the five themes of opening remarks, quotation, insurance type objection, price objection, and issuance of a policy. Then the themes of insurance type objection and price objection will be merged into one theme objection.

[0076] Step 14. Select a suitable big model, write prompt words, and let the big model act out a customer dialogue within the scope of the topic; the output results of the big model include 1) the reply to the student 2) whether the topic is over 3) the student's speech evaluation 4) the summary of the conversation content.

[0077] Step 15: Determine the real customer information of the robot customer as needed, including but not limited to 1) personnel information, age, gender, personality, etc.; 2) vehicle information, license plate, car price, etc.; 3) customer group information (customer portrait), service sensitivity, price sensitivity, etc. Select the possible topics to be discussed according to the real customer information, and arrange the topic connection relationship according to the real customer behavior.

[0078] Step 16. For the multiple calls in step 15, arrange the topics of each call according to the characteristics of each call; write prompt words for the call, use the big model to analyze the call and output the call summary and the student's speech evaluation; the summary and evaluation are used to select the topic that the robot will talk about in the next call.

[0079] Step 17. For the arrangement of topics described in step 15, a fixed order of topics is not set, but a dynamic connection rule. For example, in response to "price objection", the agent may say that an additional roadside assistance value-added service is provided, or a free glass insurance service is provided; after the price objection topic ends, the conversation content summary will include the agent's handling of the objection information. For the roadside assistance increase, the subsequent connection is to topics related to roadside assistance, such as the robot saying that it does not need roadside assistance; for the free glass insurance, the subsequent connection is to topics related to glass insurance, such as the robot asking what brand of glass it is.

[0080] In the above technical solution, the robot customer's degree of anthropomorphism far exceeds that of the existing training system, which is reflected in its strong ability to understand trainees, rich and natural responses to trainees, strong interactive ability, and the ability to decide whether to place an order or reject it based on the trainee's performance. It can simulate real customers and customer groups, allowing agents to practice their skills in actual combat, reducing training costs and enhancing training effectiveness.

[0081] It should be noted that Large Language Model (LLM) is one of the hottest topics in the field of artificial intelligence in recent years. It represents the latest progress in natural language processing (NLP) and demonstrates unprecedented ability to understand and generate human language. The following is an explanation of the Large Language Model:

[0082] 1. Definition and Principle

[0083] A big language model is a deep learning model trained with a large amount of text data that can understand and generate human language. It is based on the Transformer architecture and can learn complex patterns of language and complete various language tasks. Its core goal is to accurately learn and understand human language, enabling machines to interpret language like humans, completely changing the way computers understand and generate human language.

[0084] 2. Features and advantages

[0085] Large-scale data training: LLM is trained using massive amounts of text data, typically millions or even billions of words, thereby gaining a deep understanding of language.

[0086] Strong language understanding capabilities: Ability to understand language nuances, semantic relationships, and contextual information, which is the basis for generating coherent and contextually relevant text.

[0087] Flexible application scenarios: It can be applied to a variety of language tasks, such as text generation, sentiment analysis, question-answering systems, automatic summarization, machine translation, document classification, etc., showing strong versatility.

[0088] Emergent capabilities: As the model size increases, LLM demonstrates some amazing capabilities, especially showing potential huge advantages in solving complex tasks.

[0089] Step 2: Vectorize all topics, group them into pairs and manually annotate their similarity (preferably 1 to 5), and use existing word vector generation technology to obtain each topic vector (preferably 8 or 16 dimensions)

[0090] Step 3. Create a program (which can be considered as a virtual student) to simulate exercises with the robot customer and obtain the actual dialogue results of the robot customer. This dialogue result can be regarded as a story line composed of a series of topics; practice with the robot customer a large number of times, such as 10,000 times, to obtain a set of possible story lines for the robot customer.

[0091] Step 4: Vectorize all story lines. In the order of story line themes, concatenate the corresponding vectors of the themes in turn, and obtain the result as the vector expression of the story line. For story lines of different lengths, use a special theme "filler theme" to fill in the longest story line. Train a matrix to compress the story line vector, compressing the dimension from N to M (N is much larger than M). The matrix training method is similar to step 2.

[0092] Step 5: Merge story lines and use existing technology to calculate the story line similarity (compressed story vector), such as cosine similarity; story lines with similarity greater than a threshold are considered the same. If cosine similarity is used, the threshold is preferably between 0.7 and 1.0. Group the same story lines into one group, and select a typical story line from each group as a representative.

[0093] Step 6: Use the same method as in step 3 to practice with each robot client a certain number of times, such as 10,000 times, and count and record the frequency of each typical story line. For example, robot Zhang's story line 1 appears 5,000 times (probability 0.5), story line 2 appears 5,000 times (probability 0.5); robot Li's story line 2 appears 5,000 times (probability 0.5), story line 3 appears 3,000 times (probability 0.3), and story line 4 appears 2,000 times (probability 0.2).

[0094] Step 7: If the two robots contain the same typical story lines and the probability of the same story lines appearing is similar, then the two robots are judged to be similar and the settings of the two robots need to be adjusted.

[0095] In the above technical solution, a method for judging whether robot customers in the training system are similar is proposed, which solves the problem that the previous method can only judge whether robot customers are similar. On the one hand, it can save manpower required for the development of training systems, and on the other hand, it can avoid disputes about the number of delivered robots when the system is delivered to users.

[0096] exist Figure 2 In the embodiment of the present application, a system for analyzing and judging similarity is provided, including:

[0097] A recording module, used to obtain typical story lines included in the robot client and the probability of occurrence of the typical story lines;

[0098] The judgment module is used to judge the similarity of robot clients by using the story lines and the occurrence probabilities of the story lines.

[0099] In the above technical scheme, by obtaining the typical story lines contained in the robot client and the probability of occurrence of the typical story lines; using the typical story lines and the probability of occurrence of the typical story lines, the similarity of the robot client is judged; the problem that previously only manual judgment of whether robot clients are similar was possible is solved. On the one hand, it can save the manpower required for the research and development of the training system, and on the other hand, it can avoid disputes about the number of delivered robots when the system is delivered to users.

[0100] Those skilled in the art will appreciate that the present application may be implemented as a system, method or computer program product.

[0101] Therefore, the present disclosure may be specifically implemented in the following forms, namely: it may be completely hardware, it may be completely software (including firmware, resident software, microcode, etc.), or it may be a combination of hardware and software, generally referred to herein as a "circuit", "module" or "system". In addition, in some embodiments, the present application may also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium may contain computer-readable program code.

[0102] Any combination of one or more computer-readable media can be used. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0103] Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art can change, modify, replace and modify the above embodiments within the scope of the present application. On this basis, a variety of replacements and improvements can be made to the present application, all of which fall within the scope of protection of the present application.

Claims

1. A method for analyzing and judging similarity, characterized in that: The following steps are involved: Obtaining typical story lines included in the robot client and the probability of occurrence of the typical story lines; The similarity of robot clients is judged using the typical story lines and the occurrence probabilities of the typical story lines.

2. The method for analyzing and judging similarity according to claim 1, characterized in that: Also includes: The robot client is constructed by using a topic series method and a large model.

3. The method for analyzing and judging similarity according to claim 2, characterized in that: The steps of constructing the robot client using a large model by using a topic series method specifically include: Distill all the conversations between agents and customers in real scenarios to get the theme; The prompt words are arranged using a large model, and a topic-connected method is adopted to conduct a natural human-computer dialogue under the topic to construct the robot client.

4. The method for analyzing and judging similarity according to claim 3, characterized in that: The step of obtaining typical story lines included in the robot client and the probability of occurrence of the typical story lines specifically includes: Vectorizing the topic to obtain a vectorized topic; Using the robot client to practice the vectorized topic to obtain a set of story lines; Vectorizing the story lines in the story line set to obtain vectorized story lines; Merging the vectorized story lines to obtain the typical story line; The typical story line is practiced using the robot client to obtain the occurrence probability of the typical story line.

5. The method for analyzing and judging similarity according to claim 4, characterized in that: The steps of vectorizing the subject to obtain the vectorized subject are specifically as follows: The topics are grouped and their similarities are marked, and the vectorized topics are obtained by using a word vector generation technique.

6. The method for analyzing and judging similarity according to claim 5, characterized in that: The steps of vectorizing the story lines in the story line set to obtain vectorized story lines are specifically as follows: According to the order of the themes included in the story line, the vectors corresponding to the themes are spliced ​​in sequence to obtain a concatenation result as the vector expression of the story line, thereby obtaining the vectorized story line.

7. The method for analyzing and judging similarity according to claim 6, characterized in that: The step of merging the vectorized story lines to obtain the typical story line specifically includes: Calculating the similarity of the vectorized story lines; The same vectorized story lines are grouped according to the similarity, and a representative is selected from each group as the typical story line.

8. The method for analyzing and judging similarity according to claim 7, characterized in that: The step of using the robot client to practice the typical story line to obtain the probability of occurrence of the typical story line is specifically as follows: The typical story lines are practiced using the robot client, the occurrence frequencies of each of the typical story lines are counted and recorded, and the occurrence probabilities of the typical story lines are obtained.

9. The method for analyzing and judging similarity according to claim 8, characterized in that: The step of judging the similarity of robot clients by using the typical story lines and the occurrence probability of the typical story lines comprises: If the typical story lines contained in two robot clients are the same, and the probability of occurrence of the same typical story lines is similar, then it is determined that the two robot clients are similar.

10. A system for analyzing and judging similarity, characterized in that: include: A recording module, used to obtain typical story lines included in the robot client and the probability of occurrence of the typical story lines; The judgment module is used to judge the similarity of robot clients by using the story lines and the occurrence probabilities of the story lines.

Images