Session quality evaluation method, system, equipment and medium

Abstract

The invention provides a session quality evaluation method, system and device and a medium, and relates to the technical field of data analysis, the method comprises the following steps: determining candidate keywords based on session information of a current session; determining the keyword weight of each candidate keyword based on the undirected graph, selecting a preset number of candidate keywords as keywords based on the values of the keyword weights, and determining the sentiment weight of each keyword according to the sentiment tendency and sentiment intensity corresponding to each keyword under each evaluation dimension; according to the dynamic change condition of the emotional tendency and / or emotional intensity corresponding to each keyword in the session information, dynamically adjusting the emotional weight of each keyword; and in combination with the occurrence frequency, the emotion weight and the keyword weight of each keyword, performing weighted summation on the score of each score dimension to obtain a session quality score of the current session. According to the method, the session quality can be accurately and quickly evaluated.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a method, system, device and medium for evaluating conversation quality. Background Art

[0002] Existing session management platforms can centrally manage multiple sessions and can easily track and monitor communication records with customers, allowing sales, marketing, and service teams to better collaborate and understand customer needs, as well as more effectively manage customer resources and conversations. Some session management platforms also support automated tasks to improve efficiency by providing fast, personalized services. However, although existing session management platforms can centrally manage multiple sessions and track and monitor communication records, they pay little attention to the supervision and processing of employees. The large amount of data generated during session management makes it difficult for administrators to quickly and accurately supervise and evaluate employees, making it impossible to accurately and effectively evaluate and supervise session quality. Summary of the invention

[0003] In view of this, the first aspect of the present invention provides a method for evaluating session quality to solve the technical problem that the session quality cannot be accurately evaluated in the prior art. The method comprises:

[0004] Acquire session information of the current session, and determine candidate keywords of the session information according to multiple evaluation dimensions, wherein the session information includes session personnel information and corresponding session content;

[0005] Constructing an undirected graph for the candidate keywords under each evaluation dimension, determining a keyword weight of each candidate keyword based on the undirected graph, and selecting a preset number of the candidate keywords as keywords according to the value of the keyword weight under each evaluation dimension, wherein the size of the keyword weight is proportional to the importance of the candidate keyword;

[0006] Determine the emotional weight of each keyword according to the emotional tendency and the emotional intensity corresponding to each keyword under each evaluation dimension, wherein the emotional tendency includes positive tendency, neutral tendency and negative tendency; if the emotional tendency corresponding to the keyword is positive, the emotional weight of the keyword is proportional to the emotional intensity corresponding to the keyword; if the emotional tendency corresponding to the keyword is neutral, the emotional weight of the keyword is determined to be 1; if the emotional tendency corresponding to the keyword is negative, the emotional weight of the keyword is inversely proportional to the emotional intensity corresponding to the keyword;

[0007] Dynamically adjust the emotion weight of each keyword according to the dynamic changes of the emotion tendency and / or emotion intensity corresponding to each keyword in the conversation information;

[0008] Calculating the occurrence frequency of each of the keywords in the session information under each of the evaluation dimensions;

[0009] For each of the evaluation dimensions, the score of the current session under each of the evaluation dimensions is calculated in combination with the occurrence frequency, sentiment weight and keyword weight of each of the keywords, and the scores of each of the evaluation dimensions are weighted and summed to obtain the session quality score of the current session.

[0010] A second aspect of the present invention provides a session quality evaluation system to solve the technical problem in the prior art that session quality cannot be accurately evaluated. The system includes:

[0011] A session information acquisition module, used to acquire session information of a current session, and determine candidate keywords of the session information according to multiple evaluation dimensions, wherein the session information includes session personnel information and corresponding session content;

[0012] A keyword acquisition module is used to construct an undirected graph for the candidate keywords under each evaluation dimension, determine the keyword weight of each candidate keyword based on the undirected graph, and select a preset number of the candidate keywords as keywords according to the value of the keyword weight under each evaluation dimension, wherein the size of the keyword weight is proportional to the importance of the candidate keyword;

[0013] The sentiment weighting module is used to determine the sentiment weight of each keyword according to the sentiment tendency and the corresponding sentiment intensity corresponding to each keyword under each evaluation dimension, wherein the sentiment tendency includes positive tendency, neutral tendency and negative tendency; if the sentiment tendency corresponding to the keyword is positive, the sentiment weight of the keyword is proportional to the magnitude of the sentiment intensity corresponding to the keyword; if the sentiment tendency corresponding to the keyword is neutral, the sentiment weight of the keyword is determined to be 1; if the sentiment tendency corresponding to the keyword is negative, the sentiment weight of the keyword is inversely proportional to the magnitude of the sentiment intensity corresponding to the keyword;

[0014] A sentiment weight adjustment module, used to dynamically adjust the sentiment weight of each keyword according to the dynamic changes of the sentiment tendency and / or sentiment intensity corresponding to each keyword in the session information;

[0015] An occurrence frequency calculation module, used to calculate the occurrence frequency of each of the keywords in the session information under each of the evaluation dimensions;

[0016] The session quality assessment module is used to calculate the score of the current session under each of the assessment dimensions, combining the occurrence frequency, sentiment weight and keyword weight of each of the keywords, and perform weighted summation on the scores of each of the assessment dimensions to obtain the session quality score of the current session.

[0017] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned methods for evaluating session quality when executing the computer program.

[0018] The present invention also provides a computer-readable storage medium storing a computer program for executing any of the above-mentioned methods for evaluating session quality.

[0019] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above-mentioned technical solutions adopted in this specification include at least: providing a method for evaluating conversation quality, by extracting candidate keywords from conversation information, and determining the keyword weight of each candidate keyword through an undirected graph, and then selecting keywords with higher frequency of occurrence based on the size of the keyword weight, so as to effectively calculate the frequency of occurrence of keywords and improve the accuracy of the frequency of occurrence calculation; determining the emotional weight of each keyword through the emotional tendency corresponding to each keyword and its corresponding emotional intensity, and dynamically adjusting the emotional weight of each keyword according to the dynamic changes of the emotional tendency and / or emotional intensity in the conversation information, so that the emotional weight can be determined more accurately and dynamically, which is conducive to improving the accuracy of the emotional weight; finally, based on the frequency of occurrence of each keyword, the emotional weight and the keyword weight, the score under each evaluation dimension and the conversation quality score of the current conversation can be quickly and accurately calculated, achieving the technical effect of accurately, efficiently and multi-dimensionally evaluating the conversation quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 is a flow chart of a method for evaluating session quality provided by an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of the structure of a session quality evaluation system provided by an embodiment of the present invention;

[0023] Figure 3is a schematic diagram of a computer device provided by an embodiment of the present invention;

[0024] Attached numbers in the figure: 200, conversation quality evaluation system; 201, conversation information acquisition module; 202, keyword acquisition module; 203, sentiment weighting module; 204, sentiment weight adjustment module; 205, occurrence frequency calculation module; 206, conversation quality evaluation module; 301, memory; 302, processor. DETAILED DESCRIPTION

[0025] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0026] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0027] A method for evaluating conversation quality is provided. Keywords are extracted from conversation information through natural language processing technology, and the conversation quality is scored by weighting the sentiment of each keyword, thereby achieving the technical effect of accurately, efficiently and multi-dimensionally evaluating conversation quality.

[0028] like Figure 1 As shown, a method for evaluating session quality provided in this embodiment specifically includes the following steps:

[0029] Step S100: Acquire session information of the current session, and determine candidate keywords of the session information according to multiple evaluation dimensions, wherein the session information includes session personnel information and corresponding session content;

[0030] Specifically, the session information includes the session participants, session content, session response time, session start time, session end time and session duration of the current session.

[0031] Step S200, constructing an undirected graph for the candidate keywords under each of the evaluation dimensions, determining the keyword weight of each of the candidate keywords based on the undirected graph, and selecting a preset number of the candidate keywords as keywords according to the value of the keyword weight under each of the evaluation dimensions, wherein the size of the keyword weight is proportional to the importance of the candidate keywords;

[0032] Specifically, determine the scoring dimensions of the conversation, which include evaluation dimensions such as customer satisfaction, problem-solving efficiency, and communication quality. Set corresponding keywords according to each scoring dimension. For example, customer satisfaction: evaluated by the frequency of appearance and emotional tendency of positive and negative feedback keywords such as "satisfied", "good", and "excellent"; problem-solving efficiency: evaluated by the reduction of problem keywords and the appearance of solution keywords; professional knowledge: evaluated by the frequency of use of professional terms and the degree of customer recognition. And set the initial weight for each keyword according to its importance.

[0033] Step S300: Determine the sentiment weight of each keyword according to the sentiment tendency and the sentiment intensity corresponding to each keyword under each evaluation dimension, wherein the sentiment tendency includes positive tendency, neutral tendency and negative tendency; if the sentiment tendency corresponding to the keyword is positive, the sentiment weight of the keyword is proportional to the magnitude of the sentiment intensity corresponding to the keyword; if the sentiment tendency corresponding to the keyword is neutral, the sentiment weight of the keyword is determined to be 1; if the sentiment tendency corresponding to the keyword is negative, the sentiment weight of the keyword is inversely proportional to the magnitude of the sentiment intensity corresponding to the keyword;

[0034] Specifically, natural language processing (NLP) technology can be used to analyze the content of the current conversation and automatically identify keywords in the conversation. After identifying the corresponding keywords, sentiment analysis can be performed on the currently identified keywords based on the sentiment analysis model to obtain the sentiment tendency of the current keywords and their corresponding sentiment intensity.

[0035] Step S400: dynamically adjusting the emotion weight of each keyword according to the dynamic changes of the emotion tendency and / or emotion intensity corresponding to each keyword in the conversation information;

[0036] Step S500: Calculate the occurrence frequency of each keyword in the conversation information under each evaluation dimension;

[0037] Step S600: For each of the evaluation dimensions, the score of the current session under each of the evaluation dimensions is calculated in combination with the occurrence frequency, sentiment weight and keyword weight of each of the keywords, and the scores of each of the evaluation dimensions are weighted and summed to obtain the session quality score of the current session.

[0038] Furthermore, in step S400, in order to further ensure the accuracy of the sentiment weight of each keyword, a method for dynamically correcting the sentiment weight of each keyword is proposed, for example, according to the dynamic changes of the sentiment tendency and sentiment intensity corresponding to each keyword in the session information, the sentiment weight of each keyword is dynamically adjusted, including:

[0039] Step S401: adding a timestamp to the emotional tendency and emotional intensity of each keyword at each moment, forming a time series corresponding to each keyword according to the time sequence of the timestamps, and recording the quantitative value of the emotional intensity of the keyword at each time point in the time series;

[0040] Step S402: Based on a time series analysis method, identify the fluctuation amplitude and frequency of the emotion intensity in the time series corresponding to each keyword, and obtain a change pattern of the emotion intensity of each keyword according to the fluctuation amplitude and the frequency;

[0041] Step S403: dynamically adjusting the sentiment weight of each keyword according to the change pattern, fluctuation amplitude and frequency of the sentiment intensity.

[0042] Specifically, the emotional data corresponding to each keyword during the current session is collected, and the emotional tendency and intensity of the keyword are recorded in chronological order to form a time series. A timestamp is added to the emotional analysis result (i.e., emotional tendency and intensity) corresponding to each keyword to ensure that the time trajectory of emotional changes can be tracked. A sliding time window can be used to observe the change of emotional intensity in the time series within a specific time period, and the window size can be adjusted according to the length and characteristics of the session. After obtaining the time series corresponding to the keyword, a quantitative value is assigned to the keyword emotional intensity at each time point, such as a value between -1 and 1, and a time series analysis method, such as moving average or exponential smoothing, is used to identify the trend of emotional intensity in the time series, and the emotional intensity change pattern is identified according to the change trend of emotional intensity, such as: periodic change / non-periodic change. For example, some topics may cause stronger emotional reactions in a specific time period. Analyze the amplitude and frequency of emotional intensity fluctuations to understand the severity of emotional changes, and dynamically adjust the emotional weight of the keyword according to the change pattern and severity of emotional intensity (for example, according to the periodic change / non-periodic change pattern, adjust the size of the emotional weight according to the amplitude and frequency of fluctuations) to reflect its real-time importance in the session.

[0043] In the specific implementation, in order to further ensure the accuracy of the sentiment weight of each keyword, a method for dynamically correcting the sentiment weight of each keyword is proposed, for example,

[0044] Step S400 also includes:

[0045] Step S404: Based on natural language processing technology, context analysis is performed on each of the keywords to obtain the emotional correlation between each of the keywords and the corresponding emotional tendency;

[0046] Step S405: dynamically modify the sentiment weight of each keyword according to the sentiment relevance.

[0047] Specifically, based on the sentiment analysis model, sentiment tendency analysis is performed on each keyword, and the sentiment tendency weight corresponding to the keyword is obtained according to the sentiment tendency and the sentiment intensity. The sentiment tendency type includes positive tendency, neutral tendency and negative tendency, and each sentiment tendency result corresponds to a sentiment intensity. The sentiment tendency type and the sentiment intensity corresponding to the keyword can be output by the sentiment analysis model, and the sentiment analysis model is analyzed based on the deep learning algorithm. If the keyword is identified as a positive sentiment tendency, the keyword is sentimentally weighted, and its sentiment weight is equal to the keyword weight multiplied by a first preset multiple, and the greater the sentiment intensity corresponding to the keyword, the greater the first preset multiple; if the keyword is identified as a negative sentiment tendency, the keyword is sentimentally weighted, and its sentiment weight is equal to the keyword weight multiplied by a second preset multiple, and the greater the sentiment intensity corresponding to the keyword, the smaller the second preset multiple, and the first preset multiple is a value greater than 1, and the value of the second preset multiple is a value greater than 0 and less than 1; if the keyword is identified as a neutral sentiment tendency, the sentiment weight of the keyword is equal to the keyword weight multiplied by 1. For example, the sentiment weight of a positive keyword can be mapped to 1.25, the sentiment weight of a negative keyword can be mapped to 0.75, and the sentiment weight of a neutral keyword can be mapped to 1. After sentiment weighting / reducing for keywords based on sentiment, the current weight of the keyword = sentiment weight × keyword weight.

[0048] Furthermore, by weighting or de-weighting each keyword according to its sentiment tendency and sentiment intensity, the sentiment analysis of each keyword can be made more accurate. By de-weighting keywords with negative tendencies, their positive contribution in the sentiment analysis can be reduced.

[0049] Furthermore, step S404 further includes:

[0050] Step S4041: Determine the part of speech and meaning of each keyword in the context sentence;

[0051] Step S4042: determining the dependency relationship between each of the keywords and the context sentence;

[0052] Step S4043: Determine the sentiment correlation between the keyword and the corresponding sentiment tendency by combining the keyword's part of speech, word meaning and the dependency relationship.

[0053] Further, the context in which the keyword appears is analyzed based on natural language processing, and its relevance to the sentiment tendency (i.e., the consistency of the sentiment tendency of the keyword and the context) is determined, and the weight of the keyword is adjusted according to the sentiment tendency and the context relevance. Context analysis includes semantic annotation, dependency analysis and pattern matching to obtain context analysis results. Among them, the role of the keyword in the sentence is identified by using the semantic role labeling (SRL) technology, such as whether it is the cause of the problem or part of the solution. The grammatical relationship between the keyword and other words is determined by dependency analysis, which helps to understand the function of the keyword in the sentence. Specific patterns or phrases are identified in the text, which may indicate that the keyword appears in the context of the solution. For example, "This problem can be solved by..." or "For this problem, we proposed...". According to the results of the context analysis of the keyword, the weight of the keyword is dynamically adjusted. If the model recognizes that the keyword appears in the context of the solution, its negative weight can be reduced or its positive weight can be increased on the basis of the sentiment weighting to obtain the final sentiment weighting of the current keyword.

[0054] For example, if there is a sentence in the conversation record: "This problem can be solved by installing an update", the sentiment analysis model may identify that the sentiment tendency of the word "problem" is negative, but through context analysis, the model will understand that it is actually part of the solution and its sentiment tendency should be positive. Therefore, the sentiment weight of the word "problem" may not be reduced, or the reduction may be smaller.

[0055] Furthermore, in step S200, in order to more accurately determine the keyword weight of each keyword, it is proposed to determine the keyword weight based on an undirected graph, for example,

[0056] Constructing an undirected graph according to the candidate keywords under each evaluation dimension, wherein each node in the undirected graph corresponds to one candidate keyword;

[0057] Determine the correlation between different candidate keywords according to the correlation of each node in the undirected graph;

[0058] According to the associations between different candidate keywords, the keyword weight of each candidate keyword is calculated.

[0059] The weight of each node is calculated by the following formula, and the weight of each node is determined as the keyword weight of the candidate keyword corresponding to the node:

[0060]

[0061] Where d is the damping coefficient, In(V j ) indicates all points to node V j The collection of nodes, Out(V i ) indicates that all slave nodes V i The set of nodes connected by the outgoing edge, ω i，j Represents node V i and node V j The weight between j Represents node V j The weight of score i Represents node V i The weight, ω i,k Represents node V i and node V k The weight between k Is Out(V i ) any node in the node set; specifically, the value range of d can be 0.84 to 0.85.

[0062] Furthermore, the quality score of the current session in each dimension is calculated based on the frequency of occurrence of each keyword corresponding to each dimension, the sentiment tendency of the context, and the keyword weight. The calculation method of the score of a single scoring dimension includes:

[0063] Evaluation dimension score = ∑ (occurrence frequency × sentiment weight × keyword weight).

[0064] Furthermore, after the scores under each scoring dimension are standardized, a weighted sum method is adopted to comprehensively calculate the scores of each dimension as the session quality score of the current session.

[0065] Furthermore, in step S700, after obtaining the session quality score of the current session, the following steps are further included:

[0066] Step S701: The session information further includes session timestamp information, where the session timestamp information includes session response time, session start time, session end time, and session duration, and the response of the current session is evaluated according to the session timestamp information;

[0067] Step S702: Obtain a session response score corresponding to the current session according to the response situation;

[0068] Step S703: combining the session response score and the session quality score to obtain a comprehensive score result of the current session.

[0069] Specifically, the indicators for comprehensively scoring the current session include session response time, the proportion of session response time to the duration of a single session, and session quality. The session response scoring indicators can be preset. For example, the administrator can set the expiration time and judgment interval of a single session on the platform. The default expiration time of the session is one hour, and the default judgment interval is five minutes. The system judges the ongoing session once every judgment interval. When the time interval between the last session message in the session and the current judgment exceeds the expiration time, the session is judged to be over; and the time when the last record of the session is sent is set as the session end time. The session duration is calculated as the session end time minus the session start time. The system can also display historical session records to prevent the lack of information from effectively judging and scoring the session information.

[0070] After automatically obtaining the response score of the conversation according to the timestamp information of each conversation, the comprehensive score result of the current conversation is obtained by combining the conversation response score and the conversation quality score.

[0071] Furthermore, in step S702, after obtaining a comprehensive score result of the current session by combining the session response score and the session quality score, the following steps are performed:

[0072] Step S7021: determining whether the comprehensive scoring result is lower than the scoring threshold, and if so, sending the current session to the manual scoring process to obtain the manual scoring result;

[0073] Step S7022: Integrate the manual scoring result and the comprehensive scoring result according to a preset weight distribution to determine the final score of the current session.

[0074] Specifically, clear scoring standards and thresholds are set for the machine scoring model. Based on the preset thresholds, the scoring model automatically screens out conversation records that require manual review, such as low scores or abnormally high scores. Manual scorers score conversations based on standards and personal judgment, and can adjust machine scores, integrating manual scoring results with machine scoring results according to pre-assigned weights to determine the final score. For conversations whose comprehensive scoring results are lower than the scoring standards and thresholds, manual review is performed. The manual scoring process reduces the possible misjudgment problems caused by automatic scoring and improves the recognition and accuracy of the scoring system.

[0075] Preferably, the scorers can also re-evaluate the general conversation information. After re-evaluation, the score of a certain indicator in the conversation record can be changed, or the total score can be directly modified, and the scorer information and the scorer's comments will be retained after the modification. If there is no manual re-evaluation for a period of time after the automatic scoring, the automatic scoring content will be recorded as the final score. After the manual re-evaluation directly modifies the total score, the new total score will overwrite the total score of the original conversation record; after the manual re-evaluation modifies a certain indicator record, the platform will recalculate according to the original method based on the modified data and obtain a new total score for the conversation record.

[0076] Furthermore, after obtaining the final score, an intuitive user interface is provided to display the session score and related feedback; visualization tools such as charts and dashboards are designed to display the scoring results and trends and realize result visualization.

[0077] Based on the same inventive concept, a conversation quality evaluation system is also provided in an embodiment of the present invention, as described in the following embodiments. Since the principle of solving the problem by a conversation quality evaluation system is similar to that of a conversation quality evaluation method, the implementation of a conversation quality evaluation system can refer to the implementation of a conversation quality evaluation method, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0078] Figure 2 is a structural block diagram of a session quality evaluation system 200 according to an embodiment of the present invention. Figure 2 As shown, including:

[0079] The session information acquisition module 201 is used to acquire session information of the current session and determine candidate keywords of the session information according to multiple evaluation dimensions, wherein the session information includes session personnel information and corresponding session content;

[0080] A keyword acquisition module 202 is used to construct an undirected graph for the candidate keywords under each evaluation dimension, determine the keyword weight of each candidate keyword based on the undirected graph, and select a preset number of the candidate keywords as keywords according to the value of the keyword weight under each evaluation dimension, wherein the size of the keyword weight is proportional to the importance of the candidate keyword;

[0081] The sentiment weighting module 203 is used to determine the sentiment weight of each keyword according to the sentiment tendency and the corresponding sentiment intensity corresponding to each keyword under each evaluation dimension, wherein the sentiment tendency includes positive tendency, neutral tendency and negative tendency; if the sentiment tendency corresponding to the keyword is positive, the sentiment weight of the keyword is proportional to the magnitude of the sentiment intensity corresponding to the keyword; if the sentiment tendency corresponding to the keyword is neutral, the sentiment weight of the keyword is determined to be 1; if the sentiment tendency corresponding to the keyword is negative, the sentiment weight of the keyword is inversely proportional to the magnitude of the sentiment intensity corresponding to the keyword;

[0082] The emotion weight adjustment module 204 is used to dynamically adjust the emotion weight of each keyword according to the dynamic changes of the emotion tendency and / or emotion intensity corresponding to each keyword in the conversation information;

[0083] An occurrence frequency calculation module 205, used to calculate the occurrence frequency of each of the keywords in the session information under each of the evaluation dimensions;

[0084] The conversation quality assessment module 206 is used to calculate the score of the current conversation under each of the assessment dimensions, combining the occurrence frequency, sentiment weight and keyword weight of each of the keywords, and perform weighted summation on the scores of each of the assessment dimensions to obtain the conversation quality score of the current conversation.

[0085] In this embodiment, a computer device is provided, such as Figure 3 As shown, it includes a memory 301, a processor 302, and a computer program stored in the memory 301 and executable on the processor 302, and the processor 302 implements any of the above-mentioned methods for evaluating session quality when executing the computer program.

[0086] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0087] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing any of the above-mentioned methods for evaluating session quality.

[0088] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0089] The embodiments of the present invention achieve the following technical effects:

[0090] 1. Through the automated conversation recording and scoring system, the difficulty of conversation business management is reduced and the supervision efficiency is improved;

[0091] 2. Through natural language processing technology, conversation information is processed and data statistics are collected to extract effective information and understand the context in which keywords appear, so as to more accurately evaluate the emotional tendency of the conversation. According to the emotional tendency of keywords, NLP technology can help assign different weights to each keyword, improve the automation level of conversation processing, and enhance the accuracy and reliability of scoring;

[0092] 3. Combining multi-dimensional robot scoring and manual scoring, an accurate assessment of conversation quality is achieved;

[0093] 4. The visual display of the final scoring results not only improves the efficiency of data processing, but also enhances the user experience of the platform, which helps to better achieve business goals and improve service quality.

[0094] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for evaluating conversation quality, characterized in that: include: Acquire session information of the current session, and determine candidate keywords of the session information according to multiple evaluation dimensions, wherein the session information includes session personnel information and corresponding session content; Constructing an undirected graph for the candidate keywords under each evaluation dimension, determining a keyword weight of each candidate keyword based on the undirected graph, and selecting a preset number of the candidate keywords as keywords according to the value of the keyword weight under each evaluation dimension, wherein the size of the keyword weight is proportional to the importance of the candidate keyword; Determine the emotional weight of each keyword according to the emotional tendency and the emotional intensity corresponding to each keyword under each evaluation dimension, wherein the emotional tendency includes positive tendency, neutral tendency and negative tendency; if the emotional tendency corresponding to the keyword is positive, the emotional weight of the keyword is proportional to the emotional intensity corresponding to the keyword; if the emotional tendency corresponding to the keyword is neutral, the emotional weight of the keyword is determined to be 1; if the emotional tendency corresponding to the keyword is negative, the emotional weight of the keyword is inversely proportional to the emotional intensity corresponding to the keyword; Dynamically adjust the emotion weight of each keyword according to the dynamic changes of the emotion tendency and / or emotion intensity corresponding to each keyword in the conversation information; Calculating the occurrence frequency of each of the keywords in the session information under each of the evaluation dimensions; For each of the evaluation dimensions, the score of the current session under each of the evaluation dimensions is calculated in combination with the occurrence frequency, sentiment weight and keyword weight of each of the keywords, and the scores of each of the evaluation dimensions are weighted and summed to obtain the session quality score of the current session.

2. The method for evaluating session quality according to claim 1, characterized in that: After obtaining the session quality score of the current session, the method further includes: The session information also includes session timestamp information, which includes session response time, session start time, session end time and session duration, and the response of the current session is evaluated according to the session timestamp information; Obtaining a session response score corresponding to the current session according to the response situation; The session response score and the session quality score are combined to obtain a comprehensive score result of the current session.

3. The method for evaluating session quality according to claim 1, wherein: The step of dynamically adjusting the emotion weight of each keyword according to the dynamic changes of the emotion tendency and emotion intensity corresponding to each keyword in the session information includes: Adding a timestamp to the emotional tendency and emotional intensity of each keyword at each moment, forming a time series corresponding to each keyword according to the time sequence of the timestamps, and recording the quantitative value of the emotional intensity of the keyword at each time point in the time series; Based on the time series analysis method, identifying the fluctuation amplitude and frequency of the emotion intensity in the time series corresponding to each of the keywords, and obtaining the change pattern of the emotion intensity of each of the keywords; The sentiment weight of each keyword is dynamically adjusted according to the change pattern, fluctuation amplitude and frequency of the sentiment intensity.

4. The method for evaluating session quality according to claim 1, wherein: According to the dynamic change of the emotional tendency corresponding to each keyword in the session information, dynamically adjusting the emotional weight of each keyword includes: Based on natural language processing technology, context analysis is performed on each of the keywords to obtain the emotional relevance between each of the keywords and the corresponding emotional tendency; The sentiment weight of each keyword is dynamically modified according to the sentiment relevance.

5. The method for evaluating session quality according to claim 4, characterized in that: Obtaining the sentiment correlation between each of the keywords and the corresponding sentiment tendency includes: Determine the part of speech and meaning of each of the keywords in the context sentence; Determine the dependency relationship between each of the keywords and the context sentence; The sentiment correlation between the keyword and the corresponding sentiment tendency is determined by combining the part of speech, meaning and dependency relationship of the keyword.

6. The method for evaluating session quality according to claim 1, wherein: Constructing an undirected graph for the candidate keywords under each of the evaluation dimensions, and determining a keyword weight of each of the candidate keywords based on the undirected graph, including: Constructing an undirected graph according to the candidate keywords under each evaluation dimension, wherein each node in the undirected graph corresponds to one candidate keyword; Determine the correlation between different candidate keywords according to the correlation of each node in the undirected graph; According to the associations between different candidate keywords, the keyword weight of each candidate keyword is calculated.

7. The method for evaluating session quality according to claim 2, wherein: After obtaining a comprehensive score result of the current session by combining the session response score and the session quality score, the method further includes: Determine whether the comprehensive scoring result is lower than the scoring threshold, and if so, send the current session to the manual scoring process to obtain the manual scoring result; The manual scoring result and the comprehensive scoring result are integrated according to a preset weight distribution to determine a final score of the current session.

8. A conversation quality evaluation system, characterized in that: include: A session information acquisition module, used to acquire session information of a current session, and determine candidate keywords of the session information according to multiple evaluation dimensions, wherein the session information includes session personnel information and corresponding session content; A keyword acquisition module is used to construct an undirected graph for the candidate keywords under each evaluation dimension, determine the keyword weight of each candidate keyword based on the undirected graph, and select a preset number of the candidate keywords as keywords according to the value of the keyword weight under each evaluation dimension, wherein the size of the keyword weight is proportional to the importance of the candidate keyword; The sentiment weighting module is used to determine the sentiment weight of each keyword according to the sentiment tendency and the corresponding sentiment intensity corresponding to each keyword under each evaluation dimension, wherein the sentiment tendency includes positive tendency, neutral tendency and negative tendency; if the sentiment tendency corresponding to the keyword is positive, the sentiment weight of the keyword is proportional to the magnitude of the sentiment intensity corresponding to the keyword; if the sentiment tendency corresponding to the keyword is neutral, the sentiment weight of the keyword is determined to be 1; if the sentiment tendency corresponding to the keyword is negative, the sentiment weight of the keyword is inversely proportional to the magnitude of the sentiment intensity corresponding to the keyword; A sentiment weight adjustment module, used to dynamically adjust the sentiment weight of each keyword according to the dynamic changes of the sentiment tendency and / or sentiment intensity corresponding to each keyword in the session information; An occurrence frequency calculation module, used to calculate the occurrence frequency of each of the keywords in the session information under each of the evaluation dimensions; The session quality assessment module is used to calculate the score of the current session under each of the assessment dimensions, combining the occurrence frequency, sentiment weight and keyword weight of each of the keywords, and perform weighted summation on the scores of each of the assessment dimensions to obtain the session quality score of the current session.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for evaluating session quality according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the method for evaluating session quality according to any one of claims 1 to 7.

Images