A method and system for providing conversation reminders in a customer service system

Abstract

This invention relates to the field of customer service systems, specifically to a conversation reminder method and system for customer service systems. The invention collects real-time voice signals between customers and customer service representatives through the customer service system, i.e., a first voice signal. Then, it performs voice separation on the first voice signal to obtain a second voice signal and a third voice signal corresponding to the customer and customer service representative, respectively. Next, it calculates the voice overlap degree based on the time of the voices in the second and third voice signals, and uses a spectrogram to calculate the emotional complexity of the customer or customer service representative at any given time. These two dimensions serve as reference data for calculating the overlap degree, and a threshold is used to identify whether the customer or customer service representative is angry, thus enabling timely reminders to the customer service representative to adjust their communication style. This solves the problem that in current customer service systems, when communicating with customers via voice calls, customer service representatives often struggle to promptly perceive the emotions of both the customer and themselves during communication.

Description

Technical Field

[0001] This invention relates to the field of customer service systems, specifically to a method and system for providing conversation reminders in a customer service system. Background Technology

[0002] Customer service systems are used for communication between customers and customer service representatives, including text communication and voice calls. To ensure that customer service representatives maintain a good attitude during communication, companies usually provide training or use the customer service system to guide their speech. Currently, for text communication, the system can effectively extract keywords from text information and search for relevant content from the database to provide to customer service representatives.

[0003] However, voice communication is highly instantaneous, and both customers and customer service representatives may suddenly experience abnormal emotions such as anger during the communication process. Therefore, customer service representatives need to be highly sensitive and able to detect and adjust communication strategies in a timely manner. However, this method, which mainly relies on the customer service representative's sense of conversation to judge whether the customer or the representative is experiencing abnormal emotions, has a lot of uncertainty and is difficult to help the company's customer service team improve overall communication effectiveness. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a conversation reminder method and system for customer service systems, which solves the problem that customer service representatives often struggle to promptly detect the emotions of customers and themselves when communicating with them via voice calls in current customer service systems.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for providing session reminders in a customer service system, the method comprising the following steps:

[0007] S1. Real-time acquisition of the first voice signal between customer service personnel and customers in the customer service system;

[0008] S2. Filter the first voice signal to obtain a second voice signal containing only the voice of the customer service personnel and a third voice signal containing only the voice of the customer.

[0009] S3. Based on the sound presence time in the second and third voice signals, obtain the sound overlap interval between the two to calculate the sound overlap degree between the customer service representative and the customer.

[0010] S4. Mark the time intervals corresponding to the overlapping intervals of the second and third voice signals as the first time interval, mark the other time intervals as the second time interval, and calculate the emotional complexity of the customer service representative and the customer at any moment in the second time interval.

[0011] S5. Calculate the overlap between customer service representatives and customers based on voice overlap and emotional complexity, respectively.

[0012] S6. Based on whether the overlap between the customer service representative or the customer exceeds the overlap threshold, output a prompt message indicating whether the customer service representative or the customer's emotions are abnormal.

[0013] If so, output a message indicating that the customer service representative or customer is experiencing an emotional disturbance;

[0014] If not, then the process ends.

[0015] Preferably, step S3 specifically includes the following steps:

[0016] S31. Set a unit window, and take any moment of the second speech signal and the third speech signal as the center of the unit window, and calculate the waveform change rate and total energy at any moment respectively.

[0017] S32. Set the rate of change threshold and the energy threshold, and mark the times corresponding to the second speech signal and the third speech signal whose waveform rate of change is greater than the rate of change threshold and whose total energy is greater than the energy threshold as the second phonation time and the third phonation time, respectively.

[0018] S33. Mark the same moment in the second and third phonation moments as the overlapping moment, mark several consecutive overlapping moments as the overlapping period, and mark several consecutive second phonation moments and several consecutive third phonation moments as the phonation period.

[0019] S34. Set a first statistical window containing several vocalization periods with the current time as the endpoint;

[0020] S35. Calculate the sound overlap of the sound signals in the first statistical window based on the proportion of the overlapping sound moments in the total time within the first statistical window, and the ratio of the sound duration containing overlapping sound moments to the total number of sound durations in the corresponding first statistical window.

[0021] Preferably, step S31 specifically includes the following steps:

[0022] S311. Set a unit window, with the center of the unit window being any moment in either the second or third speech signal.

[0023] S312. Calculate the rate of change of the sound signal within a unit window at any given time; the formula for calculating the rate of change of the waveform is:

[0024] ;

[0025] in,

[0026] ;

[0027] In the above formula, This represents the rate of change of the waveforms of the second and third speech signals. Indicates the size of the unit window. Let n represent the second and third speech signals at time i, and n be the number of sampling points for the second and third speech signals.

[0028] S313. Calculate the total energy of the sound signal within a unit window at any given time; the formula for calculating the total energy is:

[0029] ;

[0030] In the above formula, This represents the total energy of the second and third speech signals within a unit window at any given time. Indicates the size of the unit window. Let n represent the second and third speech signals at time i, and n be the number of sampling points for the second and third speech signals.

[0031] Preferably, step S34 specifically includes the following steps:

[0032] S341. Obtain several vocalization periods including the current moment;

[0033] S342. Obtain the earliest time among several occurrence periods;

[0034] S343. Set the time period between the earliest time and the current time as the first statistical window of the current time.

[0035] Preferably, step S35 specifically includes the following steps:

[0036] S351. Calculate the proportion of the superimposed sound moment within the total time in the first statistical window; the formula for calculating the proportion of the superimposed sound moment within the total time in the first statistical window is:

[0037] ;

[0038] In the above formula, This represents the percentage of the total time interval within the first statistical window at time t. This represents the total number of overlapping moments within the first statistical window. This represents the total number of moments within the first statistics window;

[0039] S352. Calculate the ratio of the sounding time segment containing overlapping moments to the total number of sounding time segments within the corresponding first statistical window; the formula for calculating this ratio is:

[0040] ;

[0041] In the above formula, This represents the ratio of the number of sound periods containing overlapping sounds at time t to the total number of sound periods within the corresponding first statistical window. This represents the number of vocalization periods that include overlapping moments. This indicates the total number of speaking time periods within the first statistics window;

[0042] S353. Calculate the sound overlap at the current moment;

[0043] ;

[0044] In the above formula, Indicates the degree of overlap of sounds at the current moment. This represents the percentage of the total time interval within the first statistical window at time t. This represents the ratio of the number of sound periods containing overlapping sounds at time t to the total number of sound periods within the corresponding first statistical window. This represents the first weighting coefficient. represents the second weighting coefficient, and m represents the number of moments closest to the current moment used to calculate the sound overlap.

[0045] Preferably, step S4 specifically includes the following steps:

[0046] S41. Establish a spectrogram based on the second time period;

[0047] S42. Set up a second statistical window, using any pixel in the spectrogram as the center of the second statistical window, and calculate the first complexity of the pixel value of each pixel in the second statistical window.

[0048] S43. Calculate the emotional complexity at any given moment based on the average of the first complexity of all pixels in the spectrogram at any given moment.

[0049] Preferably, step S42 specifically includes the following steps:

[0050] S421. Set a second statistical window centered on any pixel in the spectrogram, and calculate the first probability that the pixel value corresponding to the center pixel of the second statistical window appears in the second statistical window; the formula for calculating the first probability is:

[0051] ;

[0052] In the above formula, This represents the first probability that the pixel value corresponding to the center pixel of the second statistical window appears in the second statistical window. This indicates the number of pixels in the second statistics window whose pixel values ​​are equal to those of its center pixel. This indicates the total number of pixels in the second statistics window;

[0053] S422. Calculate the first complexity of any pixel based on the first probability; the formula for calculating the first complexity is:

[0054] ;

[0055] In the above formula, Represents any pixel point corresponding to time t. The first complexity, This represents the first probability that the pixel value corresponding to the center pixel i of the second statistical window appears in the second statistical window. This center pixel corresponds to time t, and there are a total of B pixels in the second statistical window.

[0056] Preferably, step S43 specifically includes the following steps:

[0057] S431. Calculate the average of the first complexity of all pixels at any time in the spectrogram; the formula for calculating the average of the first probabilities is:

[0058] ;

[0059] In the above formula, This represents the average of the first complexity of each pixel in the spectrogram corresponding to any time t. Represents any pixel in the spectrogram at any time t. The first complexity is that the number of pixels at any time t in the spectrogram is c;

[0060] S432. Set a third statistical window and calculate the emotional complexity at any given time based on the average of the first probabilities; the formula for calculating emotional complexity is:

[0061] ;

[0062] In the above formula, Denotes the emotional complexity of j at any given time. This represents the average of the first probabilities of each pixel in the spectrogram corresponding to any time t. and These are the two time points corresponding to the third statistical window.

[0063] Preferably, in step S5, the formula for calculating the degree of overlap is:

[0064] ;

[0065] In the above formula, Indicates the degree of overlap. This represents the sound overlap coefficient. Indicates the degree of sound overlap. This represents the emotional complexity coefficient. The degree of overlap, voice overlap, and emotional complexity all correspond to the current moment.

[0066] The technical solution also provides a session reminder system, including a processor and a memory, wherein the memory is used to store a computer program, and the computer program, when executed by the processor, implements the session reminder method of the customer service system.

[0067] Compared with the prior art, the present invention provides a conversation reminder method and system for a customer service system, which has the following beneficial effects:

[0068] 1. This invention collects real-time voice signals between customers and customer service representatives through a customer service system, i.e., the first voice signal. Then, it performs voice separation on the first voice signal to obtain the second and third voice signals corresponding to the customer and customer service representatives, respectively. Subsequently, it calculates the voice overlap degree based on the time of the voices in the second and third voice signals, and calculates the emotional complexity of the customer or customer service representative at any time using a spectrogram. Thus, it uses two dimensions as reference data for calculating the overlap degree, and judges whether the customer or customer service representative is angry by using a threshold, thereby enabling timely reminders to customer service representatives to adjust their communication methods.

[0069] 2. This invention achieves the identification of the second and third phonation moments in the sound signal by calculating the waveform change rate and total energy in the second and third speech signals. This enables the extraction of the customer's phonation period and overlapping phonation periods in the second and third speech signals, facilitating subsequent calculation of the sound overlap.

[0070] 3. This invention calculates the degree of sound overlap from two dimensions: the proportion of overlapping moments in the total time within the first statistical window and the ratio of the speech period containing overlapping moments to the total number of speech periods in the corresponding first statistical window. This is based on the number of times customers and customer service interrupt each other and the duration of sound overlap when conflicts occur.

[0071] 4. This invention calculates the emotional complexity of customer service representatives and customers based on the spectrogram corresponding to the second time period, thereby deeply extracting information contained in the conversation between customers and customer service representatives, such as speech rate and volume. This allows the invention to reflect whether the customer's emotional state is angry, thus enabling the invention to obtain the emotional state of both customers and customer service representatives and notify customer service representatives to adjust communication methods in a timely manner. Attached Figure Description

[0072] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0073] Figure 1 This is a flowchart of the session reminder method for the customer service system of the present invention. Detailed Implementation

[0074] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0075] Those skilled in the art will understand that all or part of the steps in the methods of the following embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] To address the issue that customer service representatives often struggle to promptly detect customer and emotional states during voice calls within customer service systems, this invention provides a conversation reminder method for customer service systems. By extracting voice signals containing only the customer service representative's voice or only the customer's voice, and based on phenomena such as interruptions and changes in speech rate and tone that indicate anger, this method assists in identifying customer emotions. This serves to alert customer service representatives, enabling them to adjust their communication style accordingly. The method includes the following steps:

[0077] S1. Real-time acquisition of the first voice signal between customer service personnel and customers in the customer service system;

[0078] S2. Filter the first voice signal to obtain a second voice signal containing only the voice of the customer service representative and a third voice signal containing only the voice of the customer. The extraction of the second and third voice signals can be carried out using voiceprint recognition technology. The first voice signal is assigned to different people according to different voiceprints. Since voice calls are usually between two people, the voiceprints of the customer service representative can be recorded in advance and trained to obtain the second and third voice signals more quickly in real time. For example, the voice of the customer service representative can be directly assigned to the second voice signal and the voiceprint of the other person can be assigned to the third voice signal. This will not be elaborated on here.

[0079] S3. Based on the sound presence time in the second and third voice signals, obtain the sound overlap interval between the two to calculate the sound overlap degree between the customer service representative and the customer. The sound overlap degree is used to indicate the extent to which the customer service representative and the customer interrupt each other during communication. That is, when the sound presence time of the two is the same, an interruption occurs. To further refine the occurrence of interruptions, it is analyzed from two perspectives: the number of interruptions and the time when they speak simultaneously with each other during the interruption. In step S3, the specific steps are as follows:

[0080] S31. Set a unit window, taking any moment from the second and third voice signals as the center of the unit window, and calculate the waveform change rate and total energy at each moment. The waveform change rate is used to detect whether someone is speaking. When someone is speaking, the frequency of the waveform change rate is higher, and the total energy is also higher. When customer service representatives or customers are angry, the tone of voice will generally rise, so the total energy will generally also increase. Step S31 specifically includes the following steps:

[0081] S311. Set a unit window, taking any moment in the second and third speech signals as the center of the unit window. Since the waveform change rate at a single moment is meaningless, it is necessary to calculate the waveform change rate at any moment over a certain period of time, which is the unit window.

[0082] S312. Calculate the rate of change of the sound signal within a unit window at any given time; the formula for calculating the rate of change of the waveform is:

[0083] ;

[0084] in,

[0085] ;

[0086] In the above formula, This represents the rate of change of the waveforms of the second and third speech signals. Indicates the size of the unit window. Let n represent the second and third speech signals at time i, and n be the number of sampling points for the second and third speech signals. By changing the number of sampling points, the waveform change rate at the current time can be calculated more objectively, preventing data anomalies caused by unreasonable sampling points. The error can be reduced by averaging.

[0087] S313. Calculate the total energy of the sound signal within a unit window at any given time. The formula for calculating the total energy is:

[0088] ;

[0089] In the above formula, This represents the total energy of the second and third speech signals within a unit window at any given time. Indicates the size of the unit window. Let n represent the second and third speech signals at time i, and n be the number of sampling points for the second and third speech signals.

[0090] S32. Set the rate of change threshold and the energy threshold, and mark the times corresponding to the second speech signal and the third speech signal whose waveform rate of change is greater than the rate of change threshold and whose total energy is greater than the energy threshold as the second phonation time and the third phonation time, respectively.

[0091] S33. Mark the same moment in the second and third phonation moments as the overlapping moment, mark several consecutive overlapping moments as the overlapping period, and mark several consecutive second phonation moments and several consecutive third phonation moments as the phonation period.

[0092] S34. Set a first statistical window containing several sound periods, with the current time as the endpoint. When there are incomplete sound periods in the first window, except for the sound period corresponding to the current time, it is difficult to accurately determine whether there are overlapping sound periods in the sound period. Therefore, it is necessary to dynamically change the first statistical window so that all sound periods in the first statistical window are complete, except for the sound period corresponding to the current time. The following is a method for obtaining the first statistical window. In step S34, the specific steps are as follows:

[0093] S341. Obtain several vocalization periods including the current moment;

[0094] S342. Obtain the earliest time among several occurrence periods;

[0095] S343. Set the time period between the earliest time and the current time as the first statistical window of the current time.

[0096] S35. Based on the proportion of overlapping moments in the total time within the first statistical window, and the ratio of the speech periods containing overlapping moments to the total number of speech periods within the corresponding first statistical window, calculate the sound overlap degree of the sound signals within the first statistical window. To calculate the sound overlap degree more objectively, the number of times the other party's speech was interrupted and the duration of overlap between the interrupted sound and the other party's speech are taken into consideration. Specifically, step S35 includes the following steps:

[0097] S351. Calculate the proportion of the superimposed sound moment within the total time in the first statistical window; the formula for calculating the proportion of the superimposed sound moment within the total time in the first statistical window is:

[0098] ;

[0099] In the above formula, This represents the percentage of the total time interval within the first statistical window at time t. This represents the total number of overlapping moments within the first statistical window. This represents the total number of moments within the first statistics window;

[0100] S352. Calculate the ratio of the sounding time segment containing overlapping moments to the total number of sounding time segments within the corresponding first statistical window; the formula for calculating this ratio is:

[0101] ;

[0102] In the above formula, This represents the ratio of the number of sound periods containing overlapping sounds at time t to the total number of sound periods within the corresponding first statistical window. This represents the number of vocalization periods that include overlapping moments. This indicates the total number of speaking time periods within the first statistics window;

[0103] S353. Calculate the sound overlap at the current moment;

[0104] ;

[0105] In the above formula, Indicates the degree of overlap of sounds at the current moment. This represents the percentage of the total time interval within the first statistical window at time t. This represents the ratio of the number of sound periods containing overlapping sounds at time t to the total number of sound periods within the corresponding first statistical window. This represents the first weighting coefficient. represents the second weighting coefficient, and m represents the number of moments closest to the current moment used to calculate the sound overlap.

[0106] S4. Mark the time intervals corresponding to the overlapping sound intervals in the second and third speech signals as the first time interval, and mark the other time intervals as the second time interval. Calculate the emotional complexity of the customer service representative and the customer at any moment within the second time interval. The first time interval is the overlapping sound interval mentioned earlier, and the second time interval is the vocalization interval mentioned earlier minus the overlapping sound interval. To fully extract information from the second time interval, a spectrogram is constructed based on the second time interval, and information is extracted based on the spectrogram to calculate the emotional complexity. Step S4 specifically includes the following steps:

[0107] S41. Establish a spectrogram based on the second time period. The establishment of a spectrogram is a process that integrates signal processing, spectrum analysis, color coding, and software implementation. It can convert complex speech signals into intuitive two-dimensional images, thereby facilitating the analysis and understanding of the characteristics of speech signals. There are various ways to establish a spectrogram. For example, in actual operation, the spectrogram can be generated by using specialized software or programming languages ​​(such as Python). For example, the stft function in the librosa library can be used to obtain a complex spectrogram. The amplitude spectrogram can be obtained by taking the real part, and then the amplitude spectrum can be squared to obtain the power spectrogram.

[0108] S42. Set up a second statistical window, using any pixel in the spectrogram as the center of the second statistical window, and calculate the first complexity of the pixel value of each pixel in the second statistical window; when the tone or speed of the customer or customer service representative's speech changes, the first complexity will increase, thereby realizing the recognition of abnormal emotions such as anger. Step S42 specifically includes the following steps:

[0109] S421. Set a second statistical window centered on any pixel in the spectrogram, and calculate the first probability that the pixel value corresponding to the center pixel of the second statistical window appears in the second statistical window; the formula for calculating the first probability is:

[0110] ;

[0111] In the above formula, This represents the first probability that the pixel value corresponding to the center pixel of the second statistical window appears in the second statistical window. This indicates the number of pixels in the second statistics window whose pixel values ​​are equal to those of its center pixel. This indicates the total number of pixels in the second statistics window;

[0112] S422. Calculate the first complexity of any pixel based on the first probability; the formula for calculating the first complexity is:

[0113] ;

[0114] In the above formula, Represents any pixel point corresponding to time t. The first complexity, This represents the first probability that the pixel value corresponding to the center pixel i of the second statistical window appears in the second statistical window. This center pixel corresponds to time t, and there are a total of B pixels in the second statistical window.

[0115] S43. Calculate the emotional complexity at any given moment based on the average of the first complexity of all pixels at any given moment in the spectrogram. Since multiple pixels correspond to any given moment in the spectrogram, the calculation of the emotional complexity at a hot topic moment requires using the first complexity of all pixels corresponding to that moment as a basis. Step S43 specifically includes the following steps:

[0116] S431. Calculate the average of the first complexity of all pixels at any time in the spectrogram; the formula for calculating the average of the first probabilities is:

[0117] ;

[0118] In the above formula, This represents the average of the first complexity of each pixel in the spectrogram corresponding to any time t. Represents any pixel in the spectrogram at any time t. The first complexity is that the number of pixels at any time t in the spectrogram is c;

[0119] S432. Set a third statistical window and calculate the emotional complexity at any given time based on the average of the first probabilities; the formula for calculating emotional complexity is:

[0120] ;

[0121] In the above formula, Denotes the emotional complexity of j at any given time. This represents the average of the first probabilities of each pixel in the spectrogram corresponding to any time t. and These are the two time points corresponding to the third statistical window.

[0122] S5. Calculate the overlap between the customer service representative and the customer based on voice overlap and emotional complexity, respectively; the formula for calculating the overlap is:

[0123] ;

[0124] In the above formula, Indicates the degree of overlap. This represents the sound overlap coefficient. Indicates the degree of sound overlap. This represents the emotional complexity coefficient. The overlap, voice overlap, and emotional complexity all correspond to the current moment. Since the values ​​of overlap and emotional complexity differ significantly, the voice overlap coefficient is generally between 10 and 200 times that of the emotional complexity coefficient. For example, if the overlap coefficient is set to about 40 times that of the emotional complexity coefficient, then the two have roughly the same impact on the overlap.

[0125] S6. Based on whether the overlap between the customer service representative or the customer exceeds the overlap threshold, output a prompt message indicating whether the customer service representative or the customer's emotions are abnormal. The overlap threshold is affected by the voice overlap coefficient, which is an emotional complexity coefficient, and therefore needs to be determined according to the actual situation.

[0126] If so, output a message indicating that the customer service representative or customer is experiencing an emotional disturbance;

[0127] If not, then the process ends.

[0128] In implementation, this invention collects real-time voice signals between customers and customer service representatives through a customer service system, i.e., the first voice signal. Then, the first voice signal is separated into second and third voice signals corresponding to the customer and customer service representatives, respectively. Subsequently, the overlap of voices is calculated based on the time of the voices in the second and third voice signals, and the emotional complexity of the customer or customer service representative at any given time is calculated using a spectrogram. These two dimensions serve as reference data for calculating the overlap, and a threshold is used to identify whether the customer or customer service representative is angry, thereby promptly reminding the customer service representative to adjust their communication style.

[0129] Corresponding to the session reminder method of the customer service system provided in the above embodiments, this embodiment also provides a system for implementing the session reminder method of the customer service system. Since the session reminder system provided in this embodiment corresponds to the session reminder method of the customer service system provided in the above embodiments, the implementation method of the aforementioned session reminder method of the customer service system is also applicable to the session reminder system provided in this embodiment, and will not be described in detail in this embodiment.

[0130] The system includes a processor and a memory, the memory being used to store computer programs, which, when executed by the processor, implement a session reminder method for the customer service system.

[0131] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method of reminding of a session of a customer service system, characterized by, The method comprises the following steps: S1, real-time acquisition of a first voice signal of communication between a customer service staff and a customer in a customer service system; S2, filtering of the first voice signal to obtain a second voice signal containing only the voice of the customer service staff and a third voice signal containing only the voice of the customer; S3, according to the sound existing time in the second voice signal and the third voice signal, obtaining the sound overlap interval of the two to calculate the sound overlap degree between the customer service staff and the customer; in step S3, the following steps are specifically included: S31, setting a unit window, sequentially taking any time in the second voice signal and the third voice signal as the center of the unit window, and respectively calculating the waveform change rate and the total energy at any time; S32, setting a change rate threshold and an energy threshold, marking the time corresponding to the second voice signal and the third voice signal respectively as a second sound emission time and a third sound emission time when the waveform change rate is greater than the change rate threshold and the total energy is greater than the energy threshold; S33, marking the same time in the second sound emission time and the third sound emission time as a superimposed sound time, marking a plurality of continuous superimposed sound times as a superimposed sound period, and marking a plurality of continuous second sound emission times and a plurality of continuous third sound emission times as a sound emission period; S34, setting a first statistical window containing a plurality of sound emission periods with the current time as the end point; S35, calculating the sound overlap degree of the sound signal in the first statistical window according to the proportion of the total time of the superimposed sound time in the first statistical window and the ratio of the sound emission period containing the superimposed sound time to the total number of sound emission periods in the corresponding first statistical window; S4, marking the time period corresponding to the sound overlap interval in the second voice signal and the third voice signal as a first time period, marking other time periods as a second time period, and respectively calculating the emotional complexity of the customer service staff and the customer at any time in the second time period; The specific steps of respectively calculating the emotional complexity of the customer service staff and the customer at any time in the second time period are as follows: S41, establishing a spectrogram according to the second time period; S42, setting a second statistical window, taking any pixel point in the spectrogram as the center of the second statistical window, and calculating the first complexity of the pixel value of each pixel point in the second statistical window; in step S42, the following steps are specifically included: S421, setting the second statistical window with any pixel point on the spectrogram as the center, and calculating the first probability of the pixel value corresponding to the center pixel point of the second statistical window appearing in the second statistical window; the calculation formula of the first probability is: ； In the above formula, denotes the first probability that the pixel value corresponding to the center pixel point of the second statistical window appears in the second statistical window, denotes the number of pixel points in the second statistical window that are equal to the pixel value of the center pixel point thereof, denotes the total number of pixel points in the second statistical window; S422, calculating the first complexity of any pixel point according to the first probability; the calculation formula of the first complexity is: ； In the above formula, denotes any pixel point corresponding to time t a first complexity of the pixel point, denotes a first probability that the pixel value corresponding to the center pixel point i of the second statistical window appears in the second statistical window, the center pixel point corresponding to time t, and there are B pixel points in the second statistical window; S43, calculating the emotional complexity at any time according to the average value of the first complexity of all pixel points in the spectrogram; in step S43, the following steps are specifically included: S431, calculating the average value of the first complexity of all pixel points at any time in the spectrogram; the calculation formula of the average value of the first probability is: ； In the above formula, represents the average value of the first complexity of each pixel point corresponding to any time t in the spectrogram, represents the first complexity of any pixel point in the spectrogram at any time t The number of pixel points corresponding to any time t in the spectrogram is c. S432, setting a third statistical window, and calculating the emotional complexity at any time according to the average value of the first probability; the calculation formula of the emotional complexity is: ； In the above formula, denotes the emotion complexity at any time j, denotes the average value of the first probability of each pixel point corresponding to any time t in the spectrogram, and are two times corresponding to the third statistical window, respectively. S5, calculate the coincidence degree of the customer service and the customer according to the sound coincidence degree and the emotion complexity respectively; S6, output the prompt information of whether the emotion of the customer service or the customer is abnormal according to whether the coincidence degree of the customer service or the customer is greater than the coincidence degree threshold; If yes, output the prompt information that the emotion of the customer service or the customer is abnormal; If no, end.

2. The conversation reminding method according to claim 1, characterized by, In step S31, specifically comprising the following steps: S311, set a unit window, and take any time in the second voice signal and the third voice signal as the center of the unit window; S312, calculate the waveform change rate of the sound signal in the unit window at any time; The calculation formula of the waveform change rate is: ； Wherein, ； In the above formula, denotes a waveform variation rate of the second and third voice signals, denotes a size of a unit window, denotes the second and third voice signals at the i-th moment, and n is a sampling point number of the second and third voice signals; S313, calculate the total energy of the sound signal in the unit window at any time; the calculation formula of the total energy is: ； In the above formula, denotes the total energy of the second and third speech signals in a unit window at any time, denotes the size of a unit window, denotes the second and third speech signals at the i-th time, and n is the number of sampling points of the second and third speech signals.

3. The conversation reminding method according to claim 1, characterized by, In step S34, specifically comprising the following steps: S341, obtain a plurality of sound production time periods containing the current time; S342, obtain the earliest time in the plurality of sound production time periods; S343, set the time period between the earliest time and the current time as the first statistical window of the current time.

4. The conversation reminding method according to claim 1, characterized by, In step S35, specifically comprising the following steps: S351, calculate the proportion of the total time of the superposition sound time in the first statistical window; the calculation formula of the proportion of the total time of the superposition sound time in the first statistical window is: ； In the above formula, represents the proportion of the total time of the superposition time at the t time in the first statistical window, represents the total number of time of the superposition time in the first statistical window, represents the total number of time in the first statistical window; S352, calculate the ratio of the sound production time period containing the superposition sound time to the total number of sound production time periods in the corresponding first statistical window; The calculation formula of the ratio is: ； In the above formula, represents the ratio of the sound emission period containing the superposition time point at time t to the total number of sound emission periods in the corresponding first statistical window, is the number of sound emission periods containing the superposition time point, represents the total number of sound emission periods in the first statistical window; S353, calculate the sound coincidence degree of the current time; ； In the above formula, represents the sound coincidence degree at the current time, represents the proportion of the total time of the superposition time in the first statistical window at the time t, represents the ratio of the sound emission period containing the superposition time at the time t to the total sound emission period in the corresponding first statistical window, represents the first weight coefficient, represents the second weight coefficient, and m represents the number of times closest to the current time for calculating the sound coincidence degree.

5. The conversation reminding method according to claim 1, characterized by, In step S5, the calculation formula of the coincidence degree is: ； In the above formula, denotes the degree of coincidence, denotes the sound coincidence coefficient, denotes the sound coincidence, denotes the emotion complexity coefficient, denotes the emotion complexity, the degree of coincidence, the sound coincidence, and the emotion complexity all correspond to the current time.

6. A session reminding system for implementing the session reminding method according to any one of claims 1 to 5, characterized in that, The customer service system comprises a processor and a memory, the memory is used for storing a computer program, and the computer program is executed by the processor to realize the conversation prompting method of the customer service system according to any one of claims 1 to 5.

Images