A voice service content recognition method based on behavioral text processing

Abstract

The application relates to a voice service content identification method for behavior text processing, and belongs to the field of natural language processing, and comprises the following steps: collecting service audio of a customer service personnel, calculating the voiceprint similarity between each frame of the audio, and clustering the audio frames according to the similarity; establishing a customer service personnel voice tone library to determine the customer service personnel identity of the service audio; converting the service audio into voice text by using a voice recognition model; establishing a text keyword library, calculating and retrieving behavior keywords in the voice text, and constructing a behavior keyword sequence; establishing a speaker behavior recognition model, determining the customer service behavior according to the behavior keyword sequence, and constructing a customer service behavior sequence; and establishing a service content identification model, and determining the customer service content according to the customer service behavior sequence.

Description

Technical Field

[0001] This invention belongs to the field of computer natural language processing and relates to a method for recognizing speech service content based on behavioral text processing. Background Technology

[0002] The economic downturn and frequent black swan events have brought greater uncertainty, creating an urgent need for many small and medium-sized enterprises (SMEs) to improve efficiency and reduce costs. For the service industry, automating service content identification is imperative, as it can not only improve the efficiency of overall management but also help companies save on labor costs.

[0003] Currently, the main research approach for service content recognition in China is based on computer vision. Computer vision-based methods can extract service content from videos of human behavior using models; however, this method has a high learning curve and lacks universality and convenience.

[0004] Current research methods for determining service content based on speech primarily involve manually setting rules to extract service audio keywords and then using these keywords to determine the service content. Compared to computer vision-based methods, this approach is faster and has a lower barrier to entry. However, researchers need to manually annotate a large amount of data when setting rules for extracting service audio keywords. This not only consumes significant manpower but also often misses a large amount of valid data. Furthermore, this method can only identify the specified content of the keywords, resulting in poor flexibility and a high false negative rate. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method for recognizing the content of a voice service based on behavioral text processing.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for recognizing speech service content based on behavior text processing includes the following steps:

[0008] S1: Set up a voice acquisition device to obtain the audio of customer service personnel during their work. Collect the audio between each call and hang-up as a complete service audio record, and segment it into frames. Cluster the segmented audio frames based on the voiceprint characteristics of each frame to obtain customer service audio information.

[0009] S2: Establish a customer service voice database and use a voiceprint recognition model to identify the customer service representative in the service audio. Use a speech recognition model to convert the service audio into speech-to-text.

[0010] S3: Establish a text keyword database to determine service behaviors; establish a keyword acquisition module to calculate and retrieve behavioral keywords in voice text;

[0011] S4: Establish a speaker behavior recognition model; establish a behavior label library, train the constructed speaker behavior recognition model, and determine the speaker behavior through behavior keyword sequences;

[0012] S5: Establish a service content recognition model; establish a service content tag library, train the constructed service content recognition model, and use it to determine customer service content.

[0013] Furthermore, step S1 specifically includes the following steps:

[0014] S11: Obtain the service audio from the customer service representative during their work. Each time a customer service representative answers a call and hangs up, the audio is considered a complete service audio segment, denoted as Voice_Service. Divide Voice_Service into equal-length frames, with the i-th frame denoted as Voice_Frames. i Let i ∈ (1, L), where L represents the number of frames in Voice_Service. The Voice_Frames are calculated using Fast Fourier Transform (FFT). i The power spectrum is processed using a Mel filter bank, then transformed into the logarithmic domain, and finally processed again using discrete cosine transform to obtain Voice_Frames. i The voiceprint feature vector, representing MFCC_VF i ;

[0015] S12: Calculate the voiceprint similarity between the voiceprint feature vectors of each frame, cluster the service audio frames with similarity higher than the threshold Hold_S, and finally cluster Voice_Service into multiple audio segments based on the voiceprint similarity.

[0016] Furthermore, the clustering process described in step S12 is as follows:

[0017] Create a set of ending frames, using each ending frame as the end of a service audio frame cluster. The set of ending frames is denoted as Sum_END={}. First, set the first frame as the starting audio frame (Start), and the second frame as the current frame (Now). Compare the voiceprint similarity between Start and Now. The calculation formula is:

[0018]

[0019] Among them, MFCC_VF Strat The MFCC_VF represents the voiceprint feature vector of the starting frame. nowThis represents the voiceprint feature vector of the current frame. If the voiceprint similarity between the current frame and the starting audio frame is greater than the threshold Hold_S, then the next frame of the current frame is set to Now, the voiceprint similarity is calculated again, and compared with the threshold. If the voiceprint similarity between the current frame and the starting audio frame is less than the threshold Hold_S, then Now is marked as the end frame 1, denoted as end1, and end1 is sent to Sum_END. At the same time, Now is set as Start, and the next frame of Start is set to Now. This method is used to continuously find the next end frame, marking the found end frames in chronological order until the audio ends. Finally, Sum_END = {end1, end2, ..., end} n-1}, where n-1 represents the number of ending frames. Finally, the first frame and the frame preceding end1 are clustered into the same audio segment, and end1 and the frame preceding end2 are clustered into the same audio segment. This process is repeated for Sum_END, ultimately dividing the service audio into multiple segments. The set of segmented audio is represented as Sum_Segment = {Voice_Segment1, ..., Voice_Segment2}. t , ..., Voice_Segment n}, where Voice_Segment t Let t represent the t-th audio segment, and n represent the number of audio segments that Voice_Service is divided into.

[0020] S21: Establish a customer service personnel voice library, which stores customer service personnel voices collected in quiet environments, represented as Sum_Service = {Sound_Service1, ..., Sound_Service...} st , ...,Sound_Service c}, where c represents the number of customer service representatives, and Sound_Service st This indicates the voice of the st-th customer service representative, with st serving as the representative's ID.

[0021] S22: First, train the voiceprint recognition model EcapaTdnn. Use the customer service voice recordings from Sum_Service obtained in step S21 as the input for model training, with the label set to the customer service personnel number. EcapaTdnn denoises the input customer service voice recordings, segments them into frames, applies Fast Fourier Transform (FFT) to calculate the power spectrum of each frame, combines the power spectra using filters to obtain the Melp plot, and performs Discrete Cosine Transform (DCT) on the Melp plot to finally obtain the MFCC features of each audio frame. The model outputs the customer service personnel number corresponding to each audio frame based on the MFCC features, and adjusts the model parameters according to the difference between the customer service personnel number and the label. Iterate through Sum_Service, using all customer service personnel voice recordings to train the model. In the testing phase, use the segmented audio from Sum_Segment obtained in step S12 as the model input, with the t-th audio segment (Voice_Segment) as the input. t For example, the model outputs customer service voice data from the Voice_Segment library. t The most similar customer service ID, and the customer service voice message with Voice_Segment t The similarity between the voices is calculated. If the similarity between the most similar customer service voices is below the threshold Hold_C, then the Voice_Segment is determined. t For customer voice messages, label them as 0; otherwise, use the customer service number output by the model to label the Voice_Segment. t , represented as k represents the customer service ID. Iterate through Sum_Segment, marking the customer service ID for each audio segment, count the number of customer service IDs, and denote the customer service ID with the highest frequency as Mk. Use Mk to mark the service audio, denoted as Voice_Service. Mk ;

[0022] S23: Use a deep learning model for speech recognition. The recognition process is as follows: Train the speech recognition model ASRT using a Chinese speech dataset. ASRT transcribes the sound into Chinese Pinyin, and then uses a language model to convert the Pinyin sequence into Chinese text. Convert Sum_Segment into speech text, where Sum_Segment contains the t-th audio segment Voice_Segment. t After being converted to speech-to-text, it is represented as Voice_Text t The obtained voice text set is represented as Sum_Text = {Voice_Text1, ..., Voice_Text}. t , ..., Voice_Text n Note that at this point, the speech text in Sum_Text corresponds to the segmented audio in Sum_Segment.

[0023] Furthermore, step S3 includes the following steps:

[0024] S31: Establish a text keyword database capable of determining speaker behavior, denoted as Sum_Key = {key1, ..., key...} w , ..., key m}, key w This indicates that the w-th row contains keywords, and m represents the number of keywords in the keyword library;

[0025] S32: Establish a keyword acquisition module to retrieve all behavioral keywords in the Sum_Text obtained in step S23.

[0026] Furthermore, the keyword acquisition module described in step S32 consists of three parts: a word segmentation and alignment module, a tensor quantization module, and a replacement module. The keyword acquisition process is as follows:

[0027] S321: The word segmentation alignment module will perform word segmentation on the speech text in Sum_Text, and simultaneously align the number of words in the segmented speech text. Taking the t-th segment of speech text in Sum_Text as an example (Voice_Text),... t For example, input the data into the jieba word segmenter for word segmentation. After segmentation, the data is aligned according to the set maximum number of segments. If the number of segments is lower than the maximum number of segments, "null" is used to pad the data; otherwise, the data is truncated. The resulting segmented text is represented as Cut_VT. t ={word t，1 , ..., word t，j , ..., word t，Max_Word}, where word t，j Indicates Cut_VT t The j-th word in the text is defined, and Max_Word represents the maximum number of words. Iterate through Sum_Text, performing word segmentation on each segment of the audio text to obtain a set of segmented texts represented as Sum_Cut = {Cut_VT1, ..., Cut_VT}. t , ..., Cut_VT n}, where Cut_VT t Voice_Text t The segmented text after word segmentation;

[0028] S322: In the Harbin Institute of Technology (HIT) thesaurus, words are categorized into 6 levels from smallest to largest. The tensor quantization module will tensor each word into a vector of length 6 according to its position in the HIT thesaurus. First, all behavioral keywords in the Sum_Key obtained in S31 are tensor quantized, where the w-th behavioral keyword is key. wThe tensor quantization process is as follows: according to the key w In the six category levels of the thesaurus, the key will be... w This is converted into a vector that can represent 6 category numbers, represented as follows:

[0029] D_key w ={F_Key w，1 , ..., F_Key w，z , ..., F_Key w，6},z∈(1,6)

[0030] F_Key w，z Represents key w The category number at level z in the thesaurus. Traverse Sum_Key, vectorize all behavioral keywords, and obtain the set of behavioral keyword vectors represented as Sum_D_key = {D_key1, ..., D_key...} w , ..., D_key m The keyword vector set has a dimension of m*6, where m represents the number of keywords in the keyword library. All segmented texts obtained from Sum_Cut in S31 are tensorized, where the t-th segmented text is Cut_VT. t The j-th word segment t，j After vectorization, it is represented as D_word t，j ={F_Word t，j，1 F_Word t，j，2 , ......, F_Word t，j，6 At this point, the word segmentation vector is a one-dimensional vector of length 6. Traverse Cut_VT t Cut_VT t All word segmentation vectors are generated, resulting in Cut_VT. t The vector is represented as D_Cut_VT t ={D_word t，1 , ..., D_word t，j , ..., D_Word t，Max_Word The vector dimension is Max_Word*6. Iterate through Sum_Cut to vectorize all segmented text, and the resulting set of segmented vectors is represented as Sum_D_Cut={D_Cut_VT1,...,D_Cut_VT}. t ..., D_Cut_VT n The dimension of the word segmentation vector set is n*Max_Word*6, where n represents the number of audio segments that Voice_Service is segmented into.

[0031] S323: The replacement module replaces a word with its closest behavioral keyword based on its position in the Harbin Institute of Technology (HIT) thesaurus. First, weights are pre-assigned to the six category levels in the HIT thesaurus, denoted as (W1, W2, W3, W4, W5, W6). Then, the j-th word segmentation vector D_word in the t-th segmented text vector of Sum_D_Cut is calculated. t，j The key vector D_key in the w-th row of Sum_D_key w The semantic similarity between them is calculated using the following formula:

[0032]

[0033] Iterate through Sum_D_key and calculate the relationship between each keyword vector and D_word. t，j The semantic similarity between words is used to obtain the behavior keyword vector with the highest semantic similarity, denoted as D_most_key. Finally, the behavior keyword corresponding to D_most_key is used to replace the word. t，j The replaced word segmentation is represented as key_word t，j Note the key_word at this point. t，j These are keywords from the behavioral keyword library. Iterate through D_Cut_VT. t Repeat the above steps, replacing Cut_VT with the behavior keyword. t For each word segment, the keyword replacement text for the t-th segment is represented as key_VT. t ={key_word t，1 , ..., key_word t，j , ..., key_word t，Max_Word The keyword replacement text is a one-dimensional vector of length Max_Word. Finally, iterating through Sum_D_Cut, we replace all segments in each segmented text with the behavioral keywords to obtain the behavioral keyword replacement text set, denoted as Sum_key_VT = {key_VT1, ..., key_VT}. t , ..., key_VT n The set of texts for replacing behavioral keywords has a dimension of n*Max_Word.

[0034] Furthermore, step S4 includes the following steps:

[0035] S41: The behavioral keyword sequence is the behavioral keyword key from the text keyword library established in step S31. w The set of behavior keyword replacement texts, Sum_key_VT, obtained in S323, is used as the component. During the training phase, all Sum_key_VT texts are labeled to construct a behavior label library.

[0036] S42: The speech-text-based speaker behavior recognition model classifies speaker behavior based on the encoded sequence of behavior keywords. This model consists of three convolutional layers, three pooling layers, one fully connected layer, and a softmax activation function. First, one-hot encoding is used to vectorize the Sum_key_VT obtained in step S323, where the t-th keyword is replaced by the text key_VT. t After vectorization, it is represented as D_key_VT t , D_key_VT t The input undergoes feature extraction through three convolutional layers and three pooling layers, followed by processing through a softmax layer, ultimately yielding an array of dimension b, represented as a sequence. t =(p t，1 p t，2 , ..., p t，ad , ..., p t，b ), b represents the number of actions set, p t，ad D_key_VT represents the probability of the ad-th behavior occurring. t The final classification result is represented as Action. t Action t =MAX(p t，1 p t，2 , ......, p t，b Iterate through Sum_key_VT to obtain the classification results of each keyword replacement text, and use the classification results to construct a service action sequence, represented as Action_seq = {Action1, Action2, ..., Action...} n}, where n represents Voice_Service Mk The number of segments is used; note that Action_seq at this point is Voice_Service. Mk The corresponding service behavior sequence.

[0037] Furthermore, step S5 includes the following steps:

[0038] S51: During the training phase, multiple service audios are input, and service behavior sequences obtained from the multiple service audios are labeled to establish a service content tag library.

[0039] S52: Train a service content recognition model using the service content tag library obtained in S51. Align the lengths of all service behavior sequences (Action_seq) obtained in step S42, tensor them, and input them into the trained service content recognition model to achieve recognition of each Voice_Service. MkService content is categorized and identified, and also according to Voice_Service Mk The marked customer service number identifies the customer service representative who will provide the service.

[0040] Furthermore, the service content recognition model described in step S52 consists of two parts: an alignment module and a classification module. The service content recognition process is as follows:

[0041] S521: The alignment module aligns the length of all Action_seq according to the set action sequence length, which is represented as Max_Action. Missing actions are padded with "null", and excessive actions are truncated. Finally, the aligned Action_seq is one-hot encoded to obtain the service action sequence vector.

[0042] S522: The classification module consists of three convolutional layers, three pooling layers, one fully connected layer, and a softmax activation function. The service behavior sequence vector is input into the three convolutional and three pooling layers for feature extraction, and then processed by the softmax layer to finally obtain an array of dimension 'a', represented as Service. Sel = (p1, p2, ..., p sd , ..., p a ), where 'a' represents the number of service items set, and 'p' represents the number of service items set. sa Let represent the probability of the occurrence of the sd-th type of service content. The final classification result of the service behavior sequence vector is represented as Service, where Service = max(p1, p2, ..., p...). a ); Service refers to the model prediction Voice_Service Mk The represented service content, and Voice_Service Mk The customer service personnel marked in the middle are Mk, Mk∈(1,c), c represents the number of customer service personnel, and Voice_Service is the working voice of customer service personnel Mk to complete the Service.

[0043] The beneficial effects of this invention are as follows:

[0044] (1) This patent proposes a voice service content recognition method for behavioral text processing. By acquiring the service audio between customer service and customer, the service content of customer service is identified. By analyzing behavioral keywords in the service audio, the service behavior sequence of customer and customer service during the service process is determined, and the service content of customer service is predicted according to the service behavior sequence, thereby determining the work content to be completed by customer service.

[0045] (2) This patent compares the positions of service audio segmentation and service behavior keywords in the Harbin Institute of Technology thesaurus to determine the behavior keywords that are closest to each segmentation. It then uses the behavior keywords to replace the service audio segmentation to construct a behavior keyword sequence, which improves the accuracy of service behavior recognition. Finally, it constructs a behavior sequence based on the recognized service behavior and determines the customer service content by referring to the behavior sequence.

[0046] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0048] Figure 1 This is a flowchart of a service content recognition method for behavioral text processing disclosed in an embodiment of the present invention;

[0049] Figure 2 This is a diagram of the service speaker behavior recognition model disclosed in an embodiment of the present invention;

[0050] Figure 3 This is a service content identification model diagram disclosed in an embodiment of the present invention. Detailed Implementation

[0051] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0052] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0053] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0054] like Figure 1 As shown, the present invention provides a method for recognizing speech service content in behavioral text processing, comprising the following steps:

[0055] S1: Set up a voice acquisition device to collect audio recordings of communication between customer service representatives and customers. When customer service representatives provide services to customers by phone, the connection and disconnection of the phone usually represent the start and end of a service session. Therefore, the audio between each connection and disconnection is treated as a complete service audio recording, which is then framed. The framed audio is clustered based on the voiceprint characteristics of each frame to obtain customer service audio information.

[0056] S11: The voice acquisition device needs to acquire the service audio of each service completed by the customer service representative. Therefore, this patent requires the voice acquisition device to be able to clearly record the dialogue between the customer service representative and the customer in a noisy environment so that it can meet the requirements of voice recognition.

[0057] S12: Obtain the service audio from the customer service representative during their work. Each time a customer service representative answers a call and hangs up, the audio is considered a complete service audio segment, denoted as Voice_Service. Divide Voice_Service into equal-length frames, with the i-th frame denoted as Voice_Frames. i Let i ∈ (1, L), where L represents the number of frames in Voice_Service. The Voice_Frames are calculated using Fast Fourier Transform (FFT). i The power spectrum is processed using a Mel filter bank, then transformed into the logarithmic domain, and finally processed again using discrete cosine transform to obtain Voice_Frames. i The voiceprint feature vector represents MFCC_VF i .

[0058] S13: Calculate the voiceprint similarity between the voiceprint feature vectors of each frame. Cluster service audio frames with similarity meeting the threshold (80%), and finally divide Voice_Service into multiple audio segments based on voiceprint similarity. The clustering process is as follows: Create an end frame set, using the end frame as the end of a service audio frame cluster. The end frame set is represented as Sum_END={}. First, set the first frame as the starting audio frame (Start), and the second frame as the current frame (Now). Compare the voiceprint similarity between Start and Now. The calculation formula is: Among them, MFCC_VF Strat The MFCC_VF represents the voiceprint feature vector of the starting frame. now This represents the voiceprint feature vector of the current frame. If the voiceprint similarity between the current frame and the starting audio frame is greater than the threshold Hold_S, then the next frame of the current frame is set to Now, the voiceprint similarity is calculated again, and compared with the threshold. If the voiceprint similarity between the current frame and the starting audio frame is less than the threshold Hold_S, then Now is marked as the end frame 1, denoted as end1, and end1 is sent to Sum_END. At the same time, Now is set as Start, and the next frame of Start is set to Now. This method is used to continuously find the next end frame, marking the found end frames in chronological order until the audio ends. Finally, Sum_END = {end1, end2, ..., end} n-1}, where n-1 represents the number of ending frames. Finally, the first frame and the frame preceding end1 are clustered into the same audio segment, and end1 and the frame preceding end2 are clustered into the same audio segment. This process is repeated for Sum_END, ultimately dividing the service audio into multiple segments. The set of segmented audio is represented as Sum_Segment = {Voice_Segment1, ..., Voice_Segment2}. t , ..., Voice_Segment n}, where Voice_Segment t Let t represent the t-th audio segment, and n represent the number of audio segments that Voice_Service is divided into.

[0059] S2: Establish a customer service voice database and use a voiceprint recognition model to identify the customer service representative in the service audio. Use a speech recognition model to convert the service audio into speech-to-text.

[0060] S21: Establish a customer service personnel voice library, which stores customer service personnel voices collected in quiet environments, represented as Sum_Service = {Sound_Service1, ..., Sound_Service...} st , ...,Sound_Service c}, where c represents the number of customer service representatives, and Sound_Service st This indicates the voice of the st-th customer service representative, with st serving as the representative's ID.

[0061] S22: Use a voiceprint recognition model to determine the customer service representative's identity in the service audio. During the training phase, the customer service voice recordings obtained in step S21 (Sum_Service) are used as input to the model, with the label set to the customer service representative's ID. The model denoises the input customer service voice, segments it into frames, applies Fast Fourier Transform (FFT), calculates the power spectrum of each frame, combines the power spectra using filters to obtain a Melp plot, performs a Discrete Cosine Transform (DCT) on the Melp plot, and finally obtains the MFCC features of each audio frame. The model outputs the customer service representative's ID corresponding to each audio frame based on the MFCC features, and adjusts the model parameters according to the difference between the customer service representative's ID and the label. The model is trained by traversing Sum_Service and using all customer service representative voice recordings. During the testing phase, the segmented audio from Sum_Segment obtained in step S12 is used as input to the model, with the t-th audio segment (Voice_Segment) as the input. t For example, the model outputs customer service voice data from the Voice_Segment library. t The most similar customer service ID, and the customer service voice message with Voice_Segment t The similarity between the voices is calculated. If the similarity between the most similar customer service voices is below the threshold Hold_C, then the Voice_Segment is determined. t For customer voice messages, label them as 0; otherwise, use the customer service number output by the model to label the Voice_Segment. t , represented as k represents the customer service ID. Iterate through Sum_Segment, marking the customer service ID for each audio segment, count the number of customer service IDs, and denote the customer service ID with the highest frequency as Mk. Use Mk to mark the service audio, denoted as Voice_Service. Mk .

[0062] Optionally, EcapaTdnn or GMM can be used as the voiceprint recognition model.

[0063] S23: Use a deep learning model for speech recognition. The recognition process is as follows: Train a speech recognition model using a Chinese speech dataset. After transcribing the sound into Chinese Pinyin, use a language model to convert the Pinyin sequence into Chinese text. Convert Sum_Segment into speech text, where Sum_Segment contains the t-th audio segment, Voice_Segment. t After being converted to speech-to-text, it is represented as Voice_Text tThe obtained voice text set is represented as Sum_Text = {Voice_Text1, ..., Voice_Text}. t , ..., Voice_Text n Note that at this point, the speech text in Sum_Text corresponds to the segmented audio in Sum_Segment.

[0064] Optionally, ASRT can be used as a speech recognition model to convert audio into speech text.

[0065] S3: Establish a text keyword library capable of identifying different behavioral content and a keyword acquisition module capable of calculating and retrieving keywords from speech text.

[0066] S31: Establish a text keyword library capable of identifying speaker behavior. In customer service and customer phone communication scenarios, behavioral keywords mainly include: hello, product, refund, question, purchase, etc. The text keyword library is represented as Sum_Key = {key1, ..., key...} w , ..., key m}, key w This indicates that the w-th row contains keywords, and m represents the number of keywords in the keyword library.

[0067] S32: Establish a speech-text keyword acquisition module to retrieve all behavioral keywords from the Sum_Text obtained in step S23. The speech-text keyword acquisition module consists of three parts: a word segmentation and alignment module, a tensor quantization module, and a replacement module. The speech-text keyword acquisition steps are as follows:

[0068] S321: The word segmentation alignment module will perform word segmentation on the speech text in Sum_Text, and simultaneously align the number of words in the segmented speech text. Taking the t-th segment of speech text in Sum_Text as an example (Voice_Text),... t For example, the input is given to a word segmenter for word division. After segmentation, the word count is aligned according to the set maximum number of word segments. If the number of word segments is lower than the maximum number of word segments, "null" is used to pad the count; otherwise, the segmented text is truncated. The resulting segmented text is represented as Cut_VT. t ={word t，1 , ..., word t，j , ..., word t，Max_Word}, where word t，j Indicates Cut_VT t The j-th word in the text is defined, and Max_Word represents the maximum number of words. Iterate through Sum_Text, performing word segmentation on each segment of the audio text to obtain a set of segmented texts represented as Sum_Cut = {Cut_VT1, ..., Cut_VT}.t , ..., Cut_VT n}, where Cut_VT t Voice_Text t The segmented text after word segmentation.

[0069] Optionally, the jieba, LAC, LTP, or THULAC word segmenters can be used to segment the speech text into words.

[0070] S322: In the Harbin Institute of Technology (HIT) thesaurus, words are categorized into 6 levels from smallest to largest. The tensor quantization module will tensor each word into a vector of length 6 according to its position in the HIT thesaurus. First, all behavioral keywords in the Sum_Key obtained in S31 are tensor quantized, where the w-th behavioral keyword is key. w The tensor quantization process is as follows: according to the key w In the six category levels of the thesaurus, the key will be... w This is converted into a vector that can represent 6 category numbers, denoted as D_key. w ={F_Key w，1 , ..., F_Key w，z , ..., F_Key w，6}, z∈(1,6), F_Key w，z This represents the category number of keyw at level z in the thesaurus. Iterate through Sum_Key, vectorizing all behavioral keywords to obtain the set of behavioral keyword vectors represented as Sum_D_key = {D_key1, ..., D_key...}. w , ..., D_key m The keyword vector set has a dimension of m*6, where m represents the number of keywords in the keyword library. All segmented texts obtained from Sum_Cut in S31 are tensorized, where the t-th segmented text is Cut_VT. t The j-th word segment t，j After vectorization, it is represented as D_word t，j ={F_Word t，j，1 F_Word t，j，2 , ......, F_Word t，j，6 At this point, the word segmentation vector is a one-dimensional vector of length 6. Traverse Cut_VT t Cut_VT t All word segmentation vectors are generated, resulting in Cut_VT. t The vector is represented as D_Cut_VT t ={D_word t，1 , ..., D_word t，j , ..., D_Wordt，Max_Word The vector dimension is Max_Word*6. Iterate through Sum_Cut to vectorize all segmented text, and the resulting set of segmented vectors is represented as Sum_D_Cut={D_Cut_VT1,...,D_Cut_VT}. t ..., D_Cut_VT n The dimension of the word segmentation vector set is n*Max_Word*6, where n represents the number of audio segments that Voice_Service is segmented into.

[0071] S323: The replacement module replaces a word with its closest behavioral keyword based on its position in the Harbin Institute of Technology (HIT) thesaurus. First, weights are pre-assigned to the six category levels in the HIT thesaurus, denoted as (W1, W2, W3, W4, W5, W6). Then, the j-th word segmentation vector D_word in the t-th segmented text vector of Sum_D_Cut is calculated. t，j The key vector D_key in the w-th row of Sum_D_Cut w The semantic similarity between them is calculated using the following formula: Iterate through Sum_D_key and calculate the relationship between each keyword vector and D_word. t，j The semantic similarity between words is used to obtain the behavior keyword vector with the highest semantic similarity, denoted as D_most_key. Finally, the behavior keyword corresponding to D_most_key is used to replace the word. t，j The replaced word segmentation is represented as key_word t，j Note the key_word at this point. t，j These are keywords from the behavioral keyword library. Iterate through D_Cut_VT. t Repeat the above steps, replacing Cut_VT with the behavior keyword. t For each word segment, the keyword replacement text for the t-th segment is represented as key_VT. t ={key_word t，1 , ..., key_word t，j , ..., key_word t，Max_Word The keyword replacement text is a one-dimensional vector of length Max_Word. Finally, iterating through Sum_D_Cut, we replace all segments in each segmented text with the behavioral keywords to obtain the behavioral keyword replacement text set, denoted as Sum_key_VT = {key_VT1, ..., key_VT}. t , ..., key_VT n The set of texts for replacing behavioral keywords has a dimension of n*Max_Word.

[0072] S4: Service behavior identification process as follows Figure 2 As shown. A speaker behavior recognition model is established; a behavior tag library is built, and the constructed speaker behavior recognition model is trained to determine speaker behavior through behavior keyword sequences;

[0073] S41: Behavioral keyword sequence is the behavioral keyword key from the text keyword library established in S31. w The system consists of the behavior keyword replacement text set Sum_key_VT obtained in S323. A behavior tag library is then built to determine the speaker's behavior through the sequence of behavior keywords. Customer service and customer behavior tags mainly include: greetings, inquiries, complaints, purchases, refunds, thanks, farewells, etc. During the training phase, all Sum_key_VT strings are tagged to construct the behavior tag library.

[0074] S42: The speech-text-based speaker behavior recognition model can classify speaker behavior based on the encoded keyword sequence. First, one-hot encoding is used to vectorize the Sum_key_VT obtained in step S323, where the t-th keyword replacement text is key_VT. t After vectorization, it is represented as D_key_VT t The vector dimension is Max_Word*256, and D_key_VT t The input classification module consists of three convolutional layers, three pooling layers, a flattening function, a fully connected layer, and a softmax activation function. The first convolutional layer has a dimension of 256, a kernel size of 3, a ReLU activation function, and padding of 'same'. Its pooling layer is a max-pooling layer with a size of 38. The second convolutional layer has the same dimension as the first, a kernel size of 4, a ReLU activation function, and padding of 'same'. Its pooling layer is a max-pooling layer with a size of 37. The third convolutional layer maintains the same dimension, has a kernel size of 5, a ReLU activation function, and padding of 'same'. Its pooling layer is a max-pooling layer with a size of 36. The dimension of the fully connected layer is determined by the number of action labels, and its activation function is softmax. During the training phase, the input action label library is vectorized into a Max_Word*256 two-dimensional vector, which is then fed into three convolutional layers and three pooling layers for feature extraction. The extracted features are flattened into a one-dimensional vector by a flattening function, and then processed by a Softmax layer to obtain the final predicted value. The loss function Loss is used to calculate the error between the predicted and true values, and the loss is used to optimize the classification module parameters (learning rate set to 0.005, optimization method: SGD (stochastic gradient descent)). In real-world scenarios, D_key_VTt Feature extraction is performed using three convolutional layers and three pooling layers. The extracted features are then flattened into a one-dimensional vector using a flattening function, and further processed by a softmax layer to finally obtain a vector of dimension b, represented as a sequence. t =(p t，1 p t，2 , ......, p t，b ), b represents the number of actions set, p t，z D_key_VT represents the probability of the z-th behavior occurring. t The final classification result is represented as Action. t Action t =MAX(p t，1 p t，2 , ......, p t，b Iterate through Sum_key_VT to obtain the classification results for each keyword replacement text. Use the classification results to construct a service action sequence, represented as Action_seq = {Action1, Action2, ..., Action...} n}, where n represents Voice_Service Mk The number of segments into which it is divided.

[0075] S5: Service content identification process as follows Figure 3 As shown, a service content tag library is established to train a service content recognition model. This model can determine customer service content based on behavioral sequences.

[0076] S51: During the training phase, multiple service audio recordings are input, and service behavior sequences obtained from these recordings are labeled to establish a service content tag library. The service content tags for telephone customer service and customers mainly include: consultation service, refund service, complaint service, and purchase service.

[0077] S52: Service content recognition model, consisting of an alignment module and a classification module. During model training, the service content label library obtained in S51 is used. During testing, the length of the service behavior sequence Action_seq obtained in S42 is aligned, Action_seq is tensorized, and finally fed into the classification module to classify Voice_Service. Mk The services are categorized according to their content, and also categorized by Voice_Service Mk The assigned customer service number identifies the customer service representative performing the service. The service content identification process is as follows:

[0078] S521: The alignment module first aligns the Action_seq data to its length, then converts the aligned Action_seq into a service behavior vector using one-hot encoding. The Action_seq length is then aligned to the set behavior sequence length, denoted as Max_Action. Missing behaviors are padded with "null", and excessive behaviors are truncated. Finally, the aligned Action_seq is tensorized using one-hot encoding. The resulting service behavior sequence vector is denoted as D_Action_seq, with dimensions of Max_Action * 256.

[0079] S522: The classification module consists of three convolutional layers, three pooling layers, one flattening function, one fully connected layer, and one softmax activation function. The first convolutional layer has a dimension of 256, a kernel size of 3, a ReLU activation function, and padding = 'same'. Its pooling layer is a max pooling layer with a pooling size of 38. The second convolutional layer has a dimension of 256, a kernel size of 4, a ReLU activation function, and padding = 'same'. Its pooling layer is a max pooling layer with a pooling size of 37. The third convolutional layer has a dimension of 256, a kernel size of 5, a ReLU activation function, and padding = 'same'. Its pooling layer is a max pooling layer with a pooling size of 36. The dimension of the fully connected layer is determined by the number of tags in the service content, and its activation function is softmax. During the training phase, the service content recognition model is trained using Service_label. First, the Service_label is input to the alignment module, vectorized using one-hot encoding, and then converted into a 2D vector of Max_Action*256. This vector is then input to three convolutional layers and three pooling layers for feature extraction. The extracted features are input to a flattening function to convert them into a 1D vector, and then processed by a Softmax layer to obtain the predicted value. The loss function, Loss, is the error between the true and predicted values. Loss is used to optimize the classification module parameters (learning rate set to 0.005, optimization method: SGD (stochastic gradient descent)). In practical use, D_Action_seq is input to three convolutional layers and three pooling layers for feature extraction, processed by a flattening function, and then by a Softmax layer, finally obtaining a vector of dimension a, represented as Service_Sel = (p1, p2, ..., p...). a ), where 'a' represents the number of service items set, and 'p' represents the number of service items set. sdLet D_Action_seq represent the probability of the sd-th type of service content appearing. The final classification result of D_Action_seq is represented as Service, where Service = max(p1, p2, ..., p...). a Service refers to the model prediction Voice_Service. Mk The represented service content is audio, and Voice_Service Mk The customer service representative marked in the middle is Mk, where Mk∈(1,m), and m represents the number of customer service representatives. Therefore, Voice_Service is the working voice of customer service representative Mk to complete the Service.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A voice service content recognition method based on behavioral text processing, characterized by: The method comprises the following steps: S1: setting a voice collection device to obtain working audio of a customer service staff; taking audio between each time of connection and hang-up of the customer service staff as a complete service audio, and performing frame division on the audio, clustering the frame-divided audio according to a voiceprint feature of each frame of audio to obtain customer service audio information; S2: establishing a customer service staff timbre library, using a voiceprint recognition model to determine the identity of the customer service staff in the service audio, and using a speech recognition model to convert the service audio into speech text; S3: establishing a text keyword library for determining service behavior, and establishing a keyword acquisition module for calculating and retrieving behavior keywords in the speech text; S4: establishing a speaker behavior recognition model, establishing a behavior label library, training the constructed speaker behavior recognition model, and determining speaker behavior through a behavior keyword sequence; S5: establishing a service content recognition model; establishing a service content label library, training the constructed service content recognition model, and determining customer service content; The step S3 comprises the following steps: S31: Establish a text keyword library capable of determining the speaker behavior, denoted as , denotes the th behavior keyword, represents the number of keywords in the keyword library; S32: Establish a keyword acquisition module to retrieve the keywords obtained in step S23. Keywords for all behaviors; The keyword acquisition module comprises three parts of a word segmentation alignment module, a tensorization module and a replacement module; the speech text keyword acquisition step is as follows: S321: The word segmentation alignment module will... The speech text is segmented into words, and the number of words in the segmented speech text is aligned; for The Middle Segmented audio text Enter it The word segmenter divides words into segments and aligns them according to the set maximum number of segments. If the number of segments is less than the maximum, "null" is used to pad the number; otherwise, the segments are truncated. The resulting segmented text is represented as follows: ,in express The Middle Each word, Indicates the maximum number of word segments; traversal Each segment of audio text is segmented into words to obtain a set of segmented texts. ,in express The segmented text after word segmentation; S322: The tensor module will tensor each word into a vector of length 6 according to its position in the Harbin Institute of Technology thesaurus; firstly, the vector obtained in S31 will be... All behavioral keywords in the text are quantified, among which the first... Key behavioral terms Zhang's quantification process is as follows: According to... The category numbers in the six classification levels of the thesaurus will be This is converted into a vector that can represent 6 category numbers, represented as follows: ， represent In the thesaurus Category number in the level; traversal Vectorize all behavioral keywords to obtain a set of behavioral keyword vectors. The keyword vector set has a dimension of , This represents the number of keywords in the keyword database; the number obtained from S31 All word segmentation tensor quantizations in, where the first Segmented text The Middle Each word After vectorization, it is represented as At this point, the word segmentation vector is a one-dimensional vector of length 6; traversal ,Will All word segmentation vectors are obtained. Vector representation is The vector dimension is ; Traversal Vectorize all segmented text, and the resulting set of segmented vectors is represented as follows: The dimension of the word segmentation vector set is , express The number of audio segments that were divided; S323: The replacement module replaces a word with its closest behavioral keyword based on its position in the Harbin Institute of Technology (HIT) thesaurus. First, weights are pre-assigned to the six category levels in the HIT thesaurus, represented as... ;calculate The Middle The first segment of the word segment text vector Each word segmentation vector and The Middle Individual behavioral keyword vectors The semantic similarity between them is calculated using the following formula: traverse , calculate semantic similarity between each keyword vector and , obtain the behavior keyword vector with the maximum semantic similarity, denoted as , finally replace with the corresponding behavior keyword , the replaced segmented text is denoted as , is a keyword in the behavior keyword library; traverse , repeat the above operation to replace each segmented text in with a behavior keyword, and obtain the th keyword replacement text denoted as , the keyword replacement text is a one-dimensional vector with a length of ; finally traverse , replace all segmented texts in each segmented text with a behavior keyword to obtain a behavior keyword replacement text set denoted as , the behavior keyword replacement text set has a dimension of ; The step S4 comprises the following steps: S41: the behavior keywords in the text keyword library established by step S31 composition, i.e., the behavior keywords obtained in S323 replace the text set ; in the training stage, all are tagged to build a behavior label library; S42: The speaker behavior recognition model based on speech text classifies the behavior of the speaker according to the coded behavior keyword sequence, and the model comprises three convolutional layers, three pooling layers, one fully connected layer and one activation function constituted; first adopt Encoding the behavior keyword sequence obtained in step S323 Vectorization, where the Segment keyword replacement text After vectorization, it is represented as , Input into three convolutional layers and three pooling layers for feature extraction, and then pass through Layer processing, finally obtain an array with a dimension of , represented as , , represents the number of behaviors set, , represents the probability of the occurrence of the th behavior, The final classification result is represented as , ; traverse , obtain the classification result of each segment keyword replacement text, use the classification result to construct the service behavior sequence, represented as , represents The number of segments divided, note that at this time is The service behavior sequence corresponding to.

2. The voice service content recognition method based on behavioral text processing according to claim 1, characterized in that: The step S1 specifically comprises the following steps: S11: Obtain the service audio during the working time of the customer service personnel, and take the audio between each time of connection and disconnection of the customer service as a complete service audio, denoted as ; divide into frames of equal length, and the first frame is denoted as , , , the frame number of , apply fast Fourier transform (FFT) to obtain the power spectrum of , use a mel filter to perform filter bank processing on the power spectrum, transform the processed power spectrum into a logarithmic domain, and then use a discrete cosine transform to process the power spectrum, to finally obtain the voiceprint feature vector of , denoted as ; S12: Calculate the voiceprint similarity between each frame of voiceprint feature vectors, and cluster the service audio frames with similarity higher than a threshold Finally, the voiceprint similarity is referred to cluster the multi-segment audio. ​ 3. The voice service content recognition method based on behavioral text processing according to claim 2, characterized in that: The clustering process in the step S12 is as follows: A set of end frames is created to mark the end of a cluster of service audio frames, and the set of end frames is represented as , the first frame is set as the start audio frame, represented as , the second frame is set as the current frame, represented as , and the voiceprint similarity between and is compared, and the calculation formula is: in This represents the voiceprint feature vector of the starting frame. This represents the voiceprint feature vector of the current frame. If the voiceprint similarity between the current frame and the starting audio frame is greater than a threshold... Then set the next frame of the current frame to... The voiceprint similarity is recalculated and compared with the threshold; if the voiceprint similarity between the current frame and the starting audio frame is less than the threshold... Then Marked as end frame 1, indicating ,Will Send in At the same time Set as ,Will Next frame set to ; Continue searching for the next ending frame using this method, marking the found ending frames in chronological order, until the audio ends, and finally... , Indicates the number of ending frames; finally, the first frame is compared with... The previous frame was clustered into the same audio segment, and and The previous frame is clustered into an audio segment, and the process is repeated in this manner. Ultimately, the service audio is divided into multiple audio segments, and the set of segmented audio is represented as... ,in Indicates the first audio segment express The number of audio segments that are divided.

4. The voice service content recognition method based on behavioral text processing according to claim 1, characterized in that: The step S2 comprises the following steps: S21: Establish a customer service staff voice library, wherein the customer service staff voice collected in a quiet environment is stored, denoted as , denotes the number of customers, denotes the voice of the th customer service staff, and is the number of the customer service staff; S22: First, train the voiceprint recognition model. The result obtained in step S21 The customer service voice messages were used as input for model training, and the labels were set to the customer service personnel numbers; using... The input customer service voice is denoised, segmented into frames, and then Fast Fourier Transform is applied. The power spectrum of each frame is calculated, and the power spectra are combined using filters to obtain the Melp plot. A discrete cosine transform is then performed on the Melp plot to finally obtain the audio signal for each frame. Features; Model based on The feature output identifies the customer service number corresponding to each audio frame, and the model parameters are adjusted based on the difference between the customer service number and the label. This process is iterated through... The model was trained using the voice recordings of all customer service personnel; during the testing phase, the results obtained in step S12 were used... The segmented audio in the model is used as input, with the first segment being the second segment. audio segment For example, the model outputs customer service voice data from the database. The most similar customer service number, and the customer service voice and The similarity between the voices; if the most similar customer service voice has a similarity below a threshold Then judge Voice message for customers, marked as higher than Then use the customer service number output by the model as a marker. , represented as , , Represents customer service number; iterates through... Mark the customer service number for each audio segment, count the number of customer service numbers, and represent the customer service number that appears most frequently. ,use Tag service audio, represented as ; S23: Use a deep learning model for speech recognition. The recognition process is as follows: Train a speech recognition model using a Chinese speech dataset. , After transcribing the sound into Chinese Pinyin, the Pinyin sequence is converted into Chinese text using a language model; Convert to voice-to-text The Middle audio segment After being converted to speech text, it is represented as The obtained voice-text set is represented as Note this time The voice text in The segmented audio in the audio is mutually corresponding.

5. The voice service content recognition method based on behavioral text processing according to claim 1, characterized in that: The step S5 comprises the following steps: S51: in a training stage, inputting multiple service audios, marking service behavior sequences obtained through the multiple service audios, and establishing a service content label library; S52: training the service content identification model using the service content label library obtained in S51; aligning all service behavior sequences obtained in S42 according to the length, and inputting the same into the trained service content identification model to realize classification and identification of the service content of each , and determining the customer service personnel performing the service according to the customer service number marked. ​​ 6. The voice service content recognition method based on behavioral text processing according to claim 5, characterized in that: The service content recognition model in the step S52 is composed of an alignment module and a classification module, and the service content recognition process is as follows: S521: The alignment module will align all The length is aligned according to the set behavior sequence length, which is expressed as: Missing behaviors are padded with "null", excessive behaviors are truncated, and finally the aligned behaviors are... conduct Encode to obtain a service behavior sequence vector; S522: The classification module is composed of three convolutional layers, three pooling layers, one fully connected layer and one activation function The service behavior sequence vector is input into three convolutional layers and three pooling layers for feature extraction, and then processed by a fully connected layer and an activation function to obtain an array with a dimension of , which is represented as represents the number of set service contents, represents the probability of the occurrence of the th service content, and the final classification result of the service behavior sequence vector is represented as , ; , which is the service content predicted by the model , and the customer service personnel marked in is , , , which represents the number of customer service personnel, is the working voice of the customer service personnel to complete the service.​

Images