Voice recognition-based nano-clothing auxiliary method, device and equipment and medium

Abstract

The invention discloses a voice recognition-based servo assisting method, device and equipment and a medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining the audio clip data of a taxpayer and a servo person in a call process from a preset database service cluster through a preset audio sliding window mode, translating the spliced audio through the first voice recognition service; judging whether the text length of the spliced text translated by the calling party exceeds a preset trigger threshold value or not, and triggering text error correction and new question extraction operation based on a judgment result and a first preset agent; performing knowledge indexing on the target problem based on the vector knowledge base; and after the call is ended, carrying out text recombination on the translated spliced text meeting a preset call duration condition based on a second voice recognition service, and filling a telephone traffic settlement through a second preset agent and a question and answer text. And the target question is obtained based on the voice recognition service cluster and the target large language model, so that the server personnel can be helped to quickly answer, and the server working efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a service assistance method, device, equipment and medium based on speech recognition. Background Art

[0002] In traditional taxpayer service scenarios, remote taxpayer service personnel need to constantly answer consultation calls. This not only places high demands on the taxpayer service personnel's work ability, but also increases the cost of training new employees. In addition, during peak consultation periods, taxpayer service personnel are also required to fill out call summaries, which places too much burden on taxpayer service personnel.

[0003] From the above, it can be seen that how to improve the quality and efficiency of telephone service while reducing the burden on service personnel and optimizing the personnel structure is an urgent problem that needs to be solved. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method, device, equipment and medium for assisting customer service based on voice recognition, which can improve the quality and efficiency of telephone customer service work while reducing the burden on customer service personnel and optimizing the personnel structure. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a method for assisting a user based on speech recognition, comprising:

[0006] Record the audio of the taxpayer and the taxpayer service personnel during the call to obtain the calling audio and the called audio, and cache the calling audio and the called audio to a preset database service cluster;

[0007] The audio segment data corresponding to the calling audio and the called audio are obtained from the preset database service cluster by a preset audio sliding window method, and the spliced ​​audio corresponding to each of the audio segment data is translated by a first speech recognition service in the speech recognition service cluster to obtain a translated spliced ​​text; the translated spliced ​​text includes a translated spliced ​​text of the calling party and a translated spliced ​​text of the called party;

[0008] Determine whether the text length corresponding to the caller's translated concatenated text exceeds a preset trigger threshold, and based on the determination result and the first preset agent in the target large language model, trigger the operation of performing text error correction and new question extraction on the caller's translated concatenated text to obtain a target question;

[0009] Performing knowledge indexing on the target question based on the vector knowledge base to obtain a target answer, and sending the target answer to a target customer service person so as to conduct a customer service question-and-answer call through the target customer service person;

[0010] After the Nasi Talk Q&A call is over, the translated concatenated text that meets the preset call duration condition is reorganized based on the second speech recognition service in the speech recognition service cluster to obtain a question and answer text, and the call summary is filled in through the second preset intelligent agent in the target language model and the question and answer text.

[0011] Optionally, the obtaining of the audio segment data corresponding to each of the calling audio and the called audio from the preset database service cluster by a preset audio sliding window method, and translating the spliced ​​audio corresponding to each of the audio segment data by a first speech recognition service in the speech recognition service cluster to obtain a translated spliced ​​text includes:

[0012] Performing audio sliding window processing on the target audio stored in the preset database service cluster based on a preset sliding window size to obtain audio segments corresponding to the target audio from the preset database service cluster; wherein the target audio includes the target calling audio or the target called audio;

[0013] Starting from the first audio segment of each audio segment, and based on the principle of non-repetitive selection, each time selecting adjacent target number of audio segments from each audio segment for splicing, so as to obtain corresponding first spliced ​​audios; the target number is at least two;

[0014] Starting from the second audio segment of each of the audio segments, and based on the principle of non-repetitive selection, each time selecting the target number of adjacent audio segments from each of the audio segments for splicing, so as to obtain corresponding second spliced ​​audios;

[0015] Translating each of the first spliced ​​audios and each of the second spliced ​​audios respectively through the first speech recognition service in the speech recognition service cluster to obtain corresponding first translated text segments and second translated text segments, and determining the correspondence between the first translated text segments and the second translated text segments having overlapping areas therebetween;

[0016] The repeated characters in the first translated text segment and the corresponding second translated text segment and the text characters corresponding to the window sliding segmentation point are removed, and then the remaining translated text content is spliced ​​to obtain the translated spliced ​​text corresponding to the target audio.

[0017] Optionally, removing repeated characters in the first translated text segment and the corresponding second translated text segment and text characters corresponding to the window sliding segmentation point, and then splicing the remaining translated text content to obtain the translated spliced ​​text corresponding to the target audio includes:

[0018] Determining whether there are repeated characters in an overlapping area of ​​the first translated text and the corresponding second translated text;

[0019] If there are repeated characters in the overlapping area of ​​the first translated text and the corresponding second translated text, the text after the repeated characters is removed from the first translated text to obtain a first remaining text, and the text after the repeated characters is retained from the second translated text to obtain a second remaining text, and then the first remaining text and the second remaining text are spliced ​​to obtain the translated spliced ​​text corresponding to the target audio;

[0020] If there are no repeated characters in the overlapping area of ​​the first translated text and the corresponding second translated text, the overlapping area on the first translated text is replaced with the second translated text to obtain the translated concatenated text corresponding to the target audio.

[0021] Optionally, the determining whether the text length corresponding to the caller's translated concatenated text exceeds a preset trigger threshold, and triggering the operation of performing text error correction and new question extraction on the caller's translated concatenated text based on the determination result and a first preset agent in the target large language model to obtain a target question, includes:

[0022] Determine whether the length of the text corresponding to the translated and concatenated text of the caller exceeds a preset trigger threshold;

[0023] If the text length corresponding to the caller's translated spliced ​​text in the translated spliced ​​text exceeds a preset trigger threshold, the operations of text correction and new question extraction are triggered on the caller's translated spliced ​​text based on the preset prompt words in the first preset intelligent agent to obtain the target question.

[0024] Optionally, the triggering of text error correction and new question extraction operations on the caller translated and spliced ​​text based on the preset prompt words in the first preset agent to obtain a target question includes:

[0025] Based on the preset prompt words in the first preset agent, the user questions corresponding to the translated concatenated text of the caller are obtained, and the user questions are deduplicated based on the edit distance similarity between the user questions to obtain the target question.

[0026] Optionally, the second speech recognition service in the speech recognition service cluster performs text reorganization on the translated concatenated text that meets a preset call duration condition to obtain a question and answer text, including:

[0027] Obtaining a target call duration based on the calling audio and the called audio, and determining whether the target call duration meets a preset call duration condition;

[0028] If the target call duration satisfies the preset call duration condition, the translated concatenated text is obtained, and text reorganization is performed based on the translated concatenated text to obtain the reorganized question and answer text;

[0029] If the target call duration does not meet the preset call duration condition, the dual-track audio corresponding to the calling audio and the called audio is obtained, and the dual-track audio is translated based on the second speech recognition service in the speech recognition service cluster to obtain the corresponding dual-track audio translation text, and the dual-track audio translation text is reorganized to obtain a question and answer text.

[0030] Optionally, after filling in the traffic summary through the second preset agent in the target language large model and the question and answer text, the method further includes:

[0031] Obtaining evaluation data of the admission service personnel on the target answer, and reviewing the evaluation data;

[0032] If the evaluation data is a thumbs-up, obtaining a user question corresponding to the target answer based on the first preset agent, and returning the user question to the vector knowledge base;

[0033] If the evaluation data is a thumbs-up, the target answer is corrected in knowledge, and the corrected answer is synchronized to the vector knowledge base.

[0034] In a second aspect, the present application provides a service assistance device based on speech recognition, comprising:

[0035] An audio recording module, used to record the audio of the taxpayer and the taxpayer service personnel during the call, respectively, to obtain the calling audio and the called audio, and cache the calling audio and the called audio to a preset database service cluster;

[0036] An audio translation module, used to obtain the audio segment data corresponding to the calling audio and the called audio from the preset database service cluster through a preset audio sliding window method, and translate the spliced ​​audio corresponding to each of the audio segment data through the first speech recognition service in the speech recognition service cluster to obtain a translated spliced ​​text; the translated spliced ​​text includes the translated spliced ​​text of the calling party and the translated spliced ​​text of the called party;

[0037] A text length judgment module is used to judge whether the text length corresponding to the caller's translated concatenated text exceeds a preset trigger threshold, and based on the judgment result and the first preset agent in the target large language model, trigger the operation of performing text error correction and new question extraction on the caller's translated concatenated text to obtain a target question;

[0038] A knowledge indexing module, configured to perform knowledge indexing on the target question based on a vector knowledge base to obtain a target answer, and send the target answer to a target customer service person so as to conduct a customer service question-and-answer call through the target customer service person;

[0039] The call summary generation module is used to reorganize the translated concatenated text that meets the preset call duration condition based on the second speech recognition service in the speech recognition service cluster after the end of the navigation service question and answer call to obtain a question and answer text, and fill in the call summary through the second preset intelligent agent in the target language large model and the question and answer text.

[0040] In a third aspect, the present application provides an electronic device, including:

[0041] Memory, used to store computer programs;

[0042] The processor is used to execute the computer program to implement the aforementioned voice recognition-based customer service assistance method.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned voice recognition-based customer service assistance method.

[0044] The present application records the audio of the taxpayer and the taxpayer service personnel during the call respectively to obtain the calling audio and the called audio, and caches the calling audio and the called audio to a preset database service cluster; obtains the audio segment data corresponding to the calling audio and the called audio from the preset database service cluster through a preset audio sliding window method, and translates the spliced ​​audio corresponding to each of the audio segment data through the first voice recognition service in the voice recognition service cluster to obtain a translated spliced ​​text; the translated spliced ​​text includes the translated spliced ​​text of the calling party and the translated spliced ​​text of the called party; determines whether the text length corresponding to the translated spliced ​​text of the calling party exceeds a preset trigger threshold, and Based on the judgment result and the first preset agent in the target large language model, the operations of text error correction and new question extraction are triggered for the caller's translated and spliced ​​text to obtain the target question; the target question is knowledge indexed based on the vector knowledge base to obtain the target answer, and the target answer is sent to the target customer service personnel to conduct a customer service question and answer call through the target customer service personnel; after the customer service question and answer call ends, the translated and spliced ​​text that meets the preset call duration condition is reorganized based on the second speech recognition service in the speech recognition service cluster to obtain a question and answer text, and the call summary is filled in through the second preset agent in the target language large model and the question and answer text.

[0045] As can be seen from the above, the present application translates the spliced ​​audio corresponding to each of the audio clip data through the first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text, and triggers the text correction and new question extraction operations based on the first preset intelligent agent in the target large language model to obtain the target question, so as to perform knowledge indexing on the target question through the vector knowledge base to obtain the target answer, which can help the customer service personnel to answer quickly and effectively improve the efficiency of telephone customer service work; after the customer service question and answer call is over, a call summary is generated based on the first speech recognition service in the speech recognition service cluster and the second preset intelligent agent in the target language large model, without manual operation, effectively increasing the number of services provided by each customer service personnel, thereby reducing the workload of the customer service personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0047] Figure 1 This is a flow chart of a method for assisting a customer service provider based on speech recognition disclosed in this application;

[0048] Figure 2 A schematic diagram of an audio sliding window processing provided by this application;

[0049] Figure 3 A schematic diagram of overlapping character processing provided by this application;

[0050] Figure 4 A schematic diagram of a cyclic problem extraction provided for this application;

[0051] Figure 5 A traffic summary reasoning diagram provided for this application;

[0052] Figure 6 A schematic diagram of data backflow and synchronization provided for this application;

[0053] Figure 7 A specific voice recognition-based customer service assistance method provided in this application;

[0054] Figure 8 This is a structural schematic diagram of a speech recognition service auxiliary device disclosed in this application;

[0055] Fig. 9 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] At present, remote customer service personnel need to constantly answer consultation calls and fill in a call summary between each call. This not only places high demands on the work ability of customer service personnel, but also increases the cost of new employee training. To this end, the present application provides a customer service assistance method based on speech recognition, which triggers text error correction and new question extraction operations based on the first preset intelligent agent in the target large language model to obtain the target question, and indexes the target question through the vector knowledge base to obtain the target answer, which can help customer service personnel to answer quickly and effectively improve the efficiency of telephone customer service work; after the customer service question and answer call is over, a call summary is generated based on the first speech recognition service in the speech recognition service cluster and the second preset intelligent agent in the target language large model, without manual operation, effectively increasing the number of services provided by each customer service personnel, thereby reducing the workload for customer service personnel.

[0058] See also Figure 1As shown, the embodiment of the present invention discloses a service assistance method based on speech recognition, including:

[0059] Step S11, respectively record the audio of the taxpayer and the taxpayer service personnel during the call to obtain the calling audio and the called audio, and cache the calling audio and the called audio to a preset database service cluster.

[0060] In this embodiment, a dual-track recording box is used to record the audio of the taxpayer and the tax service personnel during the call based on a preset time interval to obtain the calling audio and the called audio, and the calling audio and the called audio are cached in a preset database service cluster. In a specific implementation, when the dual-track recording box is used for audio recording, the recording can be performed at a time interval of 100 milliseconds (i.e., 0.1 seconds) to ensure that subtle sound changes or rapidly occurring audio events can be captured.

[0061] Step S12: obtaining the audio segment data corresponding to the calling audio and the called audio from the preset database service cluster through a preset audio sliding window method, and translating the spliced ​​audio corresponding to each of the audio segment data through the first speech recognition service in the speech recognition service cluster to obtain a translated spliced ​​text; the translated spliced ​​text includes the translated spliced ​​text of the calling party and the translated spliced ​​text of the called party.

[0062] In this embodiment, audio sliding window processing is performed on the target audio that meets the preset specified call conditions stored in the preset database service cluster based on a preset sliding window size to obtain each audio segment corresponding to the target audio, and based on the principle of non-repetitive selection, two adjacent audio segments are selected from the first audio segment and the second audio segment of each audio segment for splicing to obtain each first spliced ​​audio and each second spliced ​​audio, and then each first spliced ​​audio and each second spliced ​​audio are translated through the first speech recognition service to obtain each first translated text segment and each second translated text segment, and the correspondence between the first translated text segment and the second translated text segment with overlapping areas is determined, so as to eliminate repeated characters and text characters corresponding to the window sliding division point in the first translated text segment and the corresponding second translated text segment based on the correspondence, and splice the remaining translated text content to obtain the translated spliced ​​text.

[0063] Specifically, the method of obtaining the audio segment data corresponding to the calling audio and the called audio from the preset database service cluster by a preset audio sliding window method, and translating the spliced ​​audio corresponding to each of the audio segment data by a first speech recognition service in the speech recognition service cluster to obtain a translated spliced ​​text includes: performing audio sliding window processing on the target audio stored in the preset database service cluster based on a preset sliding window size to obtain each audio segment corresponding to the target audio from the preset database service cluster; wherein the target audio includes the target calling audio or the target called audio; starting from the first audio segment of each audio segment, and based on the principle of non-repetitive selection, selecting adjacent target number of audio segments from each of the audio segments each time for splicing to obtain a corresponding number of first spliced ​​audios; the target number is at least two; starting from the second audio segment of each audio segment, and based on the principle of non-repetitive selection, each time selecting adjacent target number of audio segments from each audio segment for splicing to obtain a corresponding number of second spliced ​​audios; translating each of the first spliced ​​audios and each of the second spliced ​​audios respectively through the first speech recognition service in the speech recognition service cluster to obtain corresponding first translated text segments and second translated text segments, and determining the correspondence between the first translated text segments and the second translated text segments having overlapping areas with each other; removing repeated characters in the first translated text segment and the corresponding second translated text segment and text characters corresponding to the window sliding segmentation point, and then splicing the remaining translated text content to obtain the translated spliced ​​text corresponding to the target audio.

[0064] Figure 2A schematic diagram of audio sliding window processing provided for the present application, in a specific implementation, after obtaining the target caller audio under the specified conversation saved in the preset database service cluster, since the normal speaking speed of a person is about 5 words per second, the first translated text segment and the second translated text segment having a corresponding relationship with overlapping areas are spliced ​​with a sliding window size of 2 seconds, and the spliced ​​audio is input into the ASR (Automatic Speech Recognition, i.e., speech recognition service cluster) model for translation to alleviate the damage to the voice signal data at each audio cutting point. Specifically, based on a sliding window size of 2 seconds, each calling audio segment corresponding to the target calling audio in the target audio is selected, starting from the first audio segment of each calling audio segment, based on the principle of non-repetitive selection, two adjacent audio segments are selected from each calling audio segment each time for splicing, that is, the two preceding and succeeding 2-second audio segments are spliced ​​to obtain a 4-second audio segment, namely, the first spliced ​​audio; starting from the second audio segment of each calling audio segment, based on the principle of non-repetitive selection, two adjacent audio segments are selected from each calling audio segment each time for splicing, that is, the two preceding and succeeding 2-second audio segments are spliced ​​to obtain a 4-second audio segment, namely, the second spliced ​​audio; the first spliced ​​audio and the second spliced ​​audio are translated to obtain Figure 1 The first translated text segment corresponding to 0-4 seconds and the second translated text segment corresponding to 2-4 seconds, then 2-4 seconds is the overlapping area of ​​the first translated text segment and the second translated text segment. It should be pointed out that the audio splicing and translation method of the target called audio is consistent with that of the target calling audio, and the size of the sliding window can be adjusted accordingly according to actual needs, and is not specifically limited here.

[0065] Furthermore, after determining the overlapping areas of each of the first translated text segments and each of the second translated text segments, it is determined whether there are repeated characters in the overlapping areas, and based on the determination result, the repeated characters and the text characters corresponding to the window sliding segmentation points are eliminated, and the remaining translated text contents are spliced ​​to obtain the translated spliced ​​text corresponding to the target audio. Specifically, the method of removing repeated characters in the first translated text segment and the corresponding second translated text segment and the text characters corresponding to the window sliding segmentation point, and then splicing the remaining translated text content to obtain the translated spliced ​​text corresponding to the target audio includes: judging whether there are repeated characters in the overlapping area of ​​the first translated text and the corresponding second translated text; if there are repeated characters in the overlapping area of ​​the first translated text and the corresponding second translated text, then removing the text after the repeated characters from the first translated text to obtain the first remaining text, and retaining the text after the repeated characters from the second translated text to obtain the second remaining text, and then splicing the first remaining text and the second remaining text to obtain the translated spliced ​​text corresponding to the target audio; if there are no repeated characters in the overlapping area of ​​the first translated text and the corresponding second translated text, then replacing the overlapping area on the first translated text with the second translated text to obtain the translated spliced ​​text corresponding to the target audio.

[0066] Figure 3 A schematic diagram of overlapping character processing provided by the present application, in a specific implementation manner, after determining the overlapping area of ​​each of the first translated text segments and each of the second translated text segments, it is determined whether there are repeated characters in the overlapping area. If there are repeated characters in the overlapping area, that is, the corresponding Figure 3 In the first case, there is a repeated character d in the overlapping area, so it is necessary to remove the text after the repeated character in the first translated text, that is, to remove , the remaining a, b, c, d are the first remaining text, and then the text after the repeated character on the second translated text is retained, that is, e, f, g after d is retained as the second remaining text, and a, b, c, d and e, f, g are spliced ​​to obtain the translated spliced ​​text; if there is no repeated character in the overlapping area, that is, the corresponding Figure 3 In the second case, the overlapping area on the first translated text is replaced by the second translated text, that is, the overlapping area on the first translated text is replaced by the second translated text. , , e on the first translated text c, d, Replacement is performed, and a, b and the second translated text are concatenated to obtain the translated concatenated text.

[0067] Step S13, determine whether the text length corresponding to the caller's translated concatenated text exceeds a preset trigger threshold, and based on the judgment result and the first preset agent in the target large language model, trigger the operation of text error correction and new question extraction on the caller's translated concatenated text to obtain the target question.

[0068] In this embodiment, after obtaining the translated concatenated text, the translated concatenated text is monitored based on a preset monitoring frequency, and it is determined whether the text length of the translated concatenated text of the caller in the translated concatenated text exceeds a preset trigger threshold. If the text length of the translated concatenated text of the caller in the translated concatenated text exceeds the preset trigger threshold, text correction and new question extraction operations are triggered on the translated concatenated text of the caller based on the preset prompt words in the first preset agent in the target large language model to obtain the target question. Specifically, the determining whether the text length corresponding to the caller's translated and concatenated text exceeds a preset trigger threshold, and based on the determination result and the first preset agent in the target large language model, triggering the operations of text correction and new question extraction on the caller's translated and concatenated text to obtain a target question, includes: determining whether the text length corresponding to the caller's translated and concatenated text exceeds the preset trigger threshold; if the text length corresponding to the caller's translated and concatenated text in the translated and concatenated text exceeds the preset trigger threshold, triggering the operations of text correction and new question extraction on the caller's translated and concatenated text based on the preset prompt words in the first preset agent to obtain the target question.

[0069] It can be understood that the corresponding user question is obtained through the preset prompt word, and the user question is deduplicated based on the edit distance similarity between the user questions to obtain the target question, so as to ensure the uniqueness of the target question. Specifically, the triggering of text error correction and new question extraction operations on the caller's translated spliced ​​text based on the preset prompt word in the first preset intelligent agent to obtain the target question includes: obtaining the user question corresponding to the caller's translated spliced ​​text based on the preset prompt word in the first preset intelligent agent, and deduplicating the user question based on the edit distance similarity between the user questions to obtain the target question.

[0070] In a specific implementation, the caller translated concatenated text corresponding to the specified dialogue in the preset database service cluster can be monitored at a frequency of 1 second, and it is determined whether the text length corresponding to the caller translated concatenated text exceeds a preset trigger threshold (the default is 20 words, that is, it is triggered once every 20 words). If the text length corresponding to the caller translated concatenated text exceeds the preset trigger threshold, the question extraction agent (that is, the first preset agent) in the target large language model is called and the caller translated concatenated text obtained each time is input into the target large language model through the preset prompt word, so as to trigger the operation of text error correction and new question extraction for the caller translated concatenated text based on the target large language model to obtain the target question. In order to ensure the uniqueness of the target question, the preset prompt word in the first preset agent is also used to obtain the user question corresponding to the caller translated concatenated text, and the user question is determined to be repeated based on the string similarity of any two user questions, so as to deduplicate the user question to obtain the target question.

[0071] Step S14: perform knowledge indexing on the target question based on the vector knowledge base to obtain a target answer, and send the target answer to a target customer service personnel so as to conduct a customer service question-and-answer call through the target customer service personnel.

[0072] In this embodiment, after obtaining the target question, the target question is indexed in real time from the vector knowledge base to similar standard questions and answers in the form of a queue, that is, the target answer corresponding to the target question, and the target answer is pushed to the target customer service personnel in real time to serve them in customer service questions and answers. Figure 4 A schematic diagram of a cyclic question extraction is provided for the present application. After obtaining the caller's translated concatenated text, the user question corresponding to the caller's translated concatenated text is extracted through the preset prompt words in the target large language model, and the user questions are deduplicated based on the editor distance similarity between the user questions to obtain the target question, and then the target question is knowledge indexed through the vector knowledge base to obtain the target answer corresponding to the target question, and the target answer is pushed to the target customer service personnel to conduct a customer service question and answer call through the target customer service personnel.

[0073] Step S15: After the navigation service question and answer call is completed, the translated concatenated text that meets the preset call duration condition is reorganized based on the second speech recognition service in the speech recognition service cluster to obtain a question and answer text, and the call summary is filled in through the second preset intelligent agent in the target language large model and the question and answer text.

[0074] In this embodiment, after the navigation service Q&A call is completed, the task of the offline processing stage will be triggered. Figure 5 A traffic summary reasoning schematic diagram provided for the present application is provided, wherein the call duration of the service question-and-answer call is calculated, and it is determined whether the call duration exceeds a preset call duration threshold. When the call duration exceeds the preset call duration threshold, that is, the call duration meets the preset call duration condition, it indicates that the marginal benefit brought by the long conversation occupying GPU (Graphics Processing Unit, i.e., graphics processor) resources is not ideal. Then, the translated concatenated text including the translated concatenated text of the caller and the translated concatenated text of the called party is directly used, and the translated concatenated text is reorganized into the caller and called party texts to obtain the reorganized question-and-answer text; if the call duration does not exceed the preset call duration threshold, that is, the call duration does not meet the preset call duration condition, then the complete dual-track audio corresponding to the caller audio and the called audio is obtained, and the dual-track audio is input into the ASR model deployed based on the GPU (i.e., the second speech recognition service) to obtain the translated dual-track audio translated text, and the question-and-answer text is obtained by text reorganizing the dual-track audio translated text. It should be noted that the preset call duration threshold can be tested to find the most appropriate threshold based on the actual call duration distribution of the service and GPU performance, and is not specifically limited here.

[0075] Specifically, the second speech recognition service in the speech recognition service cluster performs text reorganization on the translated concatenated text that meets the preset call duration condition to obtain a question and answer text, including: obtaining a target call duration based on the calling audio and the called audio, and judging whether the target call duration meets the preset call duration condition; if the target call duration meets the preset call duration condition, obtaining the translated concatenated text, and performing text reorganization based on the translated concatenated text to obtain the reorganized question and answer text; if the target call duration does not meet the preset call duration condition, obtaining dual-track audio corresponding to the calling audio and the called audio, and translating the dual-track audio based on the second speech recognition service in the speech recognition service cluster to obtain a corresponding dual-track audio translated text, and performing text reorganization on the dual-track audio translated text to obtain a question and answer text.

[0076] It is understandable that after filling in the call summary, the evaluation data of the service personnel for the target answer is regularly refluxed based on the preset frequency, and after the operation specialist or the audit specialist passes, the data is synchronized to the vector knowledge base. Since the reviewed likes data corresponds to the user questions extracted by the question extraction agent, this type of user question is a high-quality standard question and answer generalization data, which plays a very important role in improving the accuracy of knowledge indexing and iterative reinforcement of the vector knowledge base; the evaluation data of the thumbs-down will be synchronously distributed to the operation specialist and the algorithm engineer for diagnosis, correction and optimization. Specifically, after filling in the call summary through the second preset agent in the target language large model and the question and answer text, it also includes: obtaining the evaluation data of the service personnel for the target answer and reviewing the evaluation data; if the evaluation data is a thumbs-up, the user question corresponding to the target answer is obtained based on the first preset agent, and the user question is refluxed to the vector knowledge base; if the evaluation data is a thumbs-down, the target answer is corrected in knowledge, and the corrected answer is synchronized to the vector knowledge base.

[0077] Figure 6 A data reflow and synchronization diagram provided for this application, after the service personnel evaluate the target answer, the evaluation data will be regularly reflowed based on a preset frequency. If the evaluation data is a thumbs up, the corresponding user question will be reflowed to the vector knowledge base; if the rating data is a thumbs down, the corresponding target answer will be corrected, and the corrected answer will be synchronized to the vector knowledge base. It should be pointed out that through data reflow and synchronization based on the survival of the fittest, the generalized questions in the vector knowledge base will be continuously updated. Each target question can have 50 corresponding generalized questions. The survival of the fittest will be carried out according to the semantic similarity between the generalized questions and the target questions, so as to control the unlimited growth of vector data while continuously enhancing the quality of the vector knowledge base.

[0078] As can be seen from the above, the present application translates the spliced ​​audio corresponding to each of the audio clip data through the first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text, and triggers the text correction and new question extraction operations based on the first preset intelligent agent in the target large language model to obtain the target question, so as to perform knowledge indexing on the target question through the vector knowledge base to obtain the target answer, which can help the customer service personnel to answer quickly and effectively improve the efficiency of telephone customer service work; after the customer service question and answer call is over, a call summary is generated based on the first speech recognition service in the speech recognition service cluster and the second preset intelligent agent in the target language large model, without manual operation, effectively increasing the number of services provided by each customer service personnel, thereby reducing the workload of the customer service personnel.

[0079] For further information, see Figure 7 As shown, the embodiment of the present invention provides a specific service assistance method based on speech recognition, including:

[0080] During the real-time call stage, a dual-track recording box is first used to record the call audio between the taxpayer and the taxpayer service personnel to obtain the calling audio and the called audio, and the calling audio and the called audio are saved in a preset database service cluster; audio sliding window processing is performed on the audio segment data corresponding to the calling audio and the called audio, and the spliced ​​audio corresponding to each of the audio segment data is translated based on the first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text including the calling translated spliced ​​text and the called translated spliced ​​text, and then a question extraction operation is performed on the translated spliced ​​text based on the first preset agent in the target large language model, and the extracted target question is indexed to obtain the target answer corresponding to the target question from the vector knowledge base, and the target answer is sent to the target taxpayer service personnel to conduct a taxpayer service question-and-answer call through the target taxpayer service personnel.

[0081] After the Navigation Service Q&A call is over, the offline processing phase is entered, and the translated concatenated text that meets the preset call duration condition is reorganized by the second speech recognition service in the speech recognition service cluster to obtain the question and answer text, and the call summary is filled in by the second preset agent in the target language model and the question and answer text. In addition, after the call is over, the Navigation Service staff's evaluation data on the target answer is obtained, and data is refluxed based on the evaluation data, so that the evaluation data is reviewed by the operation specialist or the review specialist, and after the review is passed, the data is synchronized to the vector knowledge base.

[0082] As can be seen from the above, this embodiment translates and splices the acquired dual-track audio to obtain a translated spliced ​​text, triggers text correction and new question extraction operations based on the first preset agent in the target large language model to obtain a target question, and performs knowledge indexing on the target question to obtain a target answer. The target answer obtained through the large language model can help customer service personnel to answer quickly, effectively improving the efficiency of telephone customer service work.

[0083] Accordingly, see Figure 8 As shown, the present application also provides a service assistance device based on speech recognition, comprising:

[0084] The audio recording module 11 is used to record the audio of the taxpayer and the taxpayer service personnel during the call to obtain the calling audio and the called audio, and cache the calling audio and the called audio to a preset database service cluster;

[0085] The audio translation module 12 is used to obtain the audio segment data corresponding to the calling audio and the called audio from the preset database service cluster through a preset audio sliding window method, and translate the spliced ​​audio corresponding to each of the audio segment data through the first speech recognition service in the speech recognition service cluster to obtain a translated spliced ​​text; the translated spliced ​​text includes the translated spliced ​​text of the calling party and the translated spliced ​​text of the called party;

[0086] The text length judgment module 13 is used to judge whether the text length corresponding to the caller's translated and spliced ​​text exceeds a preset trigger threshold, and based on the judgment result and the first preset agent in the target large language model, trigger the operation of performing text error correction and new question extraction on the caller's translated and spliced ​​text to obtain the target question;

[0087] The knowledge indexing module 14 is used to perform knowledge indexing on the target question based on the vector knowledge base to obtain a target answer, and send the target answer to the target customer service personnel so as to conduct a customer service question and answer call through the target customer service personnel;

[0088] The call summary generation module 15 is used to reorganize the translated concatenated text that meets the preset call duration condition based on the second speech recognition service in the speech recognition service cluster after the end of the navigation service question and answer call to obtain a question and answer text, and fill in the call summary through the second preset intelligent agent in the target language large model and the question and answer text.

[0089] As can be seen from the above, the present application translates the spliced ​​audio corresponding to each of the audio clip data through the first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text, and triggers the text correction and new question extraction operations based on the first preset intelligent agent in the target large language model to obtain the target question, so as to perform knowledge indexing on the target question through the vector knowledge base to obtain the target answer, which can help the customer service personnel to answer quickly and effectively improve the efficiency of telephone customer service work; after the customer service question and answer call is over, a call summary is generated based on the first speech recognition service in the speech recognition service cluster and the second preset intelligent agent in the target language large model, without manual operation, effectively increasing the number of services provided by each customer service personnel, thereby reducing the workload of the customer service personnel.

[0090] In some specific implementations, the audio translation module 12 may specifically include:

[0091] An audio segment acquisition unit, configured to perform audio sliding window processing on the target audio stored in the preset database service cluster based on a preset sliding window size, so as to acquire each audio segment corresponding to the target audio from the preset database service cluster; wherein the target audio includes the target calling audio or the target called audio;

[0092] A first audio splicing unit is used to start from the first audio segment of each audio segment and select adjacent target number of audio segments from each audio segment for splicing based on the principle of non-repetitive selection, so as to obtain corresponding first spliced ​​audios; the target number is at least two;

[0093] A second audio splicing unit is used to start from the second audio segment of each audio segment and select adjacent target number of audio segments from each audio segment for splicing based on the principle of non-repetitive selection, so as to obtain corresponding second spliced ​​audios;

[0094] an audio translation unit, configured to translate each of the first spliced ​​audios and each of the second spliced ​​audios respectively through a first speech recognition service in a speech recognition service cluster to obtain corresponding first translated text segments and second translated text segments, and determine a correspondence between the first translated text segments and the second translated text segments having overlapping areas therebetween;

[0095] The remaining text splicing unit is used to remove repeated characters in the first translated text segment and the corresponding second translated text segment and the text characters corresponding to the window sliding division point, and then splice the remaining translated text content to obtain the translated spliced ​​text corresponding to the target audio.

[0096] In some specific implementations, the audio translation module 12 may specifically include:

[0097] a repeated character determination unit, used for determining whether there are repeated characters in the overlapping area of ​​the first translated text and the corresponding second translated text;

[0098] a repeated character elimination unit, configured to, if repeated characters exist in the overlapping area between the first translated text and the corresponding second translated text, eliminate the text after the repeated characters from the first translated text to obtain a first remaining text, retain the text after the repeated characters from the second translated text to obtain a second remaining text, and then concatenate the first remaining text and the second remaining text to obtain the translated concatenated text corresponding to the target audio;

[0099] An overlapping area replacement unit is used to replace the overlapping area on the first translated text with the second translated text if there are no repeated characters in the overlapping area between the first translated text and the corresponding second translated text, so as to obtain the translated concatenated text corresponding to the target audio.

[0100] In some specific implementations, the text length determination module 13 may specifically include:

[0101] A length judgment unit, used to judge whether the text length corresponding to the translated and concatenated text of the caller exceeds a preset trigger threshold;

[0102] A text error correction unit is used to trigger text error correction and new question extraction operations on the caller's translated spliced ​​text based on the preset prompt words in the first preset intelligent agent if the text length corresponding to the caller's translated spliced ​​text in the translated spliced ​​text exceeds a preset trigger threshold, so as to obtain a target question.

[0103] In some specific implementations, the text length determination module 13 may specifically include:

[0104] A user question deduplication unit is used to obtain user questions corresponding to the caller's translated concatenated text based on the preset prompt words in the first preset intelligent agent, and to deduplicate the user questions based on the edit distance similarity between the user questions to obtain target questions.

[0105] In some specific implementations, the traffic summary generating module 15 may specifically include:

[0106] A call duration determination unit, configured to obtain a target call duration based on the calling audio and the called audio, and determine whether the target call duration satisfies a preset call duration condition;

[0107] A text reorganization unit, configured to obtain the translated concatenated text if the target call duration satisfies a preset call duration condition, and to perform text reorganization based on the translated concatenated text to obtain the reorganized question and answer text;

[0108] The dual-track audio translation unit is used to obtain the dual-track audio corresponding to the calling audio and the called audio if the target call duration does not meet the preset call duration condition, and translate the dual-track audio based on the second speech recognition service in the speech recognition service cluster to obtain the corresponding dual-track audio translation text, and reorganize the dual-track audio translation text to obtain a question and answer text.

[0109] In some specific implementations, the speech recognition-based service assistance device may further include:

[0110] An evaluation data acquisition unit, used to acquire the evaluation data of the admission service personnel for the target answer and to review the evaluation data;

[0111] A user question return unit, configured to obtain a user question corresponding to the target answer based on the first preset agent if the evaluation data is a like, and return the user question to the vector knowledge base;

[0112] The answer correction unit is used to correct the knowledge of the target answer if the evaluation data is a thumbs-up, and synchronize the corrected answer to the vector knowledge base.

[0113] Furthermore, the present application also discloses an electronic device. Fig. 9 It is a structural diagram of an electronic device 20 shown according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input and output interface 25 and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the service assistance method based on speech recognition disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment can specifically be an electronic computer.

[0114] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0115] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0116] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the voice recognition-based customer service assistance method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0117] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned disclosed method for assisting customers with speech recognition is implemented. The specific steps of the method can be referred to the corresponding contents disclosed in the aforementioned embodiments, and will not be described in detail here.

[0118] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0119] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0120] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0121] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0122] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for assisting customers based on speech recognition, characterized in that: include: Record the audio of the taxpayer and the taxpayer service personnel during the call to obtain the calling audio and the called audio, and cache the calling audio and the called audio to a preset database service cluster; The audio segment data corresponding to the calling audio and the called audio are obtained from the preset database service cluster by a preset audio sliding window method, and the spliced ​​audio corresponding to each of the audio segment data is translated by a first speech recognition service in the speech recognition service cluster to obtain a translated spliced ​​text; the translated spliced ​​text includes a translated spliced ​​text of the calling party and a translated spliced ​​text of the called party; Determine whether the text length corresponding to the caller's translated concatenated text exceeds a preset trigger threshold, and based on the determination result and the first preset agent in the target large language model, trigger the operation of performing text error correction and new question extraction on the caller's translated concatenated text to obtain a target question; Performing knowledge indexing on the target question based on the vector knowledge base to obtain a target answer, and sending the target answer to a target customer service person so as to conduct a customer service question-and-answer call through the target customer service person; After the Nasi Talk Q&A call is over, the translated concatenated text that meets the preset call duration condition is reorganized based on the second speech recognition service in the speech recognition service cluster to obtain a question and answer text, and the call summary is filled in through the second preset intelligent agent in the target language model and the question and answer text.

2. The method for assisting customers based on speech recognition according to claim 1, characterized in that: The step of obtaining the audio segment data corresponding to the calling audio and the called audio from the preset database service cluster by using a preset audio sliding window method, and translating the spliced ​​audio corresponding to each of the audio segment data by using a first speech recognition service in the speech recognition service cluster to obtain a translated spliced ​​text includes: Performing audio sliding window processing on the target audio stored in the preset database service cluster based on a preset sliding window size to obtain audio segments corresponding to the target audio from the preset database service cluster; wherein the target audio includes the target calling audio or the target called audio; Starting from the first audio segment of each audio segment, and based on the principle of non-repetitive selection, each time selecting adjacent target number of audio segments from each audio segment for splicing, so as to obtain corresponding first spliced ​​audios; the target number is at least two; Starting from the second audio segment of each of the audio segments, and based on the principle of non-repetitive selection, each time selecting the target number of adjacent audio segments from each of the audio segments for splicing, so as to obtain corresponding second spliced ​​audios; Translating each of the first spliced ​​audios and each of the second spliced ​​audios respectively through the first speech recognition service in the speech recognition service cluster to obtain corresponding first translated text segments and second translated text segments, and determining the correspondence between the first translated text segments and the second translated text segments having overlapping areas therebetween; The repeated characters in the first translated text segment and the corresponding second translated text segment and the text characters corresponding to the window sliding segmentation point are removed, and then the remaining translated text content is spliced ​​to obtain the translated spliced ​​text corresponding to the target audio.

3. The method for assisting customers based on speech recognition according to claim 2, characterized in that: The step of removing repeated characters in the first translated text segment and the corresponding second translated text segment and text characters corresponding to the window sliding segmentation point, and then splicing the remaining translated text content to obtain the translated spliced ​​text corresponding to the target audio includes: Determining whether there are repeated characters in an overlapping area of ​​the first translated text and the corresponding second translated text; If there are repeated characters in the overlapping area of ​​the first translated text and the corresponding second translated text, the text after the repeated characters is removed from the first translated text to obtain a first remaining text, and the text after the repeated characters is retained from the second translated text to obtain a second remaining text, and then the first remaining text and the second remaining text are spliced ​​to obtain the translated spliced ​​text corresponding to the target audio; If there are no repeated characters in the overlapping area of ​​the first translated text and the corresponding second translated text, the overlapping area on the first translated text is replaced with the second translated text to obtain the translated concatenated text corresponding to the target audio.

4. The method for assisting customers based on speech recognition according to claim 1, characterized in that: The determining whether the text length corresponding to the caller's translated and concatenated text exceeds a preset trigger threshold, and triggering the operations of performing text error correction and new question extraction on the caller's translated and concatenated text based on the determination result and the first preset agent in the target large language model to obtain the target question, includes: Determine whether the length of the text corresponding to the translated and concatenated text of the caller exceeds a preset trigger threshold; If the text length corresponding to the caller's translated spliced ​​text in the translated spliced ​​text exceeds a preset trigger threshold, the operations of text correction and new question extraction are triggered on the caller's translated spliced ​​text based on the preset prompt words in the first preset intelligent agent to obtain the target question.

5. The method for assisting customers based on speech recognition according to claim 4, characterized in that: The triggering of text error correction and new question extraction on the caller translated and spliced ​​text based on the preset prompt words in the first preset agent to obtain the target question includes: Based on the preset prompt words in the first preset agent, the user questions corresponding to the translated concatenated text of the caller are obtained, and the user questions are deduplicated based on the edit distance similarity between the user questions to obtain the target question.

6. The method for assisting customers based on speech recognition according to any one of claims 1 to 5, characterized in that: The second speech recognition service in the speech recognition service cluster performs text reorganization on the translated concatenated text that meets the preset call duration condition to obtain a question and answer text, including: Obtaining a target call duration based on the calling audio and the called audio, and determining whether the target call duration meets a preset call duration condition; If the target call duration satisfies the preset call duration condition, the translated concatenated text is obtained, and text reorganization is performed based on the translated concatenated text to obtain the reorganized question and answer text; If the target call duration does not meet the preset call duration condition, the dual-track audio corresponding to the calling audio and the called audio is obtained, and the dual-track audio is translated based on the second speech recognition service in the speech recognition service cluster to obtain the corresponding dual-track audio translation text, and the dual-track audio translation text is reorganized to obtain a question and answer text.

7. The method for assisting customers based on speech recognition according to claim 5, characterized in that: After the traffic summary is filled in by the second preset agent in the target language large model and the question and answer text, the method further includes: Obtaining evaluation data of the admission service personnel on the target answer, and reviewing the evaluation data; If the evaluation data is a thumbs-up, obtaining a user question corresponding to the target answer based on the first preset agent, and returning the user question to the vector knowledge base; If the evaluation data is a thumbs-up, the target answer is corrected in knowledge, and the corrected answer is synchronized to the vector knowledge base.

8. A speech recognition-based service assistance device, characterized in that: include: An audio recording module, used to record the audio of the taxpayer and the taxpayer service personnel during the call, respectively, to obtain the calling audio and the called audio, and cache the calling audio and the called audio to a preset database service cluster; An audio translation module, used to obtain the audio segment data corresponding to the calling audio and the called audio from the preset database service cluster through a preset audio sliding window method, and translate the spliced ​​audio corresponding to each of the audio segment data through the first speech recognition service in the speech recognition service cluster to obtain a translated spliced ​​text; the translated spliced ​​text includes the translated spliced ​​text of the calling party and the translated spliced ​​text of the called party; A text length judgment module is used to judge whether the text length corresponding to the caller's translated concatenated text exceeds a preset trigger threshold, and based on the judgment result and the first preset agent in the target large language model, trigger the operation of performing text error correction and new question extraction on the caller's translated concatenated text to obtain a target question; A knowledge indexing module, configured to perform knowledge indexing on the target question based on a vector knowledge base to obtain a target answer, and send the target answer to a target customer service person so as to conduct a customer service question-and-answer call through the target customer service person; The call summary generation module is used to reorganize the translated concatenated text that meets the preset call duration condition based on the second speech recognition service in the speech recognition service cluster after the end of the navigation service question and answer call to obtain a question and answer text, and fill in the call summary through the second preset intelligent agent in the target language large model and the question and answer text.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor is used to execute the computer program to implement the speech recognition-based customer service assistance method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the method for assisting a customer service based on speech recognition as described in any one of claims 1 to 7 is implemented.

Images