A speech recognition-based feeding assistance method, device, equipment and medium

Abstract

The application discloses a tax payment assistance method and device based on voice recognition, equipment and medium, relates to the field of artificial intelligence technology, and comprises the following steps: acquiring audio segment data of taxpayers and tax payment personnel in a call process from a preset database service cluster through a preset audio sliding window mode, and translating spliced audio through a first voice recognition service; judging whether the text length of the calling translated spliced text exceeds a preset trigger threshold, and triggering text correction and new question extraction operations based on the judgment result and a first preset intelligent agent; performing knowledge indexing on a target question based on a vector knowledge base; after the call ends, performing text reorganization on the translated spliced text that meets a preset call duration condition based on a second voice recognition service, and filling a traffic summary through a second preset intelligent agent and a question and answer text. Based on the voice recognition service cluster and the target large language model, the target question is obtained, which can help the tax payment personnel to quickly answer and improve the tax payment work efficiency.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a voice recognition-based service assistance method, apparatus, device, and medium. Background Technology

[0002] In traditional taxpayer service scenarios, remote taxpayer service personnel need to answer inquiries constantly. This not only places high demands on their work abilities but also increases the cost of training new employees. Furthermore, during peak inquiries, taxpayer service personnel are required to fill out call summaries, making their workload excessive.

[0003] As can be seen from the above, how to improve the efficiency and effectiveness of telephone customer service while reducing the burden on customer service personnel and optimizing the personnel structure is an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a voice recognition-based telephone customer service assistance method, device, equipment, and medium, which can improve the efficiency and effectiveness of telephone customer service work while reducing the workload of customer service personnel and optimizing the personnel structure. The specific solution is as follows:

[0005] In a first aspect, this application provides a speech recognition-based service assistance method, including:

[0006] The audio of the taxpayer and the service personnel during the call is recorded separately to obtain the calling audio and the called audio, and the calling audio and the called audio are cached in a preset database service cluster;

[0007] The system obtains audio segment data corresponding to the calling audio and the called audio from the preset database service cluster using a preset audio sliding window method, and translates the spliced ​​audio corresponding to each audio segment data through the first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text; the translated spliced ​​text includes the translated spliced ​​text of the calling audio and the translated spliced ​​text of the called audio.

[0008] Determine whether the length of the text corresponding to the transposed and concatenated text of the caller exceeds a preset trigger threshold, and trigger the operation of text correction and new question extraction on the transposed and concatenated text of the caller based on the judgment result and the first preset agent in the target large language model, so as to obtain the target question;

[0009] The target question is indexed based on a vector knowledge base to obtain the target answer, and the target answer is sent to the target service personnel to conduct a service question and answer call through the target service personnel;

[0010] After the call ends, the translated and concatenated text that meets the preset call duration condition is reconstructed based on the second speech recognition service in the speech recognition service cluster to obtain the question and answer text. The call summary is then filled in by the second preset agent in the target language big model and the question and answer text.

[0011] Optionally, the step of obtaining audio segment data corresponding to the calling and called audio respectively from the preset database service cluster through a preset audio sliding window method, and translating the spliced ​​audio corresponding to each audio segment data through the first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text, includes:

[0012] Based on a preset sliding window size, audio sliding window processing is performed on the target audio stored in the preset database service cluster to obtain each audio segment corresponding to the target audio from the preset database service cluster; wherein, the target audio includes the target calling audio or the target called audio;

[0013] Starting from the first audio segment of each audio segment, and based on the principle of non-repeated selection, a number of adjacent target audio segments are selected from each audio segment for splicing to obtain a number of corresponding first spliced ​​audio segments; the number of targets is at least two.

[0014] Starting from the second audio segment of each audio segment, and based on the principle of non-repeated selection, each time a number of adjacent target audio segments are selected from each audio segment and spliced ​​together to obtain a number of corresponding second spliced ​​audio segments;

[0015] The first speech recognition service in the speech recognition service cluster translates each of the first spliced ​​audio and each of the second spliced ​​audio respectively to obtain each of the first translated text segments and each of the second translated text segments, and determines the correspondence between the first translated text segments and the second translated text segments that have overlapping areas.

[0016] Repeated characters and text characters corresponding to window sliding division points are removed from the first translated text segment and the corresponding second translated text segment. Then, the remaining translated text content is concatenated to obtain the translated concatenated text corresponding to the target audio.

[0017] Optionally, the step of removing duplicate characters and text characters corresponding to window sliding split points from the first translated text segment and the corresponding second translated text segment, and then concatenating the remaining translated text content to obtain the translated concatenated text corresponding to the target audio, includes:

[0018] Determine whether there are duplicate characters in the overlapping area of ​​the first translated text and the corresponding second translated text;

[0019] If there are repeated characters in the overlapping area of ​​the first translated text and the corresponding second translated text, then the text following the repeated characters is removed from the first translated text to obtain the first remaining text, and the text following the repeated characters is retained from the second translated text to obtain the second remaining text. Then the first remaining text and the second remaining text are concatenated to obtain the translated concatenated text corresponding to the target audio.

[0020] If there are no duplicate characters in the overlapping area of ​​the first translated text and the corresponding second translated text, then the overlapping area on the first translated text is replaced by the second translated text to obtain the translated spliced ​​text corresponding to the target audio.

[0021] Optionally, the step of determining whether the length of the text corresponding to the transposed and concatenated text of the caller exceeds a preset trigger threshold, and triggering text correction and new question extraction operations on the transposed and concatenated text of the caller based on the determination result and the first preset agent in the target large language model, to obtain the target question, includes:

[0022] Determine whether the length of the text corresponding to the translated and concatenated text exceeds a preset trigger threshold;

[0023] If the length of the text corresponding to the caller's translated and concatenated text exceeds a preset trigger threshold, then based on the preset prompt words in the first preset agent, the operation of text correction and new question extraction is triggered on the caller's translated and concatenated text to obtain the target question.

[0024] Optionally, the operation of triggering text correction and new question extraction based on preset prompt words in the first preset intelligent agent to obtain the target question includes:

[0025] Based on the preset prompt words in the first preset intelligent agent, obtain the user question corresponding to the translated and concatenated text of the caller, and perform a deduplication operation on the user questions based on the edit distance similarity between the user questions to obtain the target question.

[0026] Optionally, the text recombination of the translated and concatenated text that meets the preset call duration condition based on the second speech recognition service in the speech recognition service cluster to obtain question-and-answer text includes:

[0027] The target call duration is obtained based on the caller's audio and the called party's audio, and it is determined whether the target call duration meets the preset call duration condition.

[0028] If the target call duration meets the preset call duration condition, the translated and concatenated text is obtained, and the text is reorganized based on the translated and concatenated text to obtain the reorganized question and answer text.

[0029] If the target call duration does not meet the preset call duration condition, then the dual-track audio corresponding to the calling audio and the called audio is obtained, and the dual-track audio is translated based on the second speech recognition service in the speech recognition service cluster to obtain the corresponding dual-track audio translated text. The dual-track audio translated text is then reconstructed to obtain the question-and-answer text.

[0030] Optionally, after filling in the call summary using the second preset agent in the target language big model and the question-and-answer text, the process further includes:

[0031] Obtain the evaluation data of the service personnel for the target answer, and review the evaluation data;

[0032] If the evaluation data is a "like", then based on the first preset intelligent agent, the user question corresponding to the target answer is obtained, and the user question is fed back to the vector knowledge base;

[0033] If the evaluation data is negative, then the target answer is corrected using knowledge, and the corrected answer is synchronized to the vector knowledge base.

[0034] Secondly, this application provides a voice recognition-based service assistance device, comprising:

[0035] The audio recording module is used to record the audio of the taxpayer and the service personnel during the call to obtain the calling audio and the called audio, and to cache the calling audio and the called audio to a preset database service cluster.

[0036] The audio translation module is used to obtain audio segment data corresponding to the calling audio and the called audio from the preset database service cluster through a preset audio sliding window method, and to translate the spliced ​​audio corresponding to each audio segment data through the first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text; the translated spliced ​​text includes the translated spliced ​​text of the calling audio and the translated spliced ​​text of the called audio.

[0037] The text length judgment module is used to determine whether the text length corresponding to the caller's translated and concatenated text exceeds a preset trigger threshold, and based on the judgment result and the first preset agent in the target large language model, triggers the operation of text correction and new question extraction on the caller's translated and concatenated text to obtain the target question;

[0038] The knowledge indexing module is used to index the target question based on a vector knowledge base to obtain the target answer, and send the target answer to the target service personnel so that the target service personnel can conduct a service question and answer call.

[0039] The call summary generation module is used to reconstruct the translated and concatenated text that meets the preset call duration condition based on the second speech recognition service in the speech recognition service cluster after the call ends, so as to obtain the question and answer text, and fill in the call summary through the second preset agent in the target language big model and the question and answer text.

[0040] Thirdly, this application provides an electronic device, comprising:

[0041] Memory, used to store computer programs;

[0042] A processor is used to execute the computer program to implement the aforementioned speech recognition-based service assistance method.

[0043] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned speech recognition-based service assistance method.

[0044] This application records the audio of the taxpayer and the service provider during a call to obtain the calling and called audio, and caches the calling and called audio in a preset database service cluster. It then retrieves the audio segment data corresponding to each of the calling and called audio segments from the preset database service cluster using a preset audio sliding window method, and translates the concatenated audio corresponding to each audio segment data using a first speech recognition service in a speech recognition service cluster to obtain translated concatenated text. The translated concatenated text includes the translated concatenated text for the calling party and the translated concatenated text for the called party. It then determines whether the text length corresponding to the translated concatenated text for the calling party exceeds a preset trigger threshold. Based on the judgment result and the first preset agent in the target large language model, the operation of text correction and new question extraction is triggered on the transcribed and concatenated text of the caller to obtain the target question; the target question is indexed based on the vector knowledge base to obtain the target answer, and the target answer is sent to the target service personnel to conduct a service question and answer call through the target service personnel; after the service question and answer call ends, the transcribed and concatenated text that meets the preset call duration condition is reconstructed based on the second speech recognition service in the speech recognition service cluster to obtain the question and answer text, and the call summary is filled in by the second preset agent in the target large language model and the question and answer text.

[0045] As can be seen from the above, this application uses the first speech recognition service in the speech recognition service cluster to translate the spliced ​​audio corresponding to each audio segment data to obtain the translated spliced ​​text. Based on the first preset intelligent agent in the target large language model, it triggers text correction and new question extraction operations to obtain the target question. The target question is then indexed using a vector knowledge base to obtain the target answer. This helps service personnel answer quickly and effectively improves the efficiency of telephone service work. After the service question-and-answer call ends, a call summary is generated based on the first speech recognition service in the speech recognition service cluster and the second preset intelligent agent in the target language large model. No manual operation is required, which effectively increases the service volume of each service personnel and reduces their workload. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0047] Figure 1 This is a flowchart of a speech recognition-based service assistance method disclosed in this application;

[0048] Figure 2 This application provides a schematic diagram of an audio sliding window processing method.

[0049] Figure 3 A schematic diagram of overlapping character processing provided in this application;

[0050] Figure 4 This application provides a schematic diagram of a cyclic problem extraction method.

[0051] Figure 5 This application provides a schematic diagram for call summary reasoning.

[0052] Figure 6 This application provides a schematic diagram of data backflow and synchronization.

[0053] Figure 7 This application provides a specific speech recognition-based service assistance method.

[0054] Figure 8 This is a schematic diagram of the structure of a voice recognition auxiliary device disclosed in this application;

[0055] Figure 9 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Currently, remote service personnel need to continuously answer inquiries and fill out call summaries between calls. This not only places high demands on their skills but also increases the cost of training new employees. To address this, this application provides a speech recognition-based service assistance method. The method uses a first preset agent in a target language model to trigger text correction and new question extraction to obtain the target question. The target question is then indexed using a vector knowledge base to obtain the target answer, helping service personnel answer quickly and effectively improving the efficiency of telephone service. After the service call ends, a call summary is generated based on the first speech recognition service in the speech recognition service cluster and the second preset agent in the target language model. This eliminates the need for manual operation, effectively increasing the number of services each service personnel can provide and reducing their workload.

[0058] See Figure 1As shown, this embodiment of the invention discloses a speech recognition-based service assistance method, including:

[0059] Step S11: Record the audio of the taxpayer and the service personnel during the call to obtain the calling audio and the called audio, and cache the calling audio and the called audio to the preset database service cluster.

[0060] In this embodiment, a dual-track recording box is used to record the audio of the taxpayer and the service provider during the call, respectively, based on a preset time interval, to obtain the calling audio and the called audio. The calling audio and the called audio are then cached in a preset database service cluster. In one specific implementation, when using the dual-track recording box for audio recording, a time interval of 100 milliseconds (i.e., 0.1 seconds) can be used to ensure that subtle changes in sound or rapidly occurring audio events can be captured.

[0061] Step S12: Obtain audio segment data corresponding to the calling audio and the called audio from the preset database service cluster through a preset audio sliding window method, and translate the spliced ​​audio corresponding to each audio segment data through the first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text; the translated spliced ​​text includes the translated spliced ​​text of the calling audio and the translated spliced ​​text of the called audio.

[0062] In this embodiment, target audio that meets preset call conditions and is stored in the preset database service cluster is subjected to audio sliding window processing based on a preset sliding window size to obtain audio segments corresponding to the target audio. Then, based on the principle of non-repeated selection, two adjacent audio segments are selected from the first and second audio segments of each audio segment and spliced ​​together to obtain first spliced ​​audio segments and second spliced ​​audio segments. The first speech recognition service then translates each first spliced ​​audio segment and each second spliced ​​audio segment to obtain first translated text segments and each second translated text segment. The correspondence between the first translated text segments and the second translated text segments with overlapping areas is determined. Based on this correspondence, duplicate characters and text characters corresponding to window sliding division points in the first translated text segments and the corresponding second translated text segments are removed. The remaining translated text content is then spliced ​​together to obtain the translated spliced ​​text.

[0063] Specifically, the step of obtaining audio segment data corresponding to the calling and called audio from the preset database service cluster using a preset audio sliding window method, and translating the concatenated audio corresponding to each audio segment data through a first speech recognition service in the speech recognition service cluster to obtain the translated concatenated text, includes: performing audio sliding window processing on the target audio stored in the preset database service cluster based on a preset sliding window size to obtain each audio segment corresponding to the target audio from the preset database service cluster; wherein, the target audio includes the target calling audio or the target called audio; starting from the first audio segment of each audio segment, and based on the principle of non-repeated selection, selecting a number of adjacent target audio segments from each audio segment for concatenation to obtain a corresponding number of first concatenated audios; the target number is... At least two; starting from the second audio segment of each audio segment, and based on the principle of non-repeated selection, each time select the target number of adjacent audio segments from each audio segment for splicing to obtain a corresponding number of second spliced ​​audios; the first speech recognition service in the speech recognition service cluster translates each first spliced ​​audio and each second spliced ​​audio to obtain corresponding first translated text segments and each second translated text segment, and determines the correspondence between the first translated text segments and the second translated text segments with overlapping areas; the repeated characters in the first translated text segments and the corresponding second translated text segments, as well as the text characters corresponding to the window sliding division points, are removed, and then the remaining translated text content is spliced ​​to obtain the translated spliced ​​text corresponding to the target audio.

[0064] Figure 2This application provides a schematic diagram of an audio sliding window processing method. In one specific embodiment, after obtaining the target caller's audio of a specified dialogue stored in the preset database service cluster, since the normal speaking speed of a person is about 5 words per second, the first translated text segment and the second translated text segment with overlapping regions are spliced ​​together with a sliding window size of 2 seconds. The spliced ​​audio is then input into the ASR (Automatic Speech Recognition, i.e., speech recognition service cluster) model for translation, so as to alleviate the damage to the speech signal data at each audio cutting point. Specifically, based on a 2-second sliding window, each calling audio segment corresponding to the target calling audio in the target audio is selected. Starting from the first audio segment of each calling audio segment, two adjacent audio segments are selected and spliced ​​together each time, based on the principle of non-repetition selection. That is, two consecutive 2-second audio segments are spliced ​​together to obtain a 4-second audio segment, which is the first spliced ​​audio. Starting from the second audio segment of each calling audio segment, two adjacent audio segments are selected and spliced ​​together each time, based on the principle of non-repetition selection. That is, two consecutive 2-second audio segments are spliced ​​together to obtain a 4-second audio segment, which is the second spliced ​​audio. The first spliced ​​audio and the second spliced ​​audio are then translated to obtain... Figure 1 The first translated text segment corresponds to 0-4 seconds, and the second translated text segment corresponds to 2-4 seconds. Therefore, 2-4 seconds is the overlapping area of ​​the first and second translated text segments. It should be noted that the audio splicing and translation methods for the target called audio and the target calling audio are consistent, and the size of the sliding window can be adjusted according to actual needs, without specific limitations here.

[0065] Furthermore, after determining the overlapping areas of each of the first and second translated text segments, it is determined whether there are duplicate characters in the overlapping areas. Based on the determination result, the duplicate characters and the text characters corresponding to the window sliding division points are removed, and the remaining translated text content is spliced ​​together to obtain the translated spliced ​​text corresponding to the target audio. Specifically, the step of removing duplicate characters and text characters corresponding to window sliding division points from the first translated text segment and the corresponding second translated text segment, and then concatenating the remaining translated text content to obtain the translated concatenated text corresponding to the target audio, includes: determining whether there are duplicate characters in the overlapping area of ​​the first translated text and the corresponding second translated text; if there are duplicate characters in the overlapping area of ​​the first translated text and the corresponding second translated text, then removing the text following the duplicate characters from the first translated text to obtain a first remaining text, and retaining the text following the duplicate characters from the second translated text to obtain a second remaining text, and then concatenating the first remaining text and the second remaining text to obtain the translated concatenated text corresponding to the target audio; if there are no duplicate characters in the overlapping area of ​​the first translated text and the corresponding second translated text, then replacing the overlapping area on the first translated text with the second translated text to obtain the translated concatenated text corresponding to the target audio.

[0066] Figure 3 This application provides a schematic diagram of overlapping character processing. In one specific embodiment, after determining the overlapping areas of each of the first translated text segments and each of the second translated text segments, it is determined whether there are duplicate characters in the overlapping areas. If there are duplicate characters in the overlapping areas, that is, the corresponding... Figure 3 In the first scenario, the overlapping region contains a repeated character 'd'. Therefore, it is necessary to remove the text following the repeated character in the first translated text, i.e., remove... The remaining a, b, c, and d constitute the first remaining text. Then, the text following the repeated characters in the second translated text is retained, i.e., e, f, and g after d are retained as the second remaining text. a, b, c, d and e, f, and g are concatenated to obtain the translated concatenated text. If the overlapping area does not contain repeated characters, i.e., the corresponding... Figure 3 In the second scenario, the overlapping areas on the first translated text are replaced using the second translated text, that is, the overlapping areas in the second translated text are replaced using the second translated text. , e corresponds to c and d in the first translated text. Replace a and b, and then concatenate them with the second translated text to obtain the translated and concatenated text.

[0067] Step S13: Determine whether the length of the text corresponding to the translated and concatenated text of the caller exceeds a preset trigger threshold, and trigger the text correction and new question extraction operations on the translated and concatenated text of the caller based on the judgment result and the first preset agent in the target large language model, so as to obtain the target question.

[0068] In this embodiment, after obtaining the translated and concatenated text, the translated and concatenated text is monitored at a preset monitoring frequency, and it is determined whether the text length of the caller's translated and concatenated text in the translated and concatenated text exceeds a preset trigger threshold. If the text length of the caller's translated and concatenated text in the translated and concatenated text exceeds the preset trigger threshold, then text correction and new question extraction operations are triggered on the caller's translated and concatenated text based on the preset prompt words in the first preset agent in the target large language model, so as to obtain the target question. Specifically, the step of determining whether the length of the text corresponding to the caller-translated concatenated text exceeds a preset trigger threshold, and triggering text correction and new question extraction operations on the caller-translated concatenated text based on the determination result and the first preset agent in the target large language model to obtain the target question, includes: determining whether the length of the text corresponding to the caller-translated concatenated text exceeds the preset trigger threshold; if the length of the text corresponding to the caller-translated concatenated text in the translated concatenated text exceeds the preset trigger threshold, then triggering text correction and new question extraction operations on the caller-translated concatenated text based on the preset prompt words in the first preset agent to obtain the target question.

[0069] It is understood that the target question is obtained by acquiring the corresponding user question through the preset prompt words and deduplicating the user questions based on the edit distance similarity between the user questions, thus ensuring the uniqueness of the target question. Specifically, the operation of triggering text correction and new question extraction on the transcribed and concatenated text of the caller ID based on the preset prompt words in the first preset agent to obtain the target question includes: acquiring the user question corresponding to the transcribed and concatenated text of the caller ID based on the preset prompt words in the first preset agent, and deduplicating the user questions based on the edit distance similarity between the user questions to obtain the target question.

[0070] In one specific implementation, the transliterated text of the caller corresponding to a specified dialogue in the preset database service cluster can be monitored at a frequency of 1 second. It is determined whether the length of the transliterated text exceeds a preset trigger threshold (default 20 characters, triggered once every 20 characters). If the length exceeds the threshold, the question extraction agent (i.e., the first preset agent) in the target large language model is invoked. The transliterated text is then input into the target large language model using preset prompts. Based on the target large language model, text correction and new question extraction are performed on the transliterated text to obtain the target question. To ensure the uniqueness of the target question, the preset prompts in the first preset agent are used to obtain user questions corresponding to the transliterated text. The string similarity between any two user questions is used to determine if they are duplicated, thus deduplicating the user questions to obtain the target question.

[0071] Step S14: Index the target question based on the vector knowledge base to obtain the target answer, and send the target answer to the target service personnel to conduct a service question and answer call through the target service personnel.

[0072] In this embodiment, after obtaining the target question, the target question is indexed from the vector knowledge base in real time in the form of a queue to similar standard questions and answers, that is, the target answer corresponding to the target question, and the target answer is pushed to the target service personnel in real time to serve them in service question answering. Figure 4 This application provides a schematic diagram of a cyclic question extraction method. After obtaining the caller ID-translated and concatenated text, user questions corresponding to the caller ID-translated and concatenated text are extracted using preset prompt words in the target large language model. The user questions are then deduplicated based on logical distance similarity to obtain the target question. The target question is then indexed using the vector knowledge base to obtain the target answer corresponding to the target question. The target answer is then pushed to the target service personnel for service question-and-answer communication.

[0073] Step S15: After the call ends, the translated and concatenated text that meets the preset call duration condition is reconstructed based on the second speech recognition service in the speech recognition service cluster to obtain the question and answer text. The call summary is then filled in using the second preset agent in the target language big model and the question and answer text.

[0074] In this embodiment, after the service Q&A call ends, an offline processing phase task will be triggered. Figure 5 This application provides a call summary reasoning diagram. The call duration of the call is calculated, and it is determined whether the call duration exceeds a preset call duration threshold. If the call duration exceeds the preset threshold, meaning the call duration meets the preset call duration condition, it indicates that the marginal benefit of the long dialogue consuming GPU (Graphics Processing Unit) resources is not ideal. Therefore, the translated and concatenated text, including the caller's translated and concatenated text and the called party's translated and concatenated text, is directly used. The caller and called party texts are then reconstructed to obtain the reconstructed question-and-answer text. If the call duration does not exceed the preset call duration threshold, meaning the call duration does not meet the preset call duration condition, the complete dual-track audio corresponding to the caller's and called party's audios is obtained. This dual-track audio is then input into a GPU-based ASR model (i.e., the second speech recognition service) to obtain the translated dual-track audio translated text. The question-and-answer text is then obtained by reconstructing the dual-track audio translated text. It should be noted that the preset call duration threshold can be tested to determine the most suitable threshold based on the actual call duration distribution of the business and GPU performance, and no specific limitation is made here.

[0075] Specifically, the step of reconstructing the translated and concatenated text that meets the preset call duration condition based on the second speech recognition service in the speech recognition service cluster to obtain question-and-answer text includes: obtaining the target call duration based on the calling audio and the called audio, and determining whether the target call duration meets the preset call duration condition; if the target call duration meets the preset call duration condition, obtaining the translated and concatenated text, and reconstructing the text based on the translated and concatenated text to obtain the reconstructed question-and-answer text; if the target call duration does not meet the preset call duration condition, obtaining the dual-track audio corresponding to the calling audio and the called audio, translating the dual-track audio based on the second speech recognition service in the speech recognition service cluster to obtain the corresponding dual-track audio translated text, and reconstructing the dual-track audio translated text to obtain the question-and-answer text.

[0076] It is understandable that after the call summary is filled out, the evaluation data of the service personnel for the target answer is periodically fed back based on a preset frequency, and the data is synchronized to the vector knowledge base after being approved by the operations specialist or review specialist. Since the approved "like" data corresponds to user questions extracted by the question extraction agent, this type of user question is high-quality standard question-and-answer generalization data, which plays a very important role in improving the accuracy of knowledge indexing and the iterative strengthening of the vector knowledge base; for the "dislike" evaluation data, it will be synchronously distributed to the operations specialist and algorithm engineer for diagnosis, correction and optimization. Specifically, after filling out the call summary through the second preset agent in the target language big model and the question-and-answer text, the process also includes: obtaining the evaluation data of the service personnel for the target answer and reviewing the evaluation data; if the evaluation data is a "like", then based on the first preset agent, obtaining the user question corresponding to the target answer and feeding the user question back to the vector knowledge base; if the evaluation data is a "dislike", then correcting the target answer and synchronizing the corrected answer to the vector knowledge base.

[0077] Figure 6 This application provides a data feedback and synchronization diagram. After the service personnel evaluate the target answer, the evaluation data is periodically fed back at a preset frequency. If the evaluation data is a "like," the corresponding user question is fed back to the vector knowledge base; if the rating data is a "dislike," the corresponding target answer is corrected, and the corrected answer is synchronized to the vector knowledge base. It should be noted that through data feedback and synchronization based on a merit-based system, the generalization questions in the vector knowledge base are continuously updated. Each target question can have 50 corresponding generalization questions. The generalization questions are selected based on their semantic similarity to the target question, thus controlling the unlimited growth of vector data while continuously improving the quality of the vector knowledge base.

[0078] As can be seen from the above, this application uses the first speech recognition service in the speech recognition service cluster to translate the spliced ​​audio corresponding to each audio segment data to obtain the translated spliced ​​text. Based on the first preset intelligent agent in the target large language model, it triggers text correction and new question extraction operations to obtain the target question. The target question is then indexed using a vector knowledge base to obtain the target answer. This helps service personnel answer quickly and effectively improves the efficiency of telephone service work. After the service question-and-answer call ends, a call summary is generated based on the first speech recognition service in the speech recognition service cluster and the second preset intelligent agent in the target language large model. No manual operation is required, which effectively increases the service volume of each service personnel and reduces their workload.

[0079] Further, see Figure 7 As shown, this embodiment of the invention provides a specific speech recognition-based service assistance method, including:

[0080] During the real-time call phase, the call audio between the taxpayer and the taxpayer service personnel is first recorded using a dual-track recording box to obtain the calling and called audio, and the calling and called audio are saved to a preset database service cluster. Audio sliding window processing is then performed on the audio segment data corresponding to each of the obtained calling and called audio segments. Based on the first speech recognition service in the speech recognition service cluster, the spliced ​​audio corresponding to each audio segment data is translated to obtain the translated spliced ​​text, which includes the translated spliced ​​text of the calling and called segments. Then, based on the first preset intelligent agent in the target large language model, a question extraction operation is performed on the translated spliced ​​text, and the extracted target questions are indexed to obtain the target answer corresponding to the target question from the vector knowledge base. The target answer is then sent to the target taxpayer service personnel to conduct a taxpayer service question-and-answer call.

[0081] After the call for data acquisition and response concludes, an offline processing phase begins. The second speech recognition service within the speech recognition service cluster reconstructs the translated and concatenated text, which meets the preset call duration condition, to obtain the question-and-answer text. A call summary is then generated using the second preset agent in the target language model and the question-and-answer text. Furthermore, after the call ends, the evaluation data of the data acquisition personnel regarding the target answer is obtained. Based on this evaluation data, data feedback is performed for review by operations or auditing specialists. Upon approval, the data is synchronized to the vector knowledge base.

[0082] As can be seen from the above, this embodiment obtains translated and spliced ​​text by translating the acquired dual-track audio. Based on the first preset intelligent agent in the target large language model, it triggers text correction and new question extraction operations to obtain the target question. The target question is then indexed by knowledge to obtain the target answer. The target answer obtained through the large language model can help service personnel answer quickly, effectively improving the efficiency of telephone service work.

[0083] Accordingly, see Figure 8 As shown, this application also provides a voice recognition-based service assistance device, comprising:

[0084] The audio recording module 11 is used to record the audio of the taxpayer and the service personnel during the call to obtain the calling audio and the called audio, and to cache the calling audio and the called audio to a preset database service cluster.

[0085] The audio translation module 12 is used to obtain audio segment data corresponding to the calling audio and the called audio from the preset database service cluster through a preset audio sliding window method, and to translate the spliced ​​audio corresponding to each audio segment data through the first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text; the translated spliced ​​text includes the translated spliced ​​text of the calling party and the translated spliced ​​text of the called party.

[0086] The text length judgment module 13 is used to judge whether the text length corresponding to the caller's translated and concatenated text exceeds the preset trigger threshold, and based on the judgment result and the first preset agent in the target large language model, trigger the operation of text correction and new question extraction on the caller's translated and concatenated text to obtain the target question;

[0087] The knowledge indexing module 14 is used to index the target question based on the vector knowledge base to obtain the target answer, and send the target answer to the target service personnel so that the target service personnel can conduct a service question and answer call.

[0088] The call summary generation module 15 is used to reconstruct the translated and spliced ​​text that meets the preset call duration condition based on the second speech recognition service in the speech recognition service cluster after the call ends, so as to obtain the question and answer text, and fill in the call summary through the second preset agent in the target language big model and the question and answer text.

[0089] As can be seen from the above, this application uses the first speech recognition service in the speech recognition service cluster to translate the spliced ​​audio corresponding to each audio segment data to obtain the translated spliced ​​text. Based on the first preset intelligent agent in the target large language model, it triggers text correction and new question extraction operations to obtain the target question. The target question is then indexed using a vector knowledge base to obtain the target answer. This helps service personnel answer quickly and effectively improves the efficiency of telephone service work. After the service question-and-answer call ends, a call summary is generated based on the first speech recognition service in the speech recognition service cluster and the second preset intelligent agent in the target language large model. No manual operation is required, which effectively increases the service volume of each service personnel and reduces their workload.

[0090] In some specific embodiments, the audio translation module 12 may specifically include:

[0091] An audio segment acquisition unit is used to perform audio sliding window processing on target audio stored in the preset database service cluster based on a preset sliding window size, so as to acquire each audio segment corresponding to the target audio from the preset database service cluster; wherein, the target audio includes the target calling audio or the target called audio;

[0092] The first audio splicing unit is used to start from the first audio segment of each audio segment and, based on the principle of non-repeated selection, select a target number of adjacent audio segments from each audio segment for splicing to obtain a corresponding number of first spliced ​​audio segments; the target number is at least two.

[0093] The second audio splicing unit is used to select the target number of adjacent audio segments from each audio segment each time, starting from the second audio segment of each audio segment, and splicing them together based on the principle of non-repeated selection, so as to obtain a number of corresponding second spliced ​​audio segments.

[0094] The audio translation unit is used to translate each of the first spliced ​​audio and each of the second spliced ​​audio through the first speech recognition service in the speech recognition service cluster, so as to obtain each of the first translated text segments and each of the second translated text segments, and to determine the correspondence between the first translated text segments and the second translated text segments that have overlapping areas.

[0095] The remaining text splicing unit is used to remove repeated characters and text characters corresponding to window sliding division points in the first translated text segment and the corresponding second translated text segment, and then splice the remaining translated text content to obtain the translated spliced ​​text corresponding to the target audio.

[0096] In some specific embodiments, the audio translation module 12 may specifically include:

[0097] The duplicate character determination unit is used to determine whether there are duplicate characters in the overlapping area of ​​the first translated text and the corresponding second translated text;

[0098] The duplicate character removal unit is configured to, if there are duplicate characters in the overlapping area of ​​the first translated text and the corresponding second translated text, remove the text located after the duplicate character from the first translated text to obtain a first remaining text, and retain the text located after the duplicate character from the second translated text to obtain a second remaining text, and then concatenate the first remaining text and the second remaining text to obtain the translated concatenated text corresponding to the target audio;

[0099] The overlapping region replacement unit is used to replace the overlapping region of the first translated text with the second translated text if there are no duplicate characters in the overlapping region of the first translated text and the corresponding second translated text, so as to obtain the translated spliced ​​text corresponding to the target audio.

[0100] In some specific embodiments, the text length determination module 13 may specifically include:

[0101] The length determination unit is used to determine whether the length of the text corresponding to the caller's translated and concatenated text exceeds a preset trigger threshold.

[0102] The text correction unit is used to trigger text correction and new question extraction operations on the caller's translated text based on the preset prompt words in the first preset agent if the length of the text corresponding to the caller's translated text in the translated and concatenated text exceeds a preset trigger threshold, so as to obtain the target question.

[0103] In some specific embodiments, the text length determination module 13 may specifically include:

[0104] The user question deduplication unit is used to obtain the user question corresponding to the translated and concatenated text of the caller based on the preset prompt words in the first preset agent, and to perform deduplication operation on the user questions based on the edit distance similarity between the user questions to obtain the target question.

[0105] In some specific embodiments, the call summary generation module 15 may specifically include:

[0106] The call duration determination unit is used to obtain the target call duration based on the caller's audio and the called party's audio, and to determine whether the target call duration meets the preset call duration conditions.

[0107] The text recombination unit is used to obtain the translated and concatenated text if the target call duration meets the preset call duration condition, and to reconstruct the text based on the translated and concatenated text to obtain the reconstructed question and answer text.

[0108] The dual-track audio translation unit is used to acquire dual-track audio corresponding to the calling audio and the called audio if the target call duration does not meet the preset call duration condition, and to translate the dual-track audio based on the second speech recognition service in the speech recognition service cluster to obtain the corresponding dual-track audio translated text, and to reconstruct the dual-track audio translated text to obtain the question-and-answer text.

[0109] In some specific embodiments, the voice recognition-based service assistance device may further include:

[0110] An evaluation data acquisition unit is used to acquire evaluation data from the service personnel regarding the target answer and to review the evaluation data.

[0111] The user question feedback unit is used to obtain the user question corresponding to the target answer based on the first preset intelligent agent if the evaluation data is a like, and to feed the user question back to the vector knowledge base;

[0112] The answer correction unit is used to correct the target answer if the evaluation data is negative, and to synchronize the corrected answer to the vector knowledge base.

[0113] Furthermore, embodiments of this application also disclose an electronic device, Figure 9 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the speech recognition-based service assistance method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0114] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0115] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0116] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the speech recognition-based service assistance method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0117] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned speech recognition-based service-assisted method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0118] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0119] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0120] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0121] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0122] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A speech recognition-based service assistance method, characterized in that, include: The audio of the taxpayer and the service personnel during the call is recorded separately to obtain the calling audio and the called audio, and the calling audio and the called audio are cached in a preset database service cluster; The system obtains audio segment data corresponding to the calling audio and the called audio from the preset database service cluster using a preset audio sliding window method, and translates the spliced ​​audio corresponding to each audio segment data through the first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text; the translated spliced ​​text includes the translated spliced ​​text of the calling audio and the translated spliced ​​text of the called audio. Determine whether the length of the text corresponding to the transposed and concatenated text of the caller exceeds a preset trigger threshold, and trigger the operation of text correction and new question extraction on the transposed and concatenated text of the caller based on the judgment result and the first preset agent in the target large language model, so as to obtain the target question; The target question is indexed based on a vector knowledge base to obtain the target answer, and the target answer is sent to the target service personnel to conduct a service question and answer call through the target service personnel; After the call ends, the translated and concatenated text that meets the preset call duration condition is reconstructed based on the second speech recognition service in the speech recognition service cluster to obtain the question and answer text. The call summary is then filled in using the second preset agent in the target language big model and the question and answer text. The step of reconstructing the translated and concatenated text that meets a preset call duration condition using the second speech recognition service in the speech recognition service cluster to obtain question-and-answer text includes: obtaining a target call duration based on the calling audio and the called audio, and determining whether the target call duration meets the preset call duration condition; if the target call duration meets the preset call duration condition, obtaining the translated and concatenated text, and reconstructing the text based on the translated and concatenated text to obtain the reconstructed question-and-answer text; if the target call duration does not meet the preset call duration condition, obtaining dual-track audio corresponding to the calling audio and the called audio, translating the dual-track audio based on the second speech recognition service in the speech recognition service cluster to obtain the corresponding dual-track audio translated text, and reconstructing the dual-track audio translated text to obtain the question-and-answer text. The step of obtaining audio segment data corresponding to the calling and called audio from the preset database service cluster using a preset audio sliding window method, and translating the spliced ​​audio corresponding to each audio segment data through a first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text, includes: performing audio sliding window processing on the target audio stored in the preset database service cluster based on a preset sliding window size to obtain each audio segment corresponding to the target audio from the preset database service cluster; wherein, the target audio includes the target calling audio or the target called audio; starting from the first audio segment of each audio segment, and based on the principle of non-repeated selection, selecting a number of adjacent target audio segments from each audio segment each time for splicing to obtain a corresponding number of first spliced ​​audios; the target number is up to Two fewer; starting from the second audio segment of each audio segment, and based on the principle of non-repeated selection, each time select the target number of adjacent audio segments from each audio segment for splicing to obtain a number of corresponding second spliced ​​audios; the first speech recognition service in the speech recognition service cluster translates each first spliced ​​audio and each second spliced ​​audio to obtain corresponding first translated text segments and each second translated text segment, and determines the correspondence between the first translated text segments and the second translated text segments with overlapping areas; the repeated characters in the first translated text segments and the corresponding second translated text segments, as well as the text characters corresponding to the window sliding division points, are removed, and then the remaining translated text content is spliced ​​to obtain the translated spliced ​​text corresponding to the target audio.

2. The speech recognition-based service assistance method according to claim 1, characterized in that, The step of removing duplicate characters and text characters corresponding to window sliding split points from the first translated text segment and the corresponding second translated text segment, and then concatenating the remaining translated text content to obtain the translated concatenated text corresponding to the target audio, includes: Determine whether there are duplicate characters in the overlapping area of ​​the first translated text and the corresponding second translated text; If there are repeated characters in the overlapping area of ​​the first translated text and the corresponding second translated text, then the text following the repeated characters is removed from the first translated text to obtain the first remaining text, and the text following the repeated characters is retained from the second translated text to obtain the second remaining text. Then the first remaining text and the second remaining text are concatenated to obtain the translated concatenated text corresponding to the target audio. If there are no duplicate characters in the overlapping area of ​​the first translated text and the corresponding second translated text, then the overlapping area on the first translated text is replaced by the second translated text to obtain the translated spliced ​​text corresponding to the target audio.

3. The speech recognition-based service assistance method according to claim 1, characterized in that, The process of determining whether the length of the text corresponding to the transposed and concatenated text of the caller exceeds a preset trigger threshold, and triggering text correction and new question extraction operations on the transposed and concatenated text of the caller based on the determination result and the first preset agent in the target large language model, to obtain the target question, includes: Determine whether the length of the text corresponding to the translated and concatenated text exceeds a preset trigger threshold; If the length of the text corresponding to the caller's translated and concatenated text exceeds a preset trigger threshold, then based on the preset prompt words in the first preset agent, the operation of text correction and new question extraction is triggered on the caller's translated and concatenated text to obtain the target question.

4. The speech recognition-based service assistance method according to claim 3, characterized in that, The operation of triggering text correction and new question extraction based on preset prompt words in the first preset intelligent agent to obtain the target question includes: Based on the preset prompt words in the first preset intelligent agent, obtain the user question corresponding to the translated and concatenated text of the caller, and perform a deduplication operation on the user questions based on the edit distance similarity between the user questions to obtain the target question.

5. The speech recognition-based service assistance method according to claim 4, characterized in that, After filling in the call summary using the second preset agent in the target language big model and the question-and-answer text, the process also includes: Obtain the evaluation data of the service personnel for the target answer, and review the evaluation data; If the evaluation data is a "like", then based on the first preset intelligent agent, the user question corresponding to the target answer is obtained, and the user question is fed back to the vector knowledge base; If the evaluation data is negative, then the target answer is corrected using knowledge, and the corrected answer is synchronized to the vector knowledge base.

6. A voice recognition-based service assistance device, characterized in that, include: The audio recording module is used to record the audio of the taxpayer and the service personnel during the call to obtain the calling audio and the called audio, and to cache the calling audio and the called audio to a preset database service cluster. The audio translation module is used to obtain audio segment data corresponding to the calling audio and the called audio from the preset database service cluster through a preset audio sliding window method, and to translate the spliced ​​audio corresponding to each audio segment data through the first speech recognition service in the speech recognition service cluster to obtain the translated spliced ​​text; the translated spliced ​​text includes the translated spliced ​​text of the calling audio and the translated spliced ​​text of the called audio. The text length judgment module is used to determine whether the text length corresponding to the caller's translated and concatenated text exceeds a preset trigger threshold, and based on the judgment result and the first preset agent in the target large language model, triggers the operation of text correction and new question extraction on the caller's translated and concatenated text to obtain the target question; The knowledge indexing module is used to index the target question based on a vector knowledge base to obtain the target answer, and send the target answer to the target service personnel so that the target service personnel can conduct a service question and answer call. The call summary generation module is used to reconstruct the translated and concatenated text that meets the preset call duration condition based on the second speech recognition service in the speech recognition service cluster after the call ends, so as to obtain the question and answer text, and fill in the call summary through the second preset agent in the target language big model and the question and answer text. Specifically, the call summary generation module is used to obtain the target call duration based on the calling audio and the called audio, and to determine whether the target call duration meets the preset call duration conditions. If the target call duration meets the preset call duration conditions, the translated and concatenated text is obtained, and text recombination is performed based on the translated and concatenated text to obtain the recombined question-and-answer text. If the target call duration does not meet the preset call duration conditions, the dual-track audio corresponding to the calling audio and the called audio is obtained, and the dual-track audio is translated based on the second speech recognition service in the speech recognition service cluster to obtain the corresponding dual-track audio translated text. The dual-track audio translated text is then recombined to obtain the question-and-answer text. Specifically, the audio translation module is used to perform audio sliding window processing on the target audio stored in the preset database service cluster based on a preset sliding window size, so as to obtain each audio segment corresponding to the target audio from the preset database service cluster; wherein, the target audio includes the target calling audio or the target called audio; starting from the first audio segment of each audio segment, and based on the principle of non-repeated selection, a number of adjacent target audio segments are selected from each audio segment each time for splicing, so as to obtain a corresponding number of first spliced ​​audios; the number of targets is at least two; starting from the second audio segment of each audio segment, and based on the principle of non-repeated selection, a number of adjacent target audio segments are selected from each audio segment each time. The target number of audio segments are concatenated to obtain several corresponding second concatenated audio segments; each of the first concatenated audio segments and each of the second concatenated audio segments are translated by the first speech recognition service in the speech recognition service cluster to obtain corresponding first translated text segments and each of the second translated text segments, and the correspondence between the first translated text segments and the second translated text segments with overlapping areas is determined; repeated characters and text characters corresponding to window sliding division points in the first translated text segments and the corresponding second translated text segments are removed, and then the remaining translated text content is concatenated to obtain the translated concatenated text corresponding to the target audio.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the speech recognition-based service assistance method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the speech recognition-based service assistance method as described in any one of claims 1 to 5.

Images