Adaptive speech recognition switching method and device, storage medium and program product

Abstract

The embodiment of the invention discloses a self-adaptive voice recognition switching method and device, a storage medium and a program product. The method comprises the following steps: in response to an acquisition instruction of a user, carrying out voice recording on the user, and obtaining a voice sample; based on the voice sample, testing a plurality of preset voice recognition models, and obtaining a comprehensive test score of each voice recognition model; judging the voice recognition model with the highest comprehensive test score as a current model, deploying the current model into a voice communication system of the user, and judging the other voice recognition models as alternative models; monitoring the translation level of the current model in real time, and obtaining the comprehensive performance score of the current model; when it is judged that the current model is abnormal according to the comprehensive performance score, the current model is replaced with the alternative model with the optimal historical score. According to the method, the voice recognition models of different suppliers can be selected and switched through the quantitative evaluation of the comprehensive test score and the performance score, and the stability and reliability of voice communication are ensured.

Description

Technical Field

[0001] The present disclosure relates to the technical field of natural language processing, and in particular to an adaptive speech recognition switching method, device, storage medium, and program product. Background Art

[0002] With the development of Internet technology, intelligent call center systems are rapidly emerging and becoming a key component of modern customer service. In the technical implementation of intelligent call centers, a speech recognition model (ASR) is usually required to convert the user's voice input into text data, a large language model (LLM) generates response content based on the text data, and then the text is converted back into voice audio through speech synthesis (TTS) technology, and finally played to the user. In this process, the execution speed and quality of each link directly affect the overall quality of the call and user experience.

[0003] However, when connecting with ASR suppliers, the stability of their services and the translation effects in specific scenarios cannot be guaranteed. If the ASR translation speed slows down or the translation quality is poor, it will directly affect the response content of the large language model, thereby affecting the quality of language calls and user experience. Summary of the invention

[0004] In view of this, the embodiments of the present disclosure provide an adaptive speech recognition switching method, device, storage medium, and program product, which can selectively deploy speech recognition models of different suppliers into the voice call system through quantitative evaluation of comprehensive test scores and performance scores, and implement the function of abnormal switching in actual voice calls to ensure the stability and reliability of voice calls.

[0005] In a first aspect, an embodiment of the present disclosure provides an adaptive speech recognition switching method, which adopts the following technical solution:

[0006] In response to a collection instruction from a user, recording the user's voice to obtain a voice sample;

[0007] Based on the speech sample, multiple preset speech recognition models are tested to obtain a comprehensive test score of each speech recognition model;

[0008] The speech recognition model with the highest comprehensive test score is determined as the current model and deployed to the user's voice call system, and the remaining speech recognition models are determined as candidate models;

[0009] Monitor the translation level of the current model in real time and obtain the comprehensive performance score of the current model;

[0010] Determine whether the current model has an abnormality according to the comprehensive performance score;

[0011] If so, the alternative model with the best historical score is used to replace the current model.

[0012] Optionally, in response to a collection instruction from a user, recording a voice of the user and obtaining a voice sample includes:

[0013] Build a text template library;

[0014] Receive several selection requests from users;

[0015] In response to each selection request, a plurality of target templates are screened out from the text template library;

[0016] Configure a capture button for each target template;

[0017] When the user triggers the acquisition button on the front-end interface, a corresponding acquisition instruction is generated;

[0018] In response to the acquisition instruction, starting an audio capture device;

[0019] The audio capture hardware records the user's speech and generates a voice sample.

[0020] Optionally, the testing of multiple preset speech recognition models based on the speech sample to obtain a comprehensive test score of each speech recognition model includes:

[0021] Dividing the speech sample into a plurality of speech segments;

[0022] Using the multiple voice clips to test each speech recognition model multiple times, and obtaining the single-word translation time, wrong character ratio and fluency parameter of each speech recognition model in each test;

[0023] Based on the single word translation duration, obtaining a duration grade score for each speech recognition model;

[0024] Based on the percentage of wrong characters and the fluency parameter, obtaining a test quality parameter of each speech recognition model;

[0025] Based on the single-word translation time, the percentage of wrong characters and the fluency parameter, obtaining a learning ability parameter of each speech recognition model;

[0026] Based on the duration grade score, the test quality parameter and the learning ability parameter, a comprehensive test score of each speech recognition model is obtained.

[0027] Optionally, the acquiring of the learning ability parameter of each speech recognition model based on the single word translation time, the wrong character ratio and the fluency parameter includes:

[0028] Based on the single word translation time, the wrong character ratio and the fluency parameter, respectively construct a time analysis graph, a wrong character analysis graph and a semantic analysis graph for each speech recognition model;

[0029] Based on the duration analysis graph, obtaining a speed improvement parameter for each speech recognition model;

[0030] Based on the wrong character analysis graph, obtain the accuracy improvement parameter of each speech recognition model;

[0031] Based on the semantic analysis graph, obtaining semantic quality improvement parameters of each speech recognition model;

[0032] Based on the speed improvement parameter, the accuracy improvement parameter and the semantic quality improvement parameter, a learning ability parameter of each speech recognition model is obtained.

[0034] Optionally, the real-time monitoring of the translation level of the current model and obtaining the comprehensive performance score of the current model includes:

[0035] Monitor the performance of the current model in the current voice call, and obtain unit processing time, typo parameters, and logic parameters;

[0036] Based on the unit processing time, obtaining a timeliness score of the current model;

[0037] Based on the wrong character parameter and the logic parameter, obtaining a usage quality parameter of the current model;

[0038] Based on the unit processing time, the wrong character parameter and the logic parameter, obtaining a performance improvement parameter of the current model;

[0039] Based on the timeliness score, the usage quality parameter and the performance improvement parameter, the performance of the current model in the current voice call is evaluated to obtain a comprehensive performance score.

[0040] Optionally, judging whether the current model is abnormal according to the comprehensive performance score includes:

[0041] Determine whether the comprehensive performance scores of the current model obtained N times in a row are less than a first preset threshold, or whether a preset proportion of the comprehensive performance scores within a first preset time range are less than a second preset threshold;

[0042] If yes, it is determined that the current model has an abnormality;

[0043] If not, it is determined that the current model runs normally.

[0044] Optionally, the adopting of an alternative model with the best historical score to replace the current model includes:

[0045] Obtaining the sum of evaluation scores of each candidate model in M ​​consecutive performance analyses, wherein the evaluation scores include comprehensive test scores and / or comprehensive performance scores;

[0046] The candidate model with the highest total score is determined as having the best historical score and replaces the current model.

[0047] In a second aspect, the embodiment of the present disclosure further provides an adaptive speech recognition switching system, which adopts the following technical solution:

[0048] A response module, used to respond to the user's collection instruction, record the user's voice and obtain a voice sample;

[0049] A testing module, used to test a plurality of preset speech recognition models based on the speech sample, and obtain a comprehensive test score of each speech recognition model;

[0050] A deployment module, used to determine the speech recognition model with the highest comprehensive test score as the current model and deploy it to the user's voice call system, and determine the remaining speech recognition models as candidate models;

[0051] A monitoring module, used to monitor the translation level of the current model in real time and obtain the comprehensive performance score of the current model;

[0052] A judgment module, used for judging whether the current model has an abnormality according to the comprehensive performance score; if so, executing a replacement module;

[0053] A replacement module is used to replace the current model with an alternative model with the best historical score.

[0054] In a third aspect, the embodiments of the present disclosure further provide a computer device, which adopts the following technical solution:

[0055] The computer device comprises:

[0056] at least one processor; and,

[0058] a memory communicatively connected to the at least one processor; wherein,

[0059] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the above-mentioned adaptive speech recognition switching methods.

[0060] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any of the above-mentioned adaptive speech recognition switching methods.

[0061] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program / instruction, which implements the steps of any of the above methods when executed by a processor.

[0062] In the adaptive speech recognition switching method provided by the embodiment of the present disclosure, after the user issues a collection instruction, the system starts recording the user's voice and obtaining a voice sample to provide a data basis for subsequent model testing and selection. The obtained voice samples are used to test a variety of preset speech recognition models, and a comprehensive test score is calculated for each model. According to the comprehensive test score, the best performing model is selected as the current model and deployed to the user's voice call system. At the same time, other models are retained as alternative models for emergency use. This method of testing and selecting models using the user's own voice samples can better adapt to the voice characteristics of different users and provide more personalized and accurate speech recognition services. The system monitors the translation level of the current model in real time and calculates its comprehensive performance score, so that the performance degradation or abnormality of the current model can be discovered in time. If the comprehensive performance score of the current model is not as expected, indicating that there may be an abnormality, the system will automatically switch to the alternative model with the best historical score to ensure the continuity and quality of the service. The method ensures the stability and reliability of the voice call system by monitoring the performance of the current model in real time and automatically switching to the alternative model when an abnormality is found, reduces service interruptions caused by model failures, and increases the flexibility of the system and the ability to cope with emergencies. In summary, this method makes the model selection and switching process of different suppliers more objective and efficient through quantitative evaluation of comprehensive test scores and performance scores, reducing the need for manual intervention. It can also dynamically adjust the model used according to the specific usage of the user, realize the self-adaptation and continuous optimization of the model, and improve the accuracy of speech recognition and user satisfaction.

[0063] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0065] Figure 1 A flowchart of an adaptive speech recognition switching method provided by an embodiment of the present disclosure;

[0066] Figure 2 A flowchart of a method for obtaining a comprehensive test score provided in an embodiment of the present disclosure;

[0067] Figure 3 A flow chart of a method for acquiring learning ability parameters provided in an embodiment of the present disclosure;

[0068] Figure 4 A functional block diagram of an adaptive speech recognition switching system provided by an embodiment of the present disclosure;

[0069] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0070] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0071] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0072] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0073] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0074] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0075] Reference Figure 1 The present disclosure provides an adaptive speech recognition switching method, comprising the following steps:

[0076] S1: In response to a user's collection instruction, the user's voice is recorded to obtain a voice sample;

[0077] S2: Based on the speech samples, multiple preset speech recognition models are tested to obtain the comprehensive test score of each speech recognition model;

[0078] S3: The speech recognition model with the highest comprehensive test score is determined as the current model and deployed to the user's voice call system, and the remaining speech recognition models are determined as candidate models;

[0079] S4: monitor the translation level of the current model in real time and obtain the comprehensive performance score of the current model;

[0080] S5: Determine whether the current model has an abnormality based on the comprehensive performance score; if so, execute S6; if not, return to S4.

[0081] S6: Use the alternative model with the best historical score to replace the current model;

[0082] The adaptive speech recognition switching method provided by the present disclosure is that after the user issues a collection instruction, the system starts recording the user's voice and obtaining voice samples to provide a data basis for subsequent model testing and selection. The obtained voice samples are used to test multiple preset speech recognition models, and a comprehensive test score is calculated for each model. According to the comprehensive test score, the best performing model is selected as the current model and deployed to the user's voice call system. At the same time, other models are retained as alternative models in case of emergency. This method of testing and selecting models using the user's own voice samples can better adapt to the voice characteristics of different users and provide more personalized and accurate speech recognition services.

[0083] The system monitors the translation level of the current model in real time and calculates its comprehensive performance score, which can promptly detect possible performance degradation or abnormalities in the current model. If the comprehensive performance score of the current model is not as expected, indicating that there may be an abnormality, the system will automatically switch to the alternative model with the best historical score to ensure service continuity and quality. This method ensures the stability and reliability of the voice call system by monitoring the performance of the current model in real time and automatically switching to the alternative model when an abnormality is found, reducing service interruptions caused by model failures and increasing the system's flexibility and ability to respond to emergencies.

[0084] In summary, this method makes the model selection and switching process of different suppliers more objective and efficient through quantitative evaluation of comprehensive test scores and performance scores, reducing the need for manual intervention. It can also dynamically adjust the model used according to the specific usage of the user, realize the self-adaptation and continuous optimization of the model, and improve the accuracy of speech recognition and user satisfaction.

[0085] In S1, a text template library is established that integrates a variety of preset text templates. These templates are marked according to different application scenarios, such as news reports, literary works, etc., to cover diverse voice call needs. Each preset text template is displayed to the user on the front-end interface, and the user selects one or more target text templates according to their own usage needs. The user can initiate a selection request by clicking on a preset text template on the front-end interface or using a voice command. The back-end receives these selection requests and selects the corresponding target template from the text template library according to the user's selection request.

[0086] A dedicated acquisition button is designed and configured for each target template. These buttons are intuitively displayed on the front-end interface, which is convenient for users to quickly access and operate. When the user triggers any acquisition button on the front-end interface by clicking the mouse or operating the touch screen, the system will immediately recognize this action and generate a acquisition instruction corresponding to the template. The instruction contains all the necessary information, such as template identification, user ID, etc., to ensure the correct association and processing of subsequent voice data.

[0087] In response to the capture instruction, the audio capture device, such as a built-in or external microphone, is automatically started. During the startup process, the connection status and permission settings of the device are checked to ensure that the audio capture device can work properly. In addition, the system automatically adjusts the sensitivity of the microphone according to the current ambient noise level to optimize the recording quality. Once the audio capture device is ready, the user reads the text in the target template, and the system automatically saves the recorded user voice as a voice sample. When the user selects multiple target templates, the user needs to click the corresponding capture button in turn and read the text on the target template. The system will record the voice of each reading separately and save it as multiple independent voice samples.

[0088] By determining the target template that meets the user's needs and recording the corresponding voice samples, the speech recognition models of different suppliers (i.e. different types of speech recognition models (ASR)) can be tested in the subsequent stage. In addition, during the testing process, the voice samples can also be used to train the speech recognition model, so that the speech recognition model can learn and adapt to the user's personal language characteristics and habits, thereby providing translation services that better meet the user's personalized needs.

[0089] In S2, a unified interface is designed to integrate speech recognition models provided by different vendors to ensure the standardization of input parameters and storage, including but not limited to parameter types, parameter meanings, return value types, return value meanings, and audio codecs. Since soft switches commonly used in voice calls (such as Freeswitch) usually use PCMU encoding, and some ASR vendors may not support this encoding, the system also needs to be able to perform audio transcoding adaptation to ensure compatibility with various ASR products. In order to track and optimize the performance of ASR products, vendor tagging is performed during each test and actual voice call to clearly record and identify the ASR vendor used in the current test and current call, which helps to monitor the performance of each vendor and train and optimize the ASR technology accordingly based on actual usage.

[0090] Reference Figure 2The flowchart of the comprehensive test score acquisition method shown in the figure, "Based on the speech sample, test multiple preset speech recognition models to obtain the comprehensive test score of each speech recognition model" includes the following steps:

[0091] S21: Divide the speech sample into multiple speech segments;

[0092] S22: using multiple voice clips to test each speech recognition model multiple times, and obtaining the single-word translation time, wrong character ratio and fluency parameters of each speech recognition model in each test;

[0093] S23: Based on the single word translation duration, obtaining a duration grade score of each speech recognition model;

[0094] S24: Based on the percentage of wrong characters and fluency parameters, obtain the test quality parameters of each speech recognition model;

[0095] S25: Based on the single word translation time, wrong character ratio and fluency parameters, the learning ability parameters of each speech recognition model are obtained;

[0096] S26: Based on the duration grade score, the test quality parameter and the learning ability parameter, a comprehensive test score of each speech recognition model is obtained.

[0097] In S21, before conducting the actual test, the collected voice samples are preprocessed and each sample is segmented into multiple voice segments. Specifically, the natural pauses when the user speaks are used as reference points, and the voice samples are segmented in combination with other acoustic features such as voice intensity and intonation changes, so as to obtain continuous and complete voice segments.

[0098] In S22, each speech segment is used to test each speech recognition model, and the total translation time and number of translated words of each speech recognition model for each speech segment are obtained. The single-word translation time of the speech recognition model for the speech segment is equal to the ratio of the total translation time to the number of translated words. The speech segment is matched with the source target template (the target template to which the speech segment belongs), and the text content corresponding to the speech segment is determined by comparative analysis, and the translation accuracy and fluency of the speech recognition model are detected according to the text content.

[0099] Regarding translation accuracy, if the translation result is completely correct, the proportion of typos is determined to be 0. If there are errors in the translation result, the proportion of typos is equal to the ratio of the number of typos to the total number of words in the text content. Regarding translation fluency, if the speech recognition model translates the sentence of the speech segment smoothly, the fluency parameter is determined to be equal to the first preset score (for example, 1 or 10). If the translation is not smooth, the fluency parameter is determined to be equal to the second preset score (for example, 0); N-gram models, GPT models or subjective evaluation methods can also be used to detect translation fluency, and the specific value of the fluency parameter is determined according to the translation fluency, but the minimum value is the second preset score.

[0100] In S23, multiple numerical ranges are preset, and different numerical ranges have different priorities. The single word translation duration is matched with multiple numerical ranges, and the duration grade score is equal to the priority of the numerical range that matches successfully. For example, when the single word translation duration is less than 100 milliseconds, the duration grade score is equal to 10; when the single word translation duration is greater than or equal to 100 milliseconds and less than 200 milliseconds, the duration grade score is equal to 9; when the single word translation duration is greater than or equal to 200 milliseconds and less than 300 milliseconds, the duration grade score is equal to 7; when the single word translation duration is greater than or equal to 300 milliseconds and less than 400 milliseconds, the duration grade score is equal to 5; when the single word translation duration is greater than or equal to 400 milliseconds, the duration grade score is equal to 2.

[0101] In S24, if the proportion of typos is equal to 0 and the fluency parameter is not equal to the second preset score, the test quality parameter is determined to be equal to the third preset score (for example, 10); if the proportion of typos is equal to 0, but the fluency parameter is equal to the second preset score, or the proportion of typos is not equal to 0, but the fluency parameter is not equal to the second preset score, the test quality parameter is determined to be equal to the fourth preset score (for example, 9); if the proportion of typos is not equal to 0 and the fluency parameter is equal to the second preset score, the test quality parameter is determined to be equal to the fifth preset score (for example, 8).

[0102] In S25, refer to Figure 3 The flowchart of the learning ability parameter acquisition method shown in the figure, "Acquiring the learning ability parameters of each speech recognition model based on the single-word translation time, the percentage of wrong characters and the fluency parameter" includes the following steps:

[0103] S251: Based on the single word translation time, wrong character ratio and fluency parameters, a time analysis graph, a wrong character analysis graph and a semantic analysis graph are respectively constructed for each speech recognition model;

[0104] S252: Based on the duration analysis graph, obtain a speed improvement parameter for each speech recognition model;

[0105] S253: Based on the wrong character analysis graph, obtain accuracy improvement parameters of each speech recognition model;

[0106] S254: Based on the semantic analysis graph, obtain semantic quality improvement parameters of each speech recognition model;

[0107] S255: Based on the speed improvement parameter, the accuracy improvement parameter, and the semantic quality improvement parameter, a learning ability parameter of each speech recognition model is obtained.

[0108] Record the collection time of single-word translation time, wrong character ratio and fluency parameters (assume that these three parameters are collected at the same time). Construct a time analysis chart based on the collection time and single-word translation time, a wrong character analysis chart based on the collection time and wrong character ratio, and a semantic analysis chart based on the collection time and fluency parameters. The time analysis chart, wrong character analysis chart and semantic analysis chart are all two-dimensional coordinate charts, where the horizontal axis represents the collection time and the vertical axis represents the collected parameter value.

[0109] Mathematical methods are used to process each analysis graph and calculate the corresponding average slope, where the average slope of the duration analysis graph represents the speed improvement parameter, the average slope of the typo analysis graph represents the accuracy improvement parameter, and the average slope of the semantic analysis graph represents the semantic quality improvement parameter. Based on the obtained speed improvement parameters, accuracy improvement parameters, and semantic quality improvement parameters, the learning ability parameters of each speech recognition model are calculated. The calculation formula for the learning ability parameters is as follows:

[0110] C i =α1×S i +α2×Z i +α3×Y i ;(Formula 1)

[0111] In Formula 1, i is the identifier of the speech recognition model; C i is the learning ability parameter of the i-th speech recognition model; α1, α2, α3 are the preset weights; S i is the speed improvement parameter of the i-th speech recognition model; Z i Y is the accuracy improvement parameter of the i-th speech recognition model; i is the semantic quality improvement parameter of the i-th speech recognition model.

[0112] In S26, the calculation formula of the comprehensive test score is as follows:

[0113]

[0114] In Formula 2, j is the identifier of the speech segment, 1≤j≤J; J is the total number of speech segments; F1 i is the comprehensive test score of the i-th speech recognition model; f1ij f2 is the duration grade score of the i-th speech recognition model for the j-th speech segment; ij is the test quality parameter of the i-th speech recognition model for the j-th speech segment; β1, β2, and β3 are preset weights.

[0115] In S3, the speech recognition model with the highest comprehensive test score is selected and identified as the current optimal speech recognition model. The current model is deployed to the voice call system used by users to provide users with the best speech recognition service. At the same time, other speech recognition models participating in the test are identified as alternative models, so that when the current model encounters performance problems or network problems, it can be quickly switched to the best performing alternative model to ensure that the user's voice call system always runs in the best state. In addition, the existence of alternative models also provides more options and flexibility for future performance optimization and model upgrades.

[0116] In S4, the total translation time of the current model for the current voice call is obtained by recording the start translation time and the end translation time of the current model for the user's current language call. When the user uses the current model to complete the current voice call, the text data translated by the current model is output, and the number of translated words of the current voice call by the current model is detected based on the text data. The unit processing time of the current model for the current voice call is equal to the ratio of the total translation time to the number of translated words. The size of the unit processing time can reflect the speed of the translation. Based on the unit processing time, the timeliness score of the current model is obtained. The principle is the same as that of S23, which will not be repeated here.

[0117] The current model will transmit the translated text data to the large language model (LLM) as the input of the large language model. The large language model will not only generate response content based on the input, and then convert the text back to voice audio through speech synthesis (TTS) technology, but also judge the sentences of the text data, such as detecting typos and fluency through prompts. The typo parameters and logic parameters are obtained through the typo and fluency detection results fed back by the large language model.

[0118] If the large language model feedback shows that there are no typos, the typo parameter is determined to be equal to 0. If the large language model feedback shows that there are typos, the typo parameter is determined to be equal to the ratio of the number of typos fed back by the large language model to the total number of words in the text data. If the large language model feedback shows that the sentence is translated smoothly, the logic parameter is determined to be equal to the first preset score. If the large language model feedback shows that the sentence is translated incoherently, the logic parameter is determined to be equal to the second preset score. The large language model can also detect the specific fluency and determine the specific value of the logic parameter according to the fluency, but the minimum value is the second preset score.

[0119] Based on the typo parameter and the logic parameter, the usage quality parameter of the current model is obtained. The principle is the same as that of S24 and will not be repeated here.

[0120] Record the collection time of unit processing time, typo parameters and logic parameters (assume that these three parameters are collected at the same time). After each unit processing time and its collection time are obtained, the time analysis graph will be updated according to the unit processing time and its collection time, and the average slope of the time analysis graph will be recalculated to update the value of the speed improvement parameter. Similarly, after each typo parameter and its collection time are obtained, the typo analysis graph will be updated according to the typo parameter and its collection time, and the average slope of the typo analysis graph will be recalculated to update the value of the accuracy improvement parameter. After each logic parameter and its collection time are obtained, the semantic analysis graph will be updated according to the logic parameter and its collection time, and the average slope of the semantic analysis graph will be recalculated to update the value of the semantic quality improvement parameter.

[0121] After the current model completes the translation of the user's current voice call, the performance improvement parameter of the current model is calculated based on the updated speed improvement parameter, accuracy improvement parameter, and semantic quality improvement parameter. The calculation formula for the performance improvement parameter is as follows:

[0122] C′ i =α1×S′ i +α2×Z′ i +α3×Y i ′; (Formula 3)

[0123] In formula 3, C′ i S′ is the performance improvement parameter of the i-th speech recognition model for the current voice call; i is the speed improvement parameter after updating the i-th speech recognition model; Z′ i Y′ is the accuracy improvement parameter of the updated i-th speech recognition model; i The updated semantic quality improvement parameters for the i-th speech recognition model.

[0124] The performance of the current model in the current voice call is evaluated through timeliness scoring, usage quality parameters, and performance improvement parameters to obtain a comprehensive performance score. The calculation formula for the comprehensive performance score is as follows:

[0125] F2 i =q×(ε1×p1+ε2×p2+ε3×C′ i ); (Formula 4)

[0126] In Formula 4, F2 iis the comprehensive performance score of the ith speech recognition model; q is the manual weight; p1 is the timeliness score of the ith speech recognition model for the current voice call; p2 is the usage quality parameter of the ith speech recognition model for the current voice call; ε1, ε2, ε3 are preset weights.

[0127] The existing large language model can correct and identify individual typos. In other words, for the input of the large language model, some of these inaccurate inputs can be ignored, and the evaluation of translation quality mainly relies on the feedback of the large language model. However, the problems of the large language model itself may lead to errors in the evaluation results of translation quality. Therefore, ε1 is designed to be greater than ε2, so that the calculation of the comprehensive performance score can focus more on the translation speed.

[0128] Through the above method, as users continue to use the voice call system, after each translation of the current model, the corresponding analysis chart can be updated to obtain new performance improvement parameters, and new timeliness scores and usage quality parameters can be collected, thereby obtaining the comprehensive performance score of the current model in each actual voice call. In addition, in actual voice calls, the current model can further learn the user's personal language characteristics and habits through continuous use by the user, achieving the effect of adjustment and optimization.

[0129] This method also introduces an adjustable artificial weight for each vendor's speech recognition model, and the initial value of this parameter is set to 1. In actual applications, the value of the artificial weight can be adjusted according to specific circumstances to achieve human intervention in a specific speech recognition model. For example, in the case of abnormal performance degradation of the model, user complaints, or the risk of privacy leakage of the model, the artificial weight is changed to 0, triggering the system to determine that the current model is abnormal, issue an alarm, and immediately stop using the current model and switch to other alternative models.

[0130] In S5, in addition to manually judging abnormalities, it is also possible to automatically judge whether the current model has abnormalities through real-time monitoring. Specifically, it is judged whether the comprehensive performance score obtained by the current model for N consecutive times is less than the first preset threshold, or within the first preset time range, a preset proportion of the comprehensive performance score is less than the second preset threshold; if so, it is judged that the current model has abnormalities; if not, it is judged that the current model is running normally, and there is no need to switch models, and the translation level of the current model continues to be monitored. For example, N is 2, the first preset time range is 1 minute, the preset proportion is 30%, and the first preset threshold and the second preset threshold are both 7, that is, when the comprehensive performance score obtained by the current model for 2 consecutive times is less than 7 points or the comprehensive performance score obtained within 1 minute exceeds 30% and is less than 7 points, the current model is judged to be abnormal.

[0131] In S6, when the current model is abnormal, an alternative model is selected to replace the current model. Specifically, the sum of the evaluation scores of each alternative model in M ​​consecutive performance analyses is obtained, where M is 3, the performance analysis includes testing and / or actual application evaluation, and the evaluation score includes a comprehensive test score and / or a comprehensive performance score. That is, if the performance analysis of the current model for the last three times is performed in the test, the comprehensive test scores obtained in the last three tests are obtained, and these comprehensive test scores are added to obtain the total; if the performance analysis of the current model for the last time is performed in the actual voice call translation of the user, and the performance analysis of the current model for the last two times is performed in the test, then the comprehensive performance score obtained by the current model once and the comprehensive test score obtained in the last two tests are obtained, and these scores are added to obtain the total; if the performance analysis of the current model for the last three times is performed in the actual voice call translation of the user, then the comprehensive performance scores obtained by the current model for the last three times are added to obtain the total.

[0132] The candidate model with the highest total score is determined as the best historical score, and the candidate model is used to replace the current model to become the new current model. This model switching method can ensure the smoothness and accuracy of user voice calls. Moreover, when the abnormal model returns to normal, the restored model will be used as a candidate model for future use when needed.

[0133] Speech recognition technology usually uses products from mature suppliers in the industry. These suppliers, with their long-term accumulation in their respective fields, are able to provide ASR products with good results in their areas of expertise. In the technical chain of intelligent call centers or other voice call scenarios, the translation effect and speed of ASR are crucial to the entire system. If the speed of ASR slows down, it will affect the speed of conversation replies, causing users to experience the unpleasant experience of no one answering; if the translation quality is poor, such as semantic errors or too many typos, it will directly affect the reply content of the large language model, thereby affecting the quality of language calls and user experience. This method realizes mutual backup and fault tolerance among suppliers when multiple ASR suppliers coexist, so that when a supplier's ASR product has problems or poor results, it can quickly switch to other suppliers' ASR products to maintain a high level of service and user experience.

[0134] Reference Figure 4 The present disclosure provides an adaptive speech recognition switching system, comprising:

[0135] The response module 101 is used to respond to the user's collection instruction and record the user's voice.

[0136] Get a voice sample;

[0137] The testing module 102 is used to test a plurality of preset speech recognition models based on speech samples and obtain a comprehensive test score of each speech recognition model;

[0138] A deployment module 103, configured to determine the speech recognition model with the highest comprehensive test score as the current model and deploy it to the user's voice call system, and determine the remaining speech recognition models as candidate models;

[0139] A monitoring module 104 is used to monitor the translation level of the current model in real time and obtain the comprehensive performance score of the current model;

[0140] The judgment module 105 is used to judge whether the current model is abnormal according to the comprehensive performance score; if so, the replacement module 106 is executed;

[0141] The replacement module 106 is used to replace the current model with the candidate model with the best historical score.

[0142] The various variations and specific examples of the adaptive speech recognition switching method provided above are also applicable to the adaptive speech recognition switching system provided in the present disclosure. Through the above detailed description of the adaptive speech recognition switching method, those skilled in the art can clearly know the implementation method of the adaptive speech recognition switching system. For the sake of brevity of the specification, it will not be described in detail here.

[0143] The computer device according to the embodiment of the present disclosure includes a memory and a processor. The memory is used to store non-temporary computer-readable instructions. Specifically, the memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may, for example, include a read-only memory (ROM), a hard disk, a flash memory, etc.

[0145] The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the computer device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory, so that the computer device performs all or part of the steps of the adaptive speech recognition switching method of each embodiment of the present disclosure.

[0146] Those skilled in the art should be able to understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the protection scope of the present disclosure.

[0147] like Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present disclosure is shown, which is a schematic diagram of the structure of a computer device suitable for implementing the embodiment of the present disclosure. Figure 5 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0148] like Figure 5 As shown, the computer device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer device are also stored. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0149] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as tapes, hard disks, etc.; and communication devices. The communication device can allow the computer device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 5 A computer device having various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0150] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of the adaptive speech recognition switching method of the embodiment of the present disclosure are executed.

[0151] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0152] According to the computer-readable storage medium of the embodiment of the present disclosure, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the adaptive speech recognition switching method of each embodiment of the present disclosure are executed.

[0153] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

[0154] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0155] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limiting. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0156] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0157] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0158] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0159] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0160] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0161] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An adaptive speech recognition switching method, characterized in that: include: In response to a collection instruction from a user, recording the user's voice to obtain a voice sample; Based on the speech sample, multiple preset speech recognition models are tested to obtain a comprehensive test score of each speech recognition model; The speech recognition model with the highest comprehensive test score is determined as the current model and deployed to the user's voice call system, and the remaining speech recognition models are determined as candidate models; Monitor the translation level of the current model in real time and obtain the comprehensive performance score of the current model; Determine whether the current model has an abnormality according to the comprehensive performance score; If so, the alternative model with the best historical score is used to replace the current model.

2. The adaptive speech recognition switching method according to claim 1, characterized in that: The step of recording the user's voice and obtaining a voice sample in response to the user's collection instruction includes: Build a text template library; Receive several selection requests from users; In response to each selection request, a plurality of target templates are screened out from the text template library; Configure a capture button for each target template; When the user triggers the acquisition button on the front-end interface, a corresponding acquisition instruction is generated; In response to the acquisition instruction, starting an audio capture device; The audio capture hardware records the user's speech and generates a voice sample.

3. The adaptive speech recognition switching method according to claim 1, characterized in that: The method of testing a plurality of preset speech recognition models based on the speech sample to obtain a comprehensive test score of each speech recognition model includes: Dividing the speech sample into a plurality of speech segments; Using the multiple voice clips to test each speech recognition model multiple times, and obtaining the single-word translation time, wrong character ratio and fluency parameter of each speech recognition model in each test; Based on the single word translation duration, obtaining a duration grade score for each speech recognition model; Based on the percentage of wrong characters and the fluency parameter, obtaining a test quality parameter of each speech recognition model; Based on the single-word translation time, the percentage of wrong characters and the fluency parameter, obtaining a learning ability parameter of each speech recognition model; Based on the duration grade score, the test quality parameter and the learning ability parameter, a comprehensive test score of each speech recognition model is obtained.

4. The adaptive speech recognition switching method according to claim 3, characterized in that: The acquiring of the learning ability parameter of each speech recognition model based on the single word translation time, the wrong character ratio and the fluency parameter includes: Based on the single word translation time, the wrong character ratio and the fluency parameter, respectively construct a time analysis graph, a wrong character analysis graph and a semantic analysis graph for each speech recognition model; Based on the duration analysis graph, obtaining a speed improvement parameter for each speech recognition model; Based on the wrong character analysis graph, obtain the accuracy improvement parameter of each speech recognition model; Based on the semantic analysis graph, obtaining semantic quality improvement parameters of each speech recognition model; Based on the speed improvement parameter, the accuracy improvement parameter and the semantic quality improvement parameter, a learning ability parameter of each speech recognition model is obtained.

5. The adaptive speech recognition switching method according to claim 1, characterized in that: The real-time monitoring of the translation level of the current model and obtaining the comprehensive performance score of the current model include: Monitor the performance of the current model in the current voice call, and obtain unit processing time, typo parameters, and logic parameters; Based on the unit processing time, obtaining a timeliness score of the current model; Based on the wrong character parameter and the logic parameter, obtaining a usage quality parameter of the current model; Based on the unit processing time, the wrong character parameter and the logic parameter, obtaining a performance improvement parameter of the current model; Based on the timeliness score, the usage quality parameter and the performance improvement parameter, the performance of the current model in the current voice call is evaluated to obtain a comprehensive performance score.

6. The adaptive speech recognition switching method according to claim 1, characterized in that: The determining whether the current model is abnormal according to the comprehensive performance score includes: Determine whether the comprehensive performance scores of the current model obtained N times in a row are less than a first preset threshold, or whether a preset proportion of the comprehensive performance scores within a first preset time range are less than a second preset threshold; If yes, it is determined that the current model has an abnormality; If not, it is determined that the current model runs normally.

7. The adaptive speech recognition switching method according to claim 1, characterized in that: The adopting of the alternative model with the best historical score to replace the current model includes: Obtaining the sum of evaluation scores of each candidate model in M ​​consecutive performance analyses, wherein the evaluation scores include comprehensive test scores and / or comprehensive performance scores; The candidate model with the highest total score is determined as having the best historical score and replaces the current model.

8. A computer device, characterized in that: The computer device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the adaptive speech recognition switching method described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the adaptive speech recognition switching method described in any one of claims 1-7.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images