Voice interaction strategy generation method and device based on large model, equipment and medium

Abstract

The invention relates to the technical field of voice processing, and discloses a voice interaction strategy generation method and device based on a large model, equipment and a medium, and the method comprises the steps: judging whether original voice information is broadcast voice information or not; under the condition that the original voice information is not the broadcast voice information, judging whether the original voice information is real voice information or not through a voice authentic identification model; and under the condition that the original voice information is real voice information, generating a voice interaction strategy according to the target semantic information of the original voice information. Through the above mode, the intelligent customer service is prevented from performing invalid processing on the original voice information, and when the original voice is confirmed to be real voice, the semantic information of the voice is recognized through the preset language large model to generate the corresponding voice interaction strategy, so that the intelligent customer service understands the real demand of a customer; and the accuracy of identifying the customer voice and generating the interactive verbal skill by the intelligent customer service system in the fields of financial science and technology, medical health, old-age care and the like is improved.

Description

Technical Field

[0001] The present application relates to the field of speech processing technology and can be applied to business areas such as financial technology, medical care, and elderly care. In particular, it relates to a method, device, equipment, and medium for generating a speech interaction strategy based on a large model. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, intelligent customer service systems are increasingly being used in various sectors, including fintech, healthcare, and senior care. Financial institutions are leveraging intelligent customer service to conduct diverse financial services, covering key areas such as insurance sales and loan product promotion. The introduction of intelligent customer service offers two advantages for financial institutions: first, it significantly improves the quality and efficiency of customer service, enabling around-the-clock, immediate responses to customer inquiries, providing accurate financial product information and business guidance, and thus enhancing customer satisfaction; second, it effectively reduces financial institutions' operating costs related to customer service personnel.

[0003] In the healthcare and elderly care sectors, many medical institutions are introducing intelligent customer service systems to address the daily influx of patient consultations, appointment bookings, and medical information inquiries. These systems offer 24 / 7 service, significantly reducing the workload of human customer service representatives while improving the patient experience.

[0004] However, existing technologies have many significant problems in processing the voices of client-side robot agents. First, the accuracy of customer identity recognition is difficult to guarantee. Telephone robot agents usually use recorded voices that are close to real human voices, which makes it easy for intelligent customer service systems to misjudge them as real human voices, or even mistakenly think they are the voices of the customer themselves, leading to a series of subsequent processing errors. Secondly, due to inaccurate customer identity recognition, the intelligent customer service and the customer robot agent often fall into multiple rounds of communication. For intelligent customer service based on artificial intelligence technology, accurately identifying the voices of such telephone robot agents faces many challenges, resulting in a large number of invalid or inefficient calls and a waste of valuable resources. Therefore, how to improve the accuracy of intelligent customer service systems in recognizing customer voices and generating interactive dialogue in business fields such as financial technology and medical health and elderly care has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] This application provides a method, device, equipment and medium for generating voice interaction strategies based on a large model to improve the accuracy of intelligent customer service systems in recognizing customer voices and generating interactive dialogues in business fields such as financial technology and medical health and elderly care.

[0006] In a first aspect, the present application provides a method for generating a voice interaction strategy based on a large model, the method comprising:

[0007] Determining whether the original voice information is broadcast voice information based on the original voice print vector and each preset voice print vector in the preset voice print vector library, wherein the original voice print vector is the voice print vector corresponding to the original voice information;

[0008] In the case where it is determined that the original voice information is not the broadcast voice information, determining whether the original voice information is real voice information by using a preset voice authentication model;

[0009] When it is determined that the original voice information is the real voice information, the target semantic information of the original voice information is identified through a preset language model, and a voice interaction strategy for the original voice information is generated according to the target semantic information.

[0010] In a second aspect, the present application further provides a device for generating a speech interaction strategy based on a large model, the device comprising:

[0011] a broadcast voice information determination module, configured to determine whether the original voice information is broadcast voice information based on the original voice print vector and each preset voice print vector in the preset voice print vector library, wherein the original voice print vector is the voice print vector corresponding to the original voice information;

[0012] A real voice information judgment module is used to judge whether the original voice information is real voice information by using a preset voice authentication model when it is determined that the original voice information is not the broadcast voice information;

[0013] A voice interaction strategy generation module is used to identify the target semantic information of the original voice information through a preset language model when the original voice information is determined to be the real voice information, and to generate a voice interaction strategy for the original voice information based on the target semantic information.

[0014] In a third aspect, the present application also provides a computer device comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the above-mentioned large model-based voice interaction strategy generation method when executing the computer program.

[0015] In a fourth aspect, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the large model-based voice interaction strategy generation method as described above.

[0016] The present application discloses a method, apparatus, device and medium for generating a voice interaction strategy based on a large model. The method for generating a voice interaction strategy based on a large model includes judging whether the original voice information is a broadcast voice information based on an original voice voiceprint vector and each preset voice voiceprint vector in a preset voiceprint vector library, wherein the original voice voiceprint vector is a voiceprint vector corresponding to the original voice information; when it is judged that the original voice information is not the broadcast voice information, judging whether the original voice information is a real voice information by using a preset voice authentication model; when it is judged that the original voice information is the real voice information, identifying the target semantic information of the original voice information by using a preset language large model, and generating a voice interaction strategy for the original voice information according to the target semantic information. Through the above method, this application compares the original voiceprint vector with the voiceprint vector in the preset voiceprint vector library to quickly determine whether the original voice information is a broadcast voice information, effectively avoiding the intelligent customer service from performing invalid processing on the original voice information. When the original voice is confirmed to be real voice, the preset language large model is used to recognize the semantic information of the voice to generate a corresponding voice interaction strategy, so that the intelligent customer service can understand the real needs of the customer, and improve the accuracy of the intelligent customer service system in recognizing customer voice and generating interactive dialogue in the fields of financial technology, medical health and elderly care. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 This is a schematic flow chart of a method for generating a voice interaction strategy based on a large model provided in an embodiment of the present application;

[0019] Figure 2 A schematic block diagram of a large-model-based voice interaction strategy generation device provided in an embodiment of the present application;

[0020] Figure 3 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0023] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0025] The embodiments of the present application provide a method, apparatus, device, and medium for generating a voice interaction strategy based on a large model. Specifically, the method can be applied to an intelligent customer service system. By comparing the original voiceprint vector with the voiceprint vectors in a preset voiceprint vector library, it can quickly determine whether the original voice information is a broadcast voice information, effectively preventing the intelligent customer service from invalidating the original voice information. When the original voice is confirmed to be real voice, the preset language large model is used to recognize the semantic information of the voice to generate a corresponding voice interaction strategy, enabling the intelligent customer service to understand the real needs of customers, thereby improving the accuracy of intelligent customer service systems in recognizing customer voices and generating interactive dialogue in fields such as financial technology, medical care, and elderly care.

[0026] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0027] See also Figure 1 , Figure 1 This is a schematic flow chart of a method for generating a large-scale model-based voice interaction strategy, as provided in an embodiment of this application. This method can be applied to intelligent customer service systems, improving the accuracy of intelligent customer service systems in recognizing customer voice and generating interactive dialogue in fields such as fintech, healthcare, and senior care.

[0028] like Figure 1 As shown, the method for generating a voice interaction strategy based on a large model specifically includes steps S10 to S30.

[0029] Step S10: determining whether the original voice information is broadcast voice information based on the original voice print vector and each preset voice print vector in the preset voice print vector library, wherein the original voice print vector is the voice print vector corresponding to the original voice information;

[0030] Specifically, the intelligent customer service system collects various common broadcast voice information, such as operator voice broadcasts, system preset voice broadcasts, etc. For each broadcast voice, it extracts its voiceprint vector and stores these voiceprint vectors in a preset voiceprint vector library. Through voice input devices or communication lines, it obtains the original voice information to be judged.

[0031] The original speech information is preprocessed, such as removing silence and voice purification, to improve the accuracy of voiceprint extraction. The voiceprint extraction model is then used to extract the voiceprint vector of the original speech, obtaining the original speech voiceprint vector. The extracted original speech voiceprint vector is then compared with each preset voiceprint vector in the preset voiceprint vector library. For example, similarity calculation methods can be used to calculate the similarity between the original speech voiceprint vector and each preset voiceprint vector.

[0032] Based on the similarity results, a threshold is set. If the similarity between the original voiceprint vector and a preset voiceprint vector in the preset voiceprint vector library is greater than or equal to the threshold, the original voice information is determined to be a broadcast voice information; otherwise, the original voice information is determined not to be a broadcast voice information.

[0033] Step S20: If it is determined that the original voice information is not the broadcast voice information, determine whether the original voice information is real voice information by using a preset voice authentication model;

[0034] Specifically, this embodiment will be explained using the financial services sector as an example. In this context, when a customer calls a financial institution's customer service hotline, the intelligent customer service system first receives the customer's original voice message. Next, the system's voiceprint recognition module begins processing the raw voice message and extracting its voiceprint vector, known as the original voiceprint vector. Simultaneously, the intelligent customer service system accesses a pre-set voiceprint vector library, which stores various common voiceprint vectors for broadcast speech, such as those from carriers and the financial institution's own system.

[0035] In the healthcare and elderly care sectors, when patients or visitors consult for medical information, make appointments, or inquire about hospital services via phone or voice messaging systems, the intelligent customer service system receives their original voice information. For example, if a patient calls to inquire about the clinic hours of a specialist in a specific department, the intelligent customer service system converts the original voice information into a raw voice voiceprint vector and matches it with a preset voice voiceprint vector in a preset voiceprint vector library. The preset voiceprint vector library contains common hospital announcements, such as voice prompts in automated medical guide systems and hospital service notification announcements. If the match result indicates that the original voice information is a broadcast voice information, the intelligent customer service system will execute a strategy to terminate the interaction.

[0036] Based on the model's judgment, if the original voice message is recognized as real, the intelligent customer service system will continue the voice interaction, such as providing the patient with the medical information they need or guiding them through the appointment process. If it is recognized as a TTS synthesized voice message, the intelligent customer service system will terminate the interaction or take other appropriate security measures to prevent potential malicious attacks or invalid interactions, ensuring safety and reliability. For example, for real voice, the intelligent customer service system can inform the patient of the specific time and location of the specialist clinic visit, as well as the required documents. For TTS synthesized voice, the intelligent customer service system may directly hang up or request more information to verify the other party's identity.

[0037] The preset voice authentication model compares the original voiceprint vector with the preset voiceprint vectors in the preset voiceprint vector library. If the comparison results indicate that the original voice information is not a broadcast voice information, the system will call the preset voice authentication model. The preset voice authentication model is based on a deep learning algorithm and is trained using a large number of real voice and TTS (Text-to-Speech) synthesized voice samples. It can accurately analyze the acoustic, semantic, and prosodic characteristics of the voice.

[0038] A pre-set voice authentication model performs a multi-dimensional analysis of the original voice information to determine whether it is authentic. If it is, the system further uses a pre-set large language model to identify the target semantic information of the voice and generate a corresponding voice interaction strategy to provide accurate and personalized financial services, such as answering customer inquiries about financial products and assisting with business transactions, thereby improving customer service quality and business processing efficiency.

[0039] Step S30: When it is determined that the original voice information is the real voice information, the target semantic information of the original voice information is identified through a preset language model, and a voice interaction strategy for the original voice information is generated according to the target semantic information.

[0040] Specifically, once the intelligent customer service system receives the customer's original voice message and verifies it as authentic through voiceprint recognition and voice authentication models, it further processes the voice. Using a pre-set language model, the system uses speech-to-text technology to accurately convert the original voice message into text. The language model then performs in-depth semantic analysis on the text, extracting key semantic information from the customer's voice. For example, the customer may be asking about specific needs, such as credit card application procedures, loan interest rates, or insurance product recommendations.

[0041] Based on the identified target semantic information, a matching voice interaction strategy is generated according to the preset business logic and dialogue strategy. For example, if a customer is inquiring about the details of a newly launched wealth management product, the interaction strategy will cover key factors such as the product's return characteristics, risk level, and investment period, providing a detailed introduction to the customer with clear and professional voice responses, ensuring that the customer receives accurate, comprehensive, and targeted financial information, thereby enhancing the customer consultation experience and improving the service efficiency and business development capabilities of financial institutions.

[0042] The present embodiment discloses a method for generating a voice interaction strategy based on a large model, which includes judging whether the original voice information is broadcast voice information based on the original voiceprint vector and each preset voice voiceprint vector in a preset voiceprint vector library, wherein the original voice voiceprint vector is the voiceprint vector corresponding to the original voice information; when it is judged that the original voice information is not the broadcast voice information, judging whether the original voice information is real voice information through a preset voice authentication model; when it is judged that the original voice information is the real voice information, identifying the target semantic information of the original voice information through a preset language large model, and generating a voice interaction strategy for the original voice information according to the target semantic information. Through the above method, this application compares the original voiceprint vector with the voiceprint vector in the preset voiceprint vector library to quickly determine whether the original voice information is a broadcast voice information, effectively avoiding the intelligent customer service from performing invalid processing on the original voice information. When the original voice is confirmed to be real voice, the preset language large model is used to recognize the semantic information of the voice to generate a corresponding voice interaction strategy, so that the intelligent customer service can understand the real needs of the customer, and improve the accuracy of the intelligent customer service system in recognizing customer voice and generating interactive dialogue in the fields of financial technology, medical health and elderly care.

[0043] based on Figure 1 In the embodiment shown, in this embodiment, step S30 includes:

[0044] Extracting original text information and original prosody information of the original speech information through the preset language large model;

[0045] Specifically, a financial institution's intelligent customer service system receives original voice messages from customers via telephone lines or other voice input channels. For example, a customer calls to inquire about the details of a financial product.

[0046] The pre-set language model within the intelligent customer service system first processes the raw voice information, converting it into corresponding raw text, accurately identifying the vocabulary and sentences contained in the customer's speech. Simultaneously, it analyzes the acoustic characteristics of the speech and extracts raw prosodic information, including fluctuations in intonation, speaking speed, placement of stress, and duration of pauses. For example, it extracts the textual information of a customer's voice, such as "I want to learn about this financial product," along with prosodic features such as the customer's calm intonation and moderate speaking speed.

[0047] determining the target semantic information according to the original text information and the original prosody information;

[0048] Specifically, based on the extracted raw text and prosodic information, a pre-set language model performs in-depth semantic analysis of the speech. This model comprehensively considers factors such as the meaning of words, the structure of sentences, and the emotion and tone conveyed by the prosody to accurately determine the target semantic information expressed by the customer. For example, it can understand that the customer wants to know detailed information about a financial product, such as its returns, risk level, and investment period, and that the customer is currently cautious about the product.

[0049] According to the target semantic information, target interaction information is determined from a preset interaction voice library, and the voice interaction strategy is generated based on the target interaction information.

[0050] Specifically, the preset interactive voice database contains a large number of interactive voice templates related to financial services, covering responses to various common customer consultation scenarios and questions. Based on the identified target semantic information, the interactive voice database is searched and matched. For example, multiple interactive voice templates related to "financial product detailed information consultation" are searched.

[0051] From the matched interactive voice templates, the target interaction information that best meets the current customer needs and context is selected. Based on this target interaction information and in conjunction with the financial institution's business policies and service processes, a specific voice interaction strategy is generated. For example, a clear, professional, and friendly tone is used to respond to customers, first explaining the expected rate of return and past performance of the wealth management product, then detailing the product's risk assessment results and suitable investor demographic, and finally explaining the product's investment period and liquidity characteristics. This creates a comprehensive voice interaction strategy that guides the intelligent customer service system in accurately and effectively responding to customer inquiries and meeting their financial needs.

[0052] based on Figure 1 In the embodiment shown, in this embodiment, before step S20, the following steps are included:

[0053] Obtaining a text-to-speech (TTS) synthesized speech training set and a real speech verification set, and extracting TTS training set features of the TTS synthesized speech training set and real speech verification set features of the real speech verification set respectively;

[0054] Specifically, we collected samples from different TTS systems, covering a variety of voice styles, speaking speeds, and intonations, to form a TTS synthetic speech training set. At the same time, we collected real speech samples to form a real speech verification set, ensuring that the speech in the real speech verification set is diverse and represents the speech characteristics of different speakers and different environments.

[0055] Perform feature extraction on each sample in the TTS synthesized speech training set to obtain TTS training set features. The extracted features can include acoustic features (such as Mel-frequency cepstral coefficients, fundamental frequency, energy, etc.), semantic features (features obtained by converting speech to text and then performing word meaning understanding), and prosodic features (such as intonation patterns, rhythm, and speaking rate).

[0056] Performing feature extraction on each sample in the real speech verification set in the same manner to obtain real speech verification set features, aggregating the real speech verification set features in the real speech verification set to generate a preset real speech information library;

[0057] Specifically, the various real speech verification set features in the real speech verification set are aggregated. A feature fusion algorithm can be used to integrate multiple real speech verification set features into a comprehensive feature library. These comprehensive feature libraries then constitute the pre-set real speech information library. For example, statistical aggregation is performed on real speech verification set features of the same type, and statistics such as mean and variance are calculated to characterize the typical characteristics of real speech.

[0058] Cross-validating the TTS training set features and each of the real voice information in the preset real voice information database using the initial voice authentication model to determine the model parameters of the initial voice authentication model;

[0059] The preset voice authentication model is determined according to the initial voice authentication model and the model parameters.

[0060] Specifically, to build the initial speech authentication model, you can choose common deep learning model architectures, such as convolutional neural networks, recurrent neural networks, etc., initialize the model parameters, and set hyperparameters such as the network's hierarchical structure and the number of neurons.

[0061] After multiple rounds of cross-validation and parameter adjustments, when the initial voice authentication model achieves satisfactory accuracy and recall in distinguishing between TTS synthesized speech and real speech, it is optimized to a pre-set voice authentication model based on the finalized model parameters. This optimized pre-set voice authentication model can be used in actual voice authentication tasks, helping financial institutions' intelligent customer service systems accurately identify the authenticity and source of speech, improving the system's ability to handle different types of speech and reducing the risk of being misled by TTS synthesized speech.

[0062] In a specific embodiment, step S20 includes:

[0063] Extracting original acoustic features, original semantic features, and original prosodic features of the original speech information;

[0064] Specifically, acoustic features are extracted from the original speech information, including Mel-frequency cepstral coefficients, acoustic spectrum, fundamental frequency, formants, harmonic structure, and energy distribution. These features can be calculated using specialized speech signal processing libraries. For example, the spectrum of the speech signal can be calculated using a short-time Fourier transform, from which the Mel-frequency cepstral coefficients are extracted.

[0065] The speech-to-text model converts raw speech into text, and then performs natural language processing on the resulting text, including part-of-speech tagging, grammatical analysis, and semantic understanding, to extract semantic features. For example, the semantic content of the speech can be determined by analyzing keywords and entity relationships in the text.

[0066] Analyze the prosodic features of the original speech information, such as intonation pattern, rhythm, speaking speed, stress position, pause position and time interval. For example, the intonation pattern can be determined by calculating the fundamental frequency change of the speech signal, and the rhythm and speaking speed can be determined by analyzing the duration and interval of the speech segments.

[0067] Matching the original acoustic features, the original semantic features, and the original prosodic features with the preset acoustic features, preset semantic features, and preset prosodic features of each of the real speech information in the preset real speech information database through the preset speech authentication model;

[0068] Specifically, the extracted original acoustic features, original semantic features, and original prosodic features are matched with preset acoustic features, preset semantic features, and preset prosodic features in a preset real speech information database. The preset real speech information database contains a large amount of feature information that has been marked as real speech.

[0069] For each feature dimension, calculate the similarity between the original feature and the preset feature. For example, cosine similarity can be used to calculate the similarity between feature vectors. Assume that the original acoustic feature vector is S = [s1, s2, ..., sn ], the preset acoustic eigenvector is R=[r1,r2,...,r n ], then their cosine similarity is cosθ=(S·R) / (|S|×|R|), where S·R is the dot product of the vectors, and |S| and |R| are the moduli of the vectors respectively.

[0070] If the target voice information exists in the preset real voice information database, determining that the original voice information is the real voice information;

[0071] If the target voice information does not exist in the preset real voice information database, determining that the original voice information is TTS synthesized voice information;

[0072] The acoustic features of the target speech information match the original acoustic features, the semantic features of the target speech information match the original semantic features, and the prosodic features of the target speech information match the original prosodic features.

[0073] Specifically, the similarity threshold is set according to the actual application scenario and requirements. For example, the similarity threshold for acoustic features is set to 0.8, the similarity threshold for semantic features is set to 0.7, and the similarity threshold for prosodic features is set to 0.6.

[0074] If a target voice message is found in the preset real voice information database and its similarity with the original voice message is greater than or equal to the respective thresholds, the original voice message is determined to be real voice information. For example, if the acoustic feature similarity of the original voice message and a real voice message in the database is 0.85, the semantic feature similarity is 0.75, and the prosodic feature similarity is 0.65, all of which are above the set thresholds, then it is determined to be real voice;

[0075] If the target speech information does not meet the similarity threshold in the preset real speech information database, the original speech information is determined to be TTS synthesized speech information. For example, if the similarity between the original speech features and the features of all real speech information in the database is lower than the set threshold, it is determined to be TTS synthesized speech.

[0076] In a specific embodiment, step S20 includes:

[0077] If the target voice information does not exist in the preset real voice information database, after determining that the original voice information is TTS synthesized voice information, the method includes:

[0078] When it is determined that the original voice information is the TTS synthesized voice information, interaction information is determined based on the original voice information and the interaction information is used as the voice identification strategy, or the voice interaction strategy is determined to be a termination interaction strategy.

[0079] Specifically, the original voice information is analyzed to extract key information, such as the customer's consultation content, demand expression, etc. Natural language processing technology can be used to perform semantic analysis on the information converted from TTS synthesized speech to text to understand the customer's intention.

[0080] Based on the analyzed customer intent, select appropriate interaction messages from the pre-set interaction template library. For example, if the customer is asking about service hours, you can choose an interaction message like "Hello, our service hours are Monday to Friday, 9:00 AM to 5:00 PM."

[0081] The selected interactive information is used as a voice interaction strategy, converted into voice through the TTS synthesis system, and played to the customer to achieve effective communication with the customer and provide corresponding services or information.

[0082] The system determines whether to terminate the interaction based on pre-set rules and scenarios. For example, if the TTS-synthesized voice message is clearly a marketing call, a malicious harassment call, or invalid information irrelevant to the business, the system will determine a termination strategy. For example, it can play a pre-set termination prompt to the customer, such as "Hello, your call is not within our service scope. Thank you for your call. Goodbye."

[0083] based on Figure 1 In the embodiment shown, in this embodiment, step S10 includes:

[0084] Converting the original speech information into the original speech voiceprint vector;

[0085] Matching the original speech voiceprint vector with each of the preset speech voiceprint vectors in the preset voiceprint vector library;

[0086] If the preset voiceprint vector library contains the preset voiceprint vector that matches the original voiceprint vector, the original voice information is determined to be the broadcast voice information.

[0087] Specifically, the preprocessed speech is analyzed using a voiceprint extraction model to extract voiceprint feature parameters and generate an original speech voiceprint vector. Voiceprint feature parameters can include Mel-frequency cepstral coefficients, linear prediction coding coefficients, waveform length variation, etc. Using a suitable matching algorithm, the original speech voiceprint vector is compared with each preset voiceprint vector in the library.

[0088] According to the matching results, if a preset voiceprint vector with a similarity higher than the set threshold to the original voiceprint vector is found, the original voice information is determined to be a broadcast voice information; if no voiceprint vector with a similarity that meets the conditions is found, the original voice information is determined not to be a broadcast voice information.

[0089] In a specific embodiment, if the preset voiceprint vector library contains the preset voiceprint vector that matches the original voiceprint vector, then after determining that the original voice information is the broadcast voice information, the method further includes:

[0090] In a case where the original voice information is the broadcast voice information, the voice interaction strategy is determined to be a termination interaction strategy.

[0091] Specifically, if the original voice message is determined to be a broadcast voice message, a termination interaction strategy will be initiated. For example, the intelligent customer service system will play a preset termination prompt voice message, such as "Hello, your call has been identified as an automatic broadcast voice message. This call will be terminated. Thank you for your call. Goodbye." After playing the termination prompt voice message, the intelligent customer service system will automatically hang up the call.

[0092] See also Figure 2 , Figure 2 The embodiment of the present application provides a schematic block diagram of a large-model-based voice interaction strategy generation device, which is used to execute the aforementioned large-model-based voice interaction strategy generation method. The large-model-based voice interaction strategy generation device can be configured on a server.

[0093] like Figure 2 As shown, the large model-based voice interaction strategy generation device 400 includes:

[0094] The broadcast voice information determination module 410 is configured to determine whether the original voice information is the broadcast voice information based on the original voice print vector and each preset voice print vector in the preset voice print vector library, wherein the original voice print vector is the voice print vector corresponding to the original voice information;

[0095] The real voice information determination module 420 is used to determine whether the original voice information is real voice information by using a preset voice authentication model when it is determined that the original voice information is not the broadcast voice information;

[0096] The voice interaction strategy generation module 430 is used to identify the target semantic information of the original voice information through a preset language model when the original voice information is determined to be the real voice information, and generate a voice interaction strategy for the original voice information based on the target semantic information.

[0097] Furthermore, the voice interaction strategy generation module 430 includes:

[0098] an original information extraction unit, configured to extract original text information and original prosody information of the original speech information through the preset language macro model;

[0099] a target semantic information determining unit, configured to determine the target semantic information based on the original text information and the original prosody information;

[0100] The voice interaction strategy generating unit is used to determine target interaction information from a preset interaction voice library according to the target semantic information, and generate the voice interaction strategy based on the target interaction information.

[0101] Furthermore, the large model-based voice interaction strategy generation device 400 includes:

[0102] A training set verification set acquisition module is used to obtain a text-to-speech conversion (TTS) synthetic speech training set and a real speech verification set, and respectively extract TTS training set features of the TTS synthetic speech training set and real speech verification set features of the real speech verification set;

[0103] A real voice information database generation module is used to aggregate the features of each real voice verification set in the real voice verification set to generate a preset real voice information database;

[0104] A model parameter determination module is used to cross-validate the TTS training set features and the real voice information in the preset real voice information library through the initial voice authentication model to determine the model parameters of the initial voice authentication model;

[0105] The preset voice authentication model determination module is used to determine the preset voice authentication model according to the initial voice authentication model and the model parameters.

[0106] Furthermore, the real voice information determination module 420 includes:

[0107] An original feature extraction unit, configured to extract original acoustic features, original semantic features, and original prosodic features of the original speech information;

[0108] a feature matching unit, configured to match the original acoustic features, the original semantic features, and the original prosodic features with the preset acoustic features, the preset semantic features, and the preset prosodic features of each of the real speech information in the preset real speech information database using the preset speech authentication model;

[0109] a real voice information determining unit, configured to determine that the original voice information is the real voice information if target voice information exists in the preset real voice information database;

[0110] a TTS synthesized voice information determining unit, configured to determine that the original voice information is TTS synthesized voice information if the target voice information does not exist in the preset real voice information database;

[0111] Furthermore, the real voice information determination module 420 includes:

[0112] The first strategy determination unit is configured to, when determining that the original voice information is the TTS synthesized voice information, determine interaction information based on the original voice information and use the interaction information as the voice identification strategy, or determine that the voice interaction strategy is a termination interaction strategy.

[0113] Furthermore, the broadcast voice information determination module 410 includes:

[0114] An original speech voiceprint vector conversion unit, configured to convert the original speech information into the original speech voiceprint vector;

[0115] a voiceprint vector matching unit, configured to match the original voiceprint vector with each of the preset voiceprint vectors in the preset voiceprint vector library;

[0116] The broadcast voice information determining unit is configured to determine that the original voice information is the broadcast voice information if the preset voiceprint vector library contains the preset voiceprint vector that matches the original voiceprint vector.

[0117] Furthermore, the broadcast voice information determination module 410 includes:

[0118] The termination interaction strategy determining unit is configured to determine, when the original voice information is the broadcast voice information, that the voice interaction strategy is a termination interaction strategy.

[0119] It should be noted that those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0120] The above-mentioned device can be realized in the form of a computer program. The computer program can be used in Figure 3 Runs on the computer equipment shown.

[0121] See also Figure 3 , Figure 3 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0122] See Figure 3 The computer device includes a processor, a memory, and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0123] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, which, when executed, can cause the processor to execute any one of the large model-based voice interaction strategy generation methods.

[0124] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0125] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any large model-based voice interaction strategy generation method.

[0126] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0127] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0128] In one embodiment, the processor is configured to execute a computer program stored in the memory to implement the following steps:

[0129] Determining whether the original voice information is broadcast voice information based on the original voice print vector and each preset voice print vector in the preset voice print vector library, wherein the original voice print vector is the voice print vector corresponding to the original voice information;

[0130] In the case where it is determined that the original voice information is not the broadcast voice information, determining whether the original voice information is real voice information by using a preset voice authentication model;

[0131] When it is determined that the original voice information is the real voice information, the target semantic information of the original voice information is identified through a preset language model, and a voice interaction strategy for the original voice information is generated according to the target semantic information.

[0132] In one embodiment, when the original voice information is determined to be the real voice information, the target semantic information of the original voice information is identified by a preset language model, and a voice interaction strategy for the original voice information is generated based on the target semantic information, so as to achieve:

[0133] Extracting original text information and original prosody information of the original speech information through the preset language large model;

[0134] determining the target semantic information according to the original text information and the original prosody information;

[0135] According to the target semantic information, target interaction information is determined from a preset interaction voice library, and the voice interaction strategy is generated based on the target interaction information.

[0136] In one embodiment, when it is determined that the original voice information is not the broadcast voice information, before determining whether the original voice information is real voice information by using a preset voice authentication model, it is used to implement:

[0137] Obtaining a text-to-speech (TTS) synthesized speech training set and a real speech verification set, and extracting TTS training set features of the TTS synthesized speech training set and real speech verification set features of the real speech verification set respectively;

[0138] Aggregating the features of each real speech verification set in the real speech verification set to generate a preset real speech information database;

[0139] Cross-validating the TTS training set features and each of the real voice information in the preset real voice information database using the initial voice authentication model to determine the model parameters of the initial voice authentication model;

[0140] The preset voice authentication model is determined according to the initial voice authentication model and the model parameters.

[0141] In one embodiment, when it is determined that the original voice information is not the broadcast voice information, a preset voice authentication model is used to determine whether the original voice information is real voice information, so as to achieve:

[0142] Extracting original acoustic features, original semantic features, and original prosodic features of the original speech information;

[0143] Matching the original acoustic features, the original semantic features, and the original prosodic features with the preset acoustic features, preset semantic features, and preset prosodic features of each of the real speech information in the preset real speech information database through the preset speech authentication model;

[0144] If the target voice information exists in the preset real voice information database, determining that the original voice information is the real voice information;

[0145] If the target voice information does not exist in the preset real voice information database, determining that the original voice information is TTS synthesized voice information;

[0146] The acoustic features of the target speech information match the original acoustic features, the semantic features of the target speech information match the original semantic features, and the prosodic features of the target speech information match the original prosodic features.

[0147] In one embodiment, if the target voice information does not exist in the preset real voice information database, after determining that the original voice information is TTS synthesized voice information, it is used to implement:

[0148] When it is determined that the original voice information is the TTS synthesized voice information, interaction information is determined based on the original voice information and the interaction information is used as the voice identification strategy, or the voice interaction strategy is determined to be a termination interaction strategy.

[0149] In one embodiment, based on the original voiceprint vector and each preset voiceprint vector in the preset voiceprint vector library, it is determined whether the original voice information is broadcast voice information, wherein the original voiceprint vector is the voiceprint vector corresponding to the original voice information, for achieving:

[0150] Converting the original speech information into the original speech voiceprint vector;

[0151] Matching the original speech voiceprint vector with each of the preset speech voiceprint vectors in the preset voiceprint vector library;

[0152] If the preset voiceprint vector library contains the preset voiceprint vector that matches the original voiceprint vector, the original voice information is determined to be the broadcast voice information.

[0153] In one embodiment, if the preset voiceprint vector library contains the preset voiceprint vector that matches the original voiceprint vector, then after determining that the original voice information is the broadcast voice information, the following steps are implemented:

[0154] In a case where the original voice information is the broadcast voice information, the voice interaction strategy is determined to be a termination interaction strategy.

[0155] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the large model-based voice interaction strategy generation methods provided in the embodiments of the present application.

[0156] The computer-readable storage medium may be an internal storage unit of the computer device described in the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a SmartMedia Card (SMC), a Secure Digital (SD) card, a flash memory card, etc., equipped on the computer device.

[0157] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for generating a speech interaction strategy based on a large model, characterized in that: include: Determining whether the original voice information is broadcast voice information based on the original voice print vector and each preset voice print vector in the preset voice print vector library, wherein the original voice print vector is the voice print vector corresponding to the original voice information; In the case where it is determined that the original voice information is not the broadcast voice information, determining whether the original voice information is real voice information by using a preset voice authentication model; When it is determined that the original voice information is the real voice information, the target semantic information of the original voice information is identified through a preset language model, and a voice interaction strategy for the original voice information is generated according to the target semantic information.

2. The method for generating a speech interaction strategy based on a large model according to claim 1, characterized in that: In the case where the original voice information is determined to be the real voice information, identifying the target semantic information of the original voice information by using a preset language model, and generating a voice interaction strategy for the original voice information according to the target semantic information, including: Extracting original text information and original prosody information of the original speech information through the preset language large model; determining the target semantic information according to the original text information and the original prosody information; According to the target semantic information, target interaction information is determined from a preset interaction voice library, and the voice interaction strategy is generated based on the target interaction information.

3. The method for generating a speech interaction strategy based on a large model according to claim 1, characterized in that: In the case where it is determined that the original voice information is not the broadcast voice information, before determining whether the original voice information is real voice information by using a preset voice authentication model, the method includes: Obtaining a text-to-speech (TTS) synthesized speech training set and a real speech verification set, and extracting TTS training set features of the TTS synthesized speech training set and real speech verification set features of the real speech verification set respectively; Aggregating the features of each real speech verification set in the real speech verification set to generate a preset real speech information database; Cross-validating the TTS training set features and each of the real voice information in the preset real voice information database using the initial voice authentication model to determine the model parameters of the initial voice authentication model; The preset voice authentication model is determined according to the initial voice authentication model and the model parameters.

4. The method for generating a speech interaction strategy based on a large model according to claim 3, characterized in that: In the case where it is determined that the original voice information is not the broadcast voice information, determining whether the original voice information is real voice information by using a preset voice authentication model includes: Extracting original acoustic features, original semantic features, and original prosodic features of the original speech information; Matching the original acoustic features, the original semantic features, and the original prosodic features with the preset acoustic features, preset semantic features, and preset prosodic features of each of the real speech information in the preset real speech information database through the preset speech authentication model; If the target voice information exists in the preset real voice information database, determining that the original voice information is the real voice information; If the target voice information does not exist in the preset real voice information database, determining that the original voice information is TTS synthesized voice information; The acoustic features of the target speech information match the original acoustic features, the semantic features of the target speech information match the original semantic features, and the prosodic features of the target speech information match the original prosodic features.

5. The method for generating a speech interaction strategy based on a large model according to claim 4, characterized in that: If the target voice information does not exist in the preset real voice information database, after determining that the original voice information is TTS synthesized voice information, the method further includes: When it is determined that the original voice information is the TTS synthesized voice information, interaction information is determined based on the original voice information and the interaction information is used as the voice identification strategy, or the voice interaction strategy is determined to be a termination interaction strategy.

6. The method for generating a speech interaction strategy based on a large model according to claim 1, characterized in that: The determining whether the original voice information is broadcast voice information based on the original voice print vector and each preset voice print vector in the preset voice print vector library, wherein the original voice print vector is a voice print vector corresponding to the original voice information, includes: Converting the original speech information into the original speech voiceprint vector; Matching the original speech voiceprint vector with each of the preset speech voiceprint vectors in the preset voiceprint vector library; If the preset voiceprint vector library contains the preset voiceprint vector that matches the original voiceprint vector, the original voice information is determined to be the broadcast voice information.

7. The method for generating a speech interaction strategy based on a large model according to claim 6, characterized in that: If the preset voiceprint vector library contains the preset voiceprint vector that matches the original voiceprint vector, then after determining that the original voice information is the broadcast voice information, the method further includes: In a case where the original voice information is the broadcast voice information, the voice interaction strategy is determined to be a termination interaction strategy.

8. A device for generating a speech interaction strategy based on a large model, characterized in that: include: a broadcast voice information determination module, configured to determine whether the original voice information is broadcast voice information based on the original voice print vector and each preset voice print vector in the preset voice print vector library, wherein the original voice print vector is the voice print vector corresponding to the original voice information; A real voice information judgment module is used to judge whether the original voice information is real voice information by using a preset voice authentication model when it is determined that the original voice information is not the broadcast voice information; A voice interaction strategy generation module is used to identify the target semantic information of the original voice information through a preset language model when the original voice information is determined to be the real voice information, and to generate a voice interaction strategy for the original voice information based on the target semantic information.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the large model-based voice interaction strategy generation method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the method for generating a speech interaction strategy based on a large model as described in any one of claims 1 to 7.

Images