An agent dialogue system and method

Abstract

The application provides an intelligent agent conversation system and method, and relates to the technical field of audio conversion, and comprises the following steps: a collection module is used for acquiring audio data of a user; an audio conversion module is used for setting a transmission mode of the audio data according to a data network signal strength of a user terminal, and when the transmission mode is communication signal transmission, the audio data is subjected to encoding processing to obtain an audio stream; a speech recognition processing module is used for performing real-time analysis on the audio stream to obtain an analysis result, and the analysis result is subjected to optimization through a language acoustic model to obtain text information; an intention recognition and answering module is used for generating answering text of the audio data according to the text information; and a speech synthesis module is used for performing speech synthesis on the answering text to obtain audio data of the answering text. The application uses text transmission to replace the audio stream, significantly reduces the data transmission amount, improves the transmission efficiency, reduces the delay and errors caused by network congestion, and improves the stability of the intelligent agent audio conversation.

Description

Technical Field

[0001] The present invention relates to the field of audio conversion technology, and in particular to an intelligent agent dialogue system and method. Background Art

[0002] In existing intelligent conversational interaction scenarios, users typically interact with intelligent agents by entering text messages through mobile apps or webpages. However, when manual input is inconvenient, such as while driving or exercising, convenient communication with intelligent agents becomes difficult. Therefore, users typically use apps on mobile devices such as mobile phones to conduct audio calls with intelligent agents.

[0003] In related technologies, most intelligent agent apps use data networks to implement conversations. Existing voice interactions often have the characteristic of being highly dependent on data network signals. Therefore, in scenarios with poor network signals, conversations with intelligent agents may not be smooth, and thus the user's demand for instant feedback from intelligent agents may not be met. Summary of the Invention

[0004] The problem that the present invention solves is how to improve the stability of intelligent agent audio dialogue.

[0005] To solve the above problems, the present invention provides an intelligent agent dialogue system and method.

[0006] In a first aspect, an agent dialogue system of the present invention includes a user end and a server end, wherein an output of the user end is communicatively connected to the server end;

[0007] The user end includes a collection module, an audio conversion module, a speech recognition processing module and a speech synthesis module which are connected in sequence; the server end includes an intention recognition and answering module;

[0008] The output end of the speech recognition processing module of the user end is connected to the input end of the intention recognition and answering module of the server end, and the output end of the intention recognition and answering module is connected to the input end of the speech synthesis module of the user end;

[0009] The acquisition module is used to obtain the user's audio data;

[0010] The audio conversion module is configured to set a transmission mode for the audio data according to the data network signal strength of the user terminal, and when the transmission mode is communication signal transmission, encode the audio data according to the transmission protocol of the communication push stream to obtain an audio stream corresponding to the audio data;

[0011] The speech recognition processing module is used to perform real-time analysis on the audio stream through deep learning to obtain an analysis result, and optimize the analysis result through a language acoustic model to obtain text information corresponding to the audio data;

[0012] The intention recognition and response module is used to determine the domain and intent of the text information corresponding to the audio data in combination with context information; and to obtain the response text of the audio data based on the domain and intent in combination with a logical reasoning algorithm and a language generation algorithm;

[0013] The speech synthesis module is used to perform speech synthesis on the answer text to obtain the audio stream corresponding to the answer text, and encode the audio stream corresponding to the answer text according to the transmission protocol of the communication push stream to obtain the audio data of the answer text.

[0014] Optionally, the audio conversion module is specifically configured to:

[0015] Obtaining the data network signal strength of the user terminal;

[0016] Setting the transmission mode of the audio data according to a magnitude relationship between the data network signal strength and a preset signal strength threshold;

[0017] Wherein, when the data network signal strength is less than the preset signal strength threshold, the transmission mode of the audio data is set to the communication signal transmission;

[0018] When the data network signal strength is greater than or equal to the preset signal strength threshold, the transmission mode of the audio data is set to data network transmission.

[0019] Optionally, the speech recognition processing module is specifically configured to:

[0020] Performing frame processing on the audio stream to divide the continuous audio signal in the audio stream into audio frames of multiple time periods;

[0021] Extracting acoustic features from each of the audio frames to obtain acoustic features of the audio frame;

[0022] Inputting the acoustic features into a deep learning model for nonlinear mapping to generate preliminary speech recognition results;

[0023] The preliminary speech recognition result is subjected to grammatical and semantic constraints by means of the language acoustic model to obtain the text information corresponding to the audio data.

[0024] Optionally, the intention recognition and answering module is specifically configured to:

[0025] Performing word segmentation and part-of-speech tagging on the text information to obtain the basic structure and grammatical features of the text information;

[0026] Using an intent classification model, based on the basic structure and the grammatical features, combined with contextual information, semantically understand and classify the text information to obtain the domain and intent of the text information;

[0027] According to the domain and the intention, combined with the logical reasoning algorithm, generate a logical reasoning result corresponding to the domain and the intention;

[0028] Then, the language generation algorithm is used to generate the answer text according to the logical reasoning result.

[0029] Optionally, the speech synthesis module is specifically configured to:

[0030] Converting the answer text into a speech signal according to preset acoustic parameters using a speech synthesis model;

[0031] Performing noise suppression and echo cancellation processing on the speech signal to obtain the processed speech signal;

[0032] The processed voice signal is encoded according to the transmission protocol of the communication push stream to generate the audio stream corresponding to the answer text.

[0033] Optionally, the audio conversion module is further configured to:

[0034] When the transmission mode is the data network transmission, the audio data is encoded according to the User Datagram Protocol to obtain network transmission data of the audio data.

[0035] Optionally, the speech synthesis module is specifically configured to:

[0036] When the transmission mode of the audio data is the data network transmission, determining the speech rate parameter, intonation parameter, volume parameter and timbre parameter of the speech synthesis model according to the data network signal strength;

[0037] Optimizing preset acoustic parameters of a speech synthesis model according to the speech rate parameter, the intonation parameter, the volume parameter, and the timbre parameter to obtain the optimized speech synthesis model;

[0038] The answer text is dynamically speech synthesized by using the optimized speech synthesis model in combination with forward error correction and redundant coding strategies to obtain the audio stream corresponding to the answer text.

[0039] Optionally, the intention recognition and answering module is specifically configured to:

[0040] determining network fluctuation data of the user terminal according to the data network signal strength;

[0041] Predicting based on the network fluctuation data to obtain future network fluctuation data during the transmission of the answer text;

[0042] The answer text is sent to the user terminal according to the future network fluctuation data.

[0043] Optionally, the intention recognition and answering module is further configured to:

[0044] determining a transmission frequency of the reply text according to the future network fluctuation data, and dividing the reply text into a plurality of text segments;

[0045] generating a sending plan for the reply text according to the transmission frequency of the reply text and the text fragment;

[0046] The text segment of the answer text is sent to the user terminal according to the sending plan.

[0047] In a second aspect, an agent dialogue method of the present invention is applied to any of the above-mentioned agent dialogue systems, wherein the agent dialogue system includes a user end and a server end, and the output of the user end is communicatively connected to the server end;

[0048] The user end includes a collection module, an audio conversion module, a speech recognition processing module and a speech synthesis module which are connected in sequence; the server end includes an intention recognition and answering module;

[0049] The output end of the speech recognition processing module of the user end is connected to the input end of the intention recognition and answering module of the server end, and the output end of the intention recognition and answering module is connected to the input end of the speech synthesis module of the user end;

[0050] The agent dialogue method comprises:

[0051] Acquire the user's audio data through the acquisition module;

[0052] The audio conversion module sets a transmission mode for the audio data according to the data network signal strength of the user terminal. When the transmission mode is communication signal transmission, the audio data is encoded according to the transmission protocol of the communication push stream to obtain an audio stream corresponding to the audio data.

[0053] The speech recognition processing module performs real-time analysis based on the audio stream through deep learning to obtain an analysis result, and optimizes the analysis result through a language acoustic model to obtain text information corresponding to the audio data;

[0054] The intention recognition and answer module determines the domain and intention of the text information according to the text information corresponding to the audio data in combination with context information; and obtains the answer text of the audio data in combination with the logical reasoning algorithm and the language generation algorithm based on the domain and the intention;

[0055] The answer text is speech-synthesized by the speech synthesis module to obtain an audio stream corresponding to the answer text, and the audio stream corresponding to the answer text is encoded according to the transmission protocol of the communication push stream to obtain audio data of the answer text.

[0056] In the intelligent agent dialogue system and method of the present invention, at the user end, after the voice recognition processing module converts the audio stream into text information, only the text information is transmitted to the server end. Since the amount of text data is much smaller than the amount of audio data, the data load during the transmission process is greatly reduced. This lightweight transmission method reduces the demand for data network bandwidth, so that the system can still communicate efficiently when the data network signal is unstable or the bandwidth is limited. Specifically, the system divides the entire dialogue process into multiple functional modules, including the user-side acquisition module, audio conversion module, voice recognition processing module, voice synthesis module and server-side intention recognition and answer module. Among them, the acquisition module is responsible for obtaining user voice commands, converting them into audio data, and providing original input for subsequent processing. The audio conversion module intelligently determines and switches the audio data transmission method based on real-time data network signal strength monitoring. When the transmission method is communication signal transmission, it encodes the audio data using a communication streaming protocol to generate an audio stream. This effectively reduces reliance on the data network and ensures continuous and stable communication even in areas with poor signal quality. This significantly expands the system's application scenarios, allowing users to maintain smooth conversations with the intelligent agent, whether in remote mountainous areas, underground parking lots, or in vehicles traveling at high speeds. After receiving the audio stream, the speech recognition processing module uses deep learning technology to perform real-time analysis, extracting key features of the audio signal and converting it into text. Simultaneously, it optimizes the initial analysis results using a language acoustic model to improve the accuracy and coherence of the text. The optimized text is then sent to the server-side intent recognition and response module. Upon receiving the text message, the intent recognition and response module combines contextual information to determine the domain and intent of the text message. Using logical reasoning and language generation algorithms, the module generates a text response that matches the user's intent. This process fully considers the coherence and consistency of the conversation, ensuring the rationality and accuracy of the response. After receiving the answer text, the speech synthesis module converts it into a natural and smooth speech signal. The converted speech signal passes through the audio conversion module again, and is encoded and processed according to the transmission protocol of the communication push stream to generate audio data of the answer audio. The present invention optimizes the communication process and adopts text transmission instead of audio stream, which significantly reduces the amount of data transmission, improves transmission efficiency, and reduces delays and errors caused by network congestion. The close collaboration of various modules gives full play to the advantages of text transmission, intelligent transmission method selection, efficient speech recognition and processing, intelligent intent recognition and answer generation, and optimized speech synthesis and transmission, and builds a stable and efficient intelligent body audio dialogue system, which solves the problem in the prior art that voice interaction is highly dependent on data network signals, resulting in unstable dialogue, and provides users with a smoother, more natural and accurate voice interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1A schematic diagram of the structure of an intelligent agent dialogue system in one embodiment of the present invention;

[0058] Figure 2 Schematic diagram of the flow of the intelligent agent dialogue method in another embodiment of the present invention. DETAILED DESCRIPTION

[0059] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0060] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0061] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to"; the term "based on" means "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0062] It should be noted that the modifications of "one" and "multiple" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0063] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0064] In response to the problems existing in the above-mentioned related technologies, this embodiment provides an intelligent agent dialogue system and method.

[0065] Combine Figure 1As shown, the intelligent agent dialogue system provided by the embodiment of the present invention includes a user end and a server end, and the output of the user end is communicatively connected to the server end; the user end includes an acquisition module, an audio conversion module, a speech recognition processing module and a speech synthesis module that are communicatively connected in sequence; the server end includes an intention recognition and answer module; the output end of the speech recognition processing module of the user end is connected to the input end of the intention recognition and answer module of the server end, and the output end of the intention recognition and answer module is connected to the input end of the speech synthesis module of the user end.

[0066] Specifically, the user-side communication chain consists of a collection module, an audio conversion module, a speech recognition processing module, and a speech synthesis module, which are sequentially connected. The collection module, as the starting point, is responsible for acquiring the user's audio data, providing the raw audio material for subsequent processing. The audio conversion module follows closely behind, playing a key role in determining the transmission method based on the user's data network signal strength. When the data network signal is poor, it can flexibly switch to communication signal transmission. It then encodes the audio data according to the communication streaming protocol to generate an audio stream, ensuring stable transmission of audio data under various network conditions. The speech recognition processing module takes over the audio stream processing task, applying deep learning technology to analyze it in real time, extracting key features of the audio signal and converting it into text information. It also optimizes the initial analysis results using a language acoustic model, improving the accuracy and coherence of the text information. The output of this module is connected to the input of the intent recognition and response module on the server side, achieving seamless integration between the user and server sides. After receiving the text message, the server-side intent recognition and response module combines contextual information to determine the domain and intent of the text message, and then uses logical reasoning and language generation algorithms to generate a text response that matches the user's intent. The output of the intent recognition and answer module is then connected to the input of the speech synthesis module on the user side. The speech synthesis module converts the answer text into a natural and fluent speech signal and generates audio data.

[0067] The acquisition module is used to obtain the user's audio data.

[0068] Specifically, as the starting module on the user side, its function is to obtain the user's audio data, such as from a microphone. This is the front-end of the entire intelligent dialogue system, ensuring that the user's voice information can be accurately collected and providing raw data input for subsequent processing modules. Its performance directly affects the effectiveness and usability of the entire system.

[0069] The audio conversion module is used to set the transmission mode of the audio data according to the data network signal strength of the user terminal. When the transmission mode is communication signal transmission, the audio data is encoded according to the transmission protocol of the communication push stream to obtain the audio stream corresponding to the audio data.

[0070] Specifically, the audio conversion module is located at the user end. First, it monitors the data network signal strength of the user end in real time, such as by calling the application programming interface (API) related to the network interface of the device to obtain the signal strength value. For example, in the Android system, you can use the ConnectivityManager and NetworkInfo classes to obtain the network signal strength. Then, the transmission method is determined according to the set signal strength threshold. When the data network signal strength is lower than the threshold, the communication signal transmission method is selected. The transmission protocol for communication streaming can adopt the Real-time Transport Protocol (RTP) and the like. The audio data is encoded and processed according to the protocol, including packetizing the audio data and adding protocol header information (such as serial number, timestamp, etc.) to ensure the order and synchronization of the audio data in network transmission, thereby obtaining the audio stream corresponding to the audio data, which is convenient for subsequent stable transmission in the communication channel.

[0071] The speech recognition processing module is used to perform real-time analysis based on the audio stream through deep learning to obtain analysis results, and optimize the analysis results through a language acoustic model to obtain text information corresponding to the audio data.

[0072] Specifically, the speech recognition processing module is located on the user end. After receiving the audio stream, it uses deep learning algorithms (such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs)) to parse it in real time. Deep learning models are pre-trained with large amounts of labeled speech data to learn the mapping between speech signals and text. During the parsing process, the model extracts features from the audio stream (such as Mel-Frequency Cepstral Coefficients (MFCCs)) and uses these features to predict the corresponding text content, producing a preliminary parsing result. The parsing result is then optimized using a language acoustic model. This language acoustic model can be based on a statistical language model or a neural network language model. It adjusts and corrects the vocabulary and sentences in the preliminary parsing results based on the language's grammatical rules and collocation habits, filtering out unreasonable content and ultimately obtaining more accurate text information corresponding to the audio data.

[0073] The intention recognition and answer module is used to determine the field and intention of the text information corresponding to the audio data in combination with context information; and obtain the answer text of the audio data based on the field and the intention in combination with the logical reasoning algorithm and the language generation algorithm.

[0074] Specifically, the intent recognition and response module, located on the server side, is responsible for receiving text messages from the speech recognition processing module and further determining the scope and intent of the text messages based on contextual information. Specifically, the scope and intent of the text messages are first determined based on contextual information. Contextual information can include the user's previous conversation content and the conversation context (such as whether the user is driving or in a customer service setting). For example, if the user previously asked about the weather, then the current conversation question, "Is it suitable to wear short sleeves today?" can be used to determine that the scope is weather and the intent is about the relationship between weather conditions and clothing. After determining the scope and intent, logical reasoning algorithms (such as rule-based reasoning algorithms and Bayesian network reasoning algorithms) and language generation algorithms (such as sequence-to-sequence generation models, such as Seq2Seq) are used to generate the corresponding text responses. The logical reasoning algorithm performs logical analysis and deduction on the question, while the language generation algorithm generates natural and fluent text responses based on the deduction results. In addition, if the user end is a device connected via Bluetooth, the answer text is transmitted to the Bluetooth module of the device using the Bluetooth protocol. If it is connected through a mobile data network or Wi-Fi network, the answer text is transmitted through the corresponding network protocol.

[0075] The speech synthesis module is used to perform speech synthesis on the answer text to obtain the audio stream corresponding to the answer text, and encode the audio stream corresponding to the answer text according to the transmission protocol of the communication push stream to obtain the audio data of the answer text.

[0076] Specifically, the speech synthesis module resides on the user side. Speech synthesis can utilize a concatenative synthesis approach, pre-storing a large number of phonemes and a corpus. Based on the content of the response text, appropriate phonemes are selected from the corpus and concatenated to generate the corresponding speech signal. Alternatively, a parametric synthesis approach can be employed, controlling speech synthesizer parameters (such as fundamental frequency and timbre) to generate a speech signal tailored to the response text. The generated speech signal undergoes digital signal processing (such as filtering and gain control) to produce an audio stream corresponding to the response text, tailoring its sound quality, speaking rate, and pitch to the user's listening preferences. This ultimately results in playable response audio data. The speech synthesis module can adjust parameters such as speaking rate, pitch, and timbre of the synthesized speech based on different scenarios and needs, making the synthesized speech more natural, fluent, and emotive, thereby enhancing user acceptance and satisfaction with the agent's response.

[0077] In the intelligent agent dialogue system and method of the present invention, at the user end, after the voice recognition processing module converts the audio stream into text information, only the text information is transmitted to the server end. Since the amount of text data is much smaller than the amount of audio data, the data load during the transmission process is greatly reduced. This lightweight transmission method reduces the demand for data network bandwidth, so that the system can still communicate efficiently when the data network signal is unstable or the bandwidth is limited. Specifically, the system divides the entire dialogue process into multiple functional modules, including the user-side acquisition module, audio conversion module, voice recognition processing module, voice synthesis module and server-side intention recognition and answer module. Among them, the acquisition module is responsible for obtaining user voice commands, converting them into audio data, and providing original input for subsequent processing. The audio conversion module intelligently determines and switches the audio data transmission method based on real-time data network signal strength monitoring. When the transmission method is communication signal transmission, it encodes the audio data using a communication streaming protocol to generate an audio stream. This effectively reduces reliance on the data network and ensures continuous and stable communication even in areas with poor signal quality. This significantly expands the system's application scenarios, allowing users to maintain smooth conversations with the intelligent agent, whether in remote mountainous areas, underground parking lots, or in vehicles traveling at high speeds. After receiving the audio stream, the speech recognition processing module uses deep learning technology to perform real-time analysis, extracting key features of the audio signal and converting it into text. Simultaneously, it optimizes the initial analysis results using a language acoustic model to improve the accuracy and coherence of the text. The optimized text is then sent to the server-side intent recognition and response module. Upon receiving the text message, the intent recognition and response module combines contextual information to determine the domain and intent of the text message. Using logical reasoning and language generation algorithms, the module generates a text response that matches the user's intent. This process fully considers the coherence and consistency of the conversation, ensuring the rationality and accuracy of the response. After receiving the answer text, the speech synthesis module converts it into a natural and smooth speech signal. The converted speech signal passes through the audio conversion module again, and is encoded and processed according to the transmission protocol of the communication push stream to generate audio data of the answer audio. The present invention optimizes the communication process and adopts text transmission instead of audio stream, which significantly reduces the amount of data transmission, improves transmission efficiency, and reduces delays and errors caused by network congestion. The close collaboration of various modules gives full play to the advantages of text transmission, intelligent transmission method selection, efficient speech recognition and processing, intelligent intent recognition and answer generation, and optimized speech synthesis and transmission, and builds a stable and efficient intelligent body audio dialogue system, which solves the problem in the prior art that voice interaction is highly dependent on data network signals, resulting in unstable dialogue, and provides users with a smoother, more natural and accurate voice interaction experience.

[0078] Optionally, the audio conversion module is specifically configured to:

[0079] Obtaining the data network signal strength of the user terminal;

[0080] Setting the transmission mode of the audio data according to a magnitude relationship between the data network signal strength and a preset signal strength threshold;

[0081] Wherein, when the data network signal strength is less than the preset signal strength threshold, the transmission mode of the audio data is set to the communication signal transmission;

[0082] When the data network signal strength is greater than or equal to the preset signal strength threshold, the transmission mode of the audio data is set to data network transmission.

[0083] Specifically, on a user's smart device (such as a smartphone or tablet), the audio conversion module obtains data network signal strength by calling the application programming interface (API) provided by the device's operating system. For example, in Android, the TelephonyManager class can be used to obtain signal strength information. This class provides methods for obtaining mobile signal strength, enabling real-time access to the signal strength of the data network (such as 4G or 5G) to which the device is currently connected. This is typically measured using parameters such as the Received Signal Strength Indicator (RSSI). To accurately reflect changes in network signal status, the audio conversion module periodically obtains data network signal strength (for example, every 1-2 seconds) or immediately obtains updated signal strength upon detecting potential changes in network status (such as when the user switches networks or enters or leaves coverage). This ensures that the system can quickly detect changes in network signal strength in varying network environments.

[0084] In the client-side software configuration file of the intelligent dialogue system, a signal strength threshold is determined based on extensive network test data and actual user experience. Generally, this threshold is set at a signal strength level that ensures that audio data transmitted over the data network will experience minimal interruptions or delays that affect normal conversation. For example, for 4G networks, the threshold is set at a certain RSSI value (e.g., around 85dBm). When the signal strength is above this value, the data network transmission quality is considered good, and normal audio data transmission is possible. When the signal strength is below this value, the data network may be unstable, and a switch to a communication signal transmission method is necessary. It is worth noting that communication signal transmission primarily relies on traditional telecommunications network infrastructure, with common implementation methods including circuit switching and packet switching. Circuit switching technologies, such as the public switched telephone network, establish a dedicated physical circuit between the communicating parties, ensuring stable transmission of voice signals. Data network transmission primarily relies on the internet, using network protocols such as the Real-Time Transport Protocol and the Real-Time Messaging Protocol to efficiently transmit audio data. The advantage of data network transmission is that it can fully utilize existing internet infrastructure to provide efficient and flexible communication services, making it particularly suitable for use in scenarios with good network conditions.

[0085] When the acquired data network signal strength is greater than or equal to a preset signal strength threshold, the audio conversion module sets the audio data transmission mode to data network transmission. In this case, the audio data can be sent directly over existing data network channels (such as the user's mobile data network connection). For example, if the user is in an area with good 5G network coverage and strong signal strength, the audio data can leverage the 5G network's high bandwidth and low latency to quickly transmit from the user end to the server end. The processed audio data from the server end can also be quickly returned to the user end via the 5G network. During data network transmission, the audio data is encapsulated and transmitted according to common network data transmission protocols (such as TCP / IP). When the data network signal strength falls below the preset signal strength threshold, the audio conversion module switches the audio data transmission mode to communication signal transmission. Communication signal transmission generally utilizes traditional communication networks (such as circuit-switched networks). For example, in the case of mobile phones, this involves using the communication channels that their voice call function relies on to transmit audio data. In implementation, the audio data is encoded according to the communication streaming transmission protocol. This encoding process is typically achieved using specific speech coding algorithms (such as AMR, NB / WB, etc.). These algorithms compress the audio data into an audio stream format suitable for transmission over a communication channel while ensuring acceptable audio quality. The device's communication module (such as a mobile phone's baseband processor) then transmits the encoded audio stream over the communication network.

[0086] In an embodiment of the present invention, by flexibly selecting a transmission mode based on the data network signal strength, audio data transmission can be effectively prevented from being interrupted, stuck, or lost due to poor data network signal strength. For example, when a user is in a basement or a corner of a building with poor data network coverage, the system automatically switches to a communication signal transmission mode, utilizing the communication signal to ensure stable transmission of audio data. Compared to traditional methods that rely solely on the data network for voice interaction, this method greatly improves the stability of audio conversations, allowing users to maintain smooth conversations with intelligent agents in various complex network environments. When the data network signal is poor, communication signal transmission, as a relatively stable transmission mode, can ensure that audio data is sent intact from the user end to the server end, and that the processed audio data of the server end is also returned intact to the user end. This can avoid incomplete conversation content or misunderstanding of user intent due to data loss, thereby improving the reliability of the intelligent agent dialogue system and user experience. When the data network signal is good, using the data network to transmit audio data can save communication signal resources. This intelligent switching mechanism also helps reduce data traffic consumption when using data networks, as audio data is only transmitted over the data network when the signal is strong enough. This avoids the waste of data traffic caused by repeated retransmissions to ensure transmission quality when the signal is poor. For telecom operators, this also helps balance the load on data and communication networks, improving resource utilization across the entire communication system.

[0087] Optionally, the speech recognition processing module is specifically configured to:

[0088] Performing frame processing on the audio stream to divide the continuous audio signal in the audio stream into audio frames of multiple time periods;

[0089] Extracting acoustic features from each of the audio frames to obtain acoustic features of the audio frame;

[0090] Inputting the acoustic features into a deep learning model for nonlinear mapping to generate preliminary speech recognition results;

[0091] The preliminary speech recognition result is subjected to grammatical and semantic constraints by means of the language acoustic model to obtain the text information corresponding to the audio data.

[0092] Specifically, in an intelligent agent dialogue system, the speech recognition processing module is responsible for converting audio streams into text messages. This process is crucial to the efficiency and stability of the entire system. The module first performs frame processing on the audio stream, dividing the continuous audio signal into multiple audio frames. Each frame is typically 20-30 milliseconds long, depending on the short-term characteristics of the speech signal, which is considered stationary over a short period of time. For an audio stream with a 16kHz sampling rate, for example, 320-480 samples constitute an audio frame. In software, frame processing can be achieved by looping through the audio stream data and storing it in a buffer, extracting audio frames once the data volume reaches a certain level. In hardware, dedicated audio processing chips can also directly process audio signals by frame. Extracting acoustic features from each audio frame is a key step in speech recognition. Common acoustic features include Mel-frequency cepstral coefficients (MFCCs), filter bank energy, and linear prediction cepstral coefficients (LPCCs). Taking MFCCs as an example, the extraction process primarily involves pre-emphasis, frame windowing, fast Fourier transform (FFT), Mel filter bank processing, and discrete cosine transform (DCT). Pre-emphasis emphasizes high frequencies and flattens the spectrum. Frame windowing (such as a Hamming window) reduces spectral leakage. FFT converts time-domain signals into frequency-domain signals, typically with 256 or 512 points. Mel filter banks process the spectrum, simulating the characteristics of human hearing. DCT extracts MFCC coefficients from the filter bank output, with the first 12-13 coefficients serving as acoustic features. Deep learning models, such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), and long short-term memory networks (LSTMs), are used to map acoustic features to speech text. During training, the model is trained using large amounts of text-labeled speech data. For example, models can be built using frameworks like TensorFlow or PyTorch. LSTMs are well-suited to processing the temporal nature of speech signals. Their input is a sequence of acoustic features from audio frames, and their output is a conditional probability distribution over the corresponding word at each time step. In practice, the extracted acoustic features are fed into the trained model, which then outputs preliminary speech recognition results, including a sequence of word candidates and their probabilities. The initial speech recognition results are then fed into a language acoustic model for grammatical and semantic constraints, optimizing them into text messages that conform to language norms and semantic logic. This model, built on statistical or neural network language models, adjusts the initial results based on grammatical rules and vocabulary collocation habits. For example, it replaces ungrammatical word combinations to ensure the accuracy of the output text.

[0093] In embodiments of the present invention, framing enables better analysis of speech signals, and acoustic feature extraction can extract important information that effectively characterizes speech characteristics. Appropriate framing methods and acoustic feature extraction algorithms can improve the ability to distinguish speech signals. For example, MFCC features can effectively reflect speech characteristics such as timbre and pitch, making homophone distinction more accurate. Accurate acoustic features fed into a deep learning model can improve the model's recognition performance. Deep learning models can fully exploit the underlying mapping patterns between acoustic features and text, enabling more accurate conversion of acoustic features into text compared to traditional rule-based speech recognition methods. The language acoustic model uses grammatical and semantic constraints to correct errors in preliminary speech recognition results. For example, if the preliminary recognition result is an ungrammatical sentence, the language acoustic model will select a more appropriate word from possible alternatives based on grammatical rules and semantic knowledge to replace the incorrect word, thereby obtaining more accurate text information. This can effectively reduce the word error rate in speech recognition and improve overall recognition accuracy. Furthermore, efficient acoustic feature extraction algorithms can extract useful features in a shorter time, reducing the computational complexity of the entire speech recognition process. Optimized deep learning models can process input acoustic features and output preliminary speech recognition results in a relatively short time, improving the real-time performance of speech recognition. For example, voice assistant applications on some smartphones can complete speech recognition and provide a response within seconds of the user speaking. This relies heavily on the efficient reasoning capabilities of deep learning models. By segmenting continuous audio signals into audio frames and extracting acoustic features, the speech recognition processing module effectively reduces data volume. Compared to directly transmitting raw audio data, the amount of text information is significantly reduced, reducing network bandwidth requirements, minimizing transmission delays and errors, and improving system stability.

[0094] Optionally, the intention recognition and answering module is specifically configured to:

[0095] Performing word segmentation and part-of-speech tagging on the text information to obtain the basic structure and grammatical features of the text information;

[0096] Using an intent classification model, based on the basic structure and the grammatical features, combined with contextual information, semantically understand and classify the text information to obtain the domain and intent of the text information;

[0097] According to the domain and the intention, combined with the logical reasoning algorithm, generate a logical reasoning result corresponding to the domain and the intention;

[0098] Then, the language generation algorithm is used to generate the answer text according to the logical reasoning result.

[0099] Specifically, using a pre-constructed dictionary, continuous text information is segmented into a sequence of words or phrases through string matching. For example, for Chinese word segmentation, the longest matching method, the forward maximum matching method, etc. can be used. Taking the forward maximum matching method as an example, starting from the beginning of the sentence, the longest matching word is searched in the dictionary in sequence until the end of the sentence. For example, for the sentence "I love natural language processing", using the forward maximum matching method and matching according to the words in the dictionary, it may first match "I", then "love", then "natural language" (assuming this word exists in the dictionary), and finally "processing". Automatically segment words or phrases by training a statistical model. Such as using the Hidden Markov Model (HMM) or Conditional Random Field (CRF), etc. During the training process, the model learns the occurrence probability of words and the relationship between contexts. When segmenting words, the boundaries of words or phrases are determined based on these probability relationships. For example, the CRF model can consider the information of the adjacent words on the left and right of each character to determine whether this character belongs to the previous word or the next word. Label the segmented words with their corresponding词性 according to the词性标注 and grammar rules of each word in the dictionary. For example, in a simple rule dictionary, "I" is labeled as a pronoun, "love" is labeled as a verb, and "natural language processing" is labeled as a noun. According to the grammar rules, when a word appears in different positions in a sentence, its词性 is determined based on its collocation relationship and sentence structure. For example, when "natural language processing" is used as the object of "love" in a sentence, its词性 remains a noun.

[0100] Train a machine learning model (such as HMM, CRF, etc.) using the training corpus labeled with词性. The model learns the词性 probability of each word in different context environments. During actual annotation, label the词性 based on the context of the sentence and the probability calculated by the model. For example, in the sentence "He plays basketball", "plays" is labeled as a verb, while in "fetch water", "fetch" will be labeled as a verb or other词性 according to the context, depending on the learning result of the model.

[0101] It should be noted that there is an unclear term "词性" in the original text which might need further clarification in the source language context for a more accurate translation. Here it's translated as "词性" first as it's not clear what it exactly means in this context.Taking convolutional neural networks (CNNs) as an example, text features are automatically extracted. During training, text information after word segmentation and part-of-speech tagging is converted into a sequence of word vectors (which can be obtained using pre-trained word vector models such as Word2Vec and GloVe). This is then input into the CNN model. The convolutional layers in the CNN extract local features, the pooling layers reduce the dimensionality of the features, and finally, the fully connected layers output the intent classification results. For example, for intent classification in a customer service scenario, the model can learn the features corresponding to different expressions for intents such as consultation, complaint, and suggestion. A large amount of intent-annotated training data is prepared. This data includes text information from various domains and their corresponding domain and intent labels. The training data is input into the intent classification model, and the model parameters (such as weights and biases) are adjusted to minimize the difference between predicted intent and actual intent (for example, using a cross-entropy loss function). After multiple rounds of iterative training, the model learns how to accurately classify intent based on the structural features of the text (such as the basic structure and grammatical features obtained after word segmentation and part-of-speech tagging) and contextual information. In a dialogue system, multiple rounds of conversation between the user and the agent are recorded. For example, a queue or list can be used to store the text, intent, and domain of previous conversations. When semantic understanding and classification of the current text is needed, the previous conversation can be referenced to better determine its domain and intent. In the intent classification model, the current text can be concatenated or fused with contextual information. For example, the word vector sequence of the current sentence and the word vector sequence of the previous sentence can be concatenated and input into the model. Alternatively, an attention mechanism can be used to focus the model on contextual components relevant to the current intent. For example, in a multi-turn conversation, if the user previously mentioned a travel consultation, the model will be more likely to prioritize travel-related content in subsequent conversations to determine intent. In domains with clear rules, reasoning can be performed based on predefined knowledge rules. For example, in mathematical calculations, the four arithmetic operations can be used to derive the answer. If the intent of the text is to evaluate a mathematical expression, a logical reasoning algorithm can be used to calculate and generate a result based on the operation rules. Bayesian networks can represent probabilistic relationships between variables. After determining the domain and intent of the text, the corresponding reasoning method is invoked from the corresponding logical reasoning rule library or model library. For example, in the field of travel planning, if the query is about the best route, a logical reasoning algorithm can plan a route based on rules such as travel time, location, and distance between attractions. Inference rules or models are used to reason about input information. For example, in the field of question answering, if the query is about the time of a certain event, a logical reasoning algorithm can find the corresponding time based on the event records in the existing knowledge base and generate a result.

[0102] In some domains, predefined answer templates are available. For example, in the weather query domain, a template for answering weather inquiries can be designed, such as "Today's weather at [location] is [weather conditions], and the temperature range is [temperature range]." Specific information from the logical reasoning results (such as location, weather conditions, and temperature) is added to the template to generate a textual answer. For example, a Seq2Seq model consists of an encoder and a decoder. The encoder encodes the input logical reasoning results and other relevant information (such as context) into a fixed-length vector, and the decoder generates a textual answer based on this vector. During training, the model is trained using a large number of question-answer pairs, learning how to generate appropriate responses based on different inputs. The logical reasoning results, along with any necessary contextual information, are provided as input to the language generation algorithm. For example, in a multi-turn conversation, in addition to the logical reasoning results, previous conversation content is also required to generate a coherent response. Based on this input, the language generation algorithm generates a textual answer according to its trained model structure and parameters. For example, in a deep learning-based model, the decoder generates each word of the answer step by step until a complete sentence is generated as the answer.

[0103] In embodiments of the present invention, word segmentation and part-of-speech tagging can clearly analyze the basic structure and grammatical features of text information, providing an accurate foundation for subsequent semantic understanding. Intent classification models combine these structural features with contextual information to more accurately understand the semantics of text information, avoiding erroneous intent judgments due to grammatical or semantic misunderstandings. Considering contextual information allows intent recognition to take into account the coherence and overall semantics of the conversation. For example, in multi-turn conversations, users may omit some information. By integrating contextual information, the intent classification model can accurately determine the intent of the current text message based on the content of previous conversations. For example, if a user first asks "What are the attractions in Beijing?" and then asks "What are the ticket prices for these attractions?" the model can determine from the context that "these attractions" refers to attractions in Beijing, accurately identifying the intent of the second question as an inquiry about ticket prices for attractions in Beijing. Rule-based logical reasoning algorithms can provide accurate and reliable inference results in domains with clear rules, while probabilistic reasoning algorithms such as Bayesian networks can provide reasonable inference results in domains with uncertainty. Template-based language generation algorithms can ensure the accuracy of the format and content of answers, especially in structured answer scenarios such as information queries. Deep learning-based language generation algorithms can generate more natural and flexible responses, producing coherent responses tailored to user needs based on varying contexts. For example, for an open-ended question about a travel experience, a deep learning-based language generation algorithm can generate rich, engaging responses based on logical reasoning (such as the characteristics of the travel destination and the user's interests and preferences), thereby increasing user satisfaction. Furthermore, text-based responses interact with the user, reducing data volume and improving transmission efficiency and stability, ensuring the coherence and accuracy of multiple rounds of dialogue.

[0104] Optionally, the speech synthesis module is specifically configured to:

[0105] Converting the answer text into a speech signal according to preset acoustic parameters using a speech synthesis model;

[0106] Performing noise suppression and echo cancellation processing on the speech signal to obtain the processed speech signal;

[0107] The processed voice signal is encoded according to the transmission protocol of the communication push stream to generate the audio stream corresponding to the answer text.

[0108] Specifically, common speech synthesis models include concatenative synthesis, parametric synthesis, and deep learning-based methods (such as Tacotron and WaveNet). Deep learning-based models currently offer superior speech synthesis quality. For example, the Tacotron model is an end-to-end speech synthesis system that directly converts text into speech spectral features. These features are then converted into raw speech waveforms using a vocoder such as WaveNet. The model is trained using a large amount of text-annotated speech data, typically including speech segments from different speakers at varying speeds and intonations. Preset acoustic parameters include the speaker's acoustic characteristics (such as pitch frequency and timbre), speaking speed, and intonation. During training, the model learns how to map text to a speech signal that meets these preset acoustic parameters. For example, the quality of synthesized speech can be improved by adjusting model hyperparameters (such as the learning rate and hidden layer size) and optimization objectives (such as minimizing the difference between generated and real speech). The response text is preprocessed, including text normalization (e.g., converting numbers and special symbols to text) and word segmentation. For example, "20%" is converted to "twenty percent" to help the speech synthesis model better understand the text. The preprocessed text is input into the trained speech synthesis model. The model generates the corresponding speech signal based on preset acoustic parameters. For example, during the generation process, the model controls the playback speed of the speech according to preset speech rate parameters and generates a speech signal with appropriate intonation according to preset intonation parameters.

[0109] To perform noise suppression and echo cancellation on speech signals, the speech signal is first subjected to a short-time Fourier transform (STFT) to convert the time domain signal into a spectrum. In the spectral domain, the noise power spectrum is estimated. The estimated noise power spectrum is then subtracted from the speech signal's power spectrum to obtain a relatively clean speech spectrum. For example, a spectrum of clean speech in a quiet environment is obtained as a reference. If the speech signal contains noise, the noise spectrum is subtracted to suppress the noise. Finally, the processed spectrum is converted back to a time domain signal using an inverse short-time Fourier transform (ISTR). Deep learning models (such as LSTM and CRNN) are used to suppress noise. The model input is the spectrum of a noisy speech signal, and the output is the spectrum of a clean speech signal. During training, the model is trained using a large amount of noisy speech and the corresponding clean speech data. For example, a trained deep learning noise suppression model can effectively remove noise and improve speech quality even when the speech signal is disturbed by background music, ambient noise, or other interference. Echoes are typically caused by the speech signal emitted by a speaker being picked up by a microphone. Adaptive filters can simulate the echo signal received by the microphone. By continuously adjusting the parameters of the adaptive filter, its output signal is made as close as possible to the actual echo signal. The adaptive filter's output signal is then subtracted from the echo-containing speech signal received by the microphone to produce a relatively pure speech signal. For example, in a full-duplex communication system, the adaptive filter can adjust the filter coefficients in real time based on the speech signal played by the speaker and the signal received by the microphone, effectively eliminating echoes. Similar to noise suppression, echo cancellation can be achieved using a deep learning model. The model input is an echo-containing speech signal, and the output is the speech signal after echo removal. Through extensive data training, the model learns the difference between echo and pure speech, effectively removing echoes.

[0110] Common audio coding formats include PCM (Pulse Code Modulation), MP3, and Opus. In speech synthesis systems, efficient coding formats are typically selected, taking into account the characteristics of voice signals and transmission efficiency. For example, Opus coding offers low latency and a high compression ratio, making it suitable for real-time voice communication. It reduces voice data size and improves transmission efficiency while maintaining voice quality. After noise suppression and echo cancellation, the voice signal is encoded according to the specifications of the selected coding format. For example, in Opus coding, the voice signal is divided into small blocks, each of which undergoes spectral analysis and quantization. The quantized data is then encapsulated to generate an audio stream conforming to the Opus format. Based on the actual communication environment and requirements, appropriate streaming protocols, such as RTP (Real-time Transport Protocol) and RTMP (Real-time Messaging Protocol), are selected. For example, RTP, a network control protocol, is used to transmit real-time data, such as audio and video, over existing networks. Before sending the audio stream to the user end, the audio stream will be packaged according to the RTP protocol and necessary header information (such as sequence number, timestamp, etc.) will be added so that the receiving end can correctly decode and play the audio signal.

[0111] In this embodiment of the present invention, a speech synthesis module converts text responses into speech signals and efficiently transmits them, significantly improving the stability and user experience of the intelligent dialogue system. Speech synthesis models, such as Tacotron and WaveNet, based on deep learning, efficiently convert text into speech signals. Model training utilizes a large amount of text-annotated speech data, covering a variety of speakers, speaking rates, and intonations, to generate natural speech. Preset acoustic parameters ensure that the timbre and intonation of the speech signal meet user expectations, improving speech quality. During the speech synthesis process, the system preprocesses the response text, including text normalization and word segmentation, to help the model better understand the text content. The generated speech signal undergoes noise suppression and echo cancellation to enhance speech clarity and intelligibility. For example, the Short-Time Fourier Transform (STFT) converts the time-domain signal into a frequency spectrum, estimating and subtracting the noise power spectrum. Deep learning models can also effectively remove background noise and echoes. The processed speech signal is encoded according to a communication streaming protocol to generate an audio stream corresponding to the response text. The system uses efficient audio coding formats such as Opus to significantly reduce the size of the speech data and improve transmission efficiency. Combined with text transmission, the system reduces data volume and network bandwidth requirements, minimizing latency and error rates, ensuring smooth conversations. In low-bandwidth or unstable network environments, efficient encoding formats and protocols ensure stable transmission of voice signals, avoiding interruptions or delays. This text transmission method optimizes the communication process, reduces data volume, and improves transmission efficiency and system stability. Through the efficient processing and transmission mechanisms of the speech synthesis module, users can receive the agent's responses promptly and clearly, enhancing the naturalness and fluency of conversations and providing users with a more stable and efficient agent conversation experience.

[0112] Optionally, the audio conversion module is further configured to:

[0113] When the transmission mode is the data network transmission, the audio data is encoded according to the User Datagram Protocol to obtain network transmission data of the audio data.

[0114] Specifically, the audio conversion module receives raw audio data from an audio capture device (such as a microphone) or an audio processing module. This data is typically in PCM (Pulse-Code Modulation) format. Preprocessing the PCM data involves operations such as removing silence segments, adjusting gain, and converting audio formats to optimize audio quality and reduce unnecessary data transmission. The processed PCM data is segmented into blocks of appropriate size, typically around 1000 bytes per block, to accommodate network transmission requirements. A UDP protocol header is added to each block. The UDP header includes fields such as the source port, destination port, data length, and checksum; this information ensures proper routing and integrity checking of the data within the network. The audio data is encoded using a specific audio coding algorithm (such as AAC or MP3) to reduce data size and improve transmission efficiency. The selection of the coding algorithm should comprehensively consider factors such as audio quality, compression ratio, and computational complexity. The encoded audio data and UDP header information are combined to form complete UDP datagrams, which represent the network transmission data for the audio data and can be sent to the designated destination. The intent recognition and response module processes the decoded audio data, extracting semantic information and identifying the user's intent and needs. Based on the recognition results, the module generates a corresponding response. Responses are typically in text, as text facilitates semantic generation and logical reasoning, and can more accurately express the content and logical structure of the response.

[0115] In this embodiment of the present invention, UDP, as a connectionless transmission protocol, offers low latency and can quickly transmit audio data. This enables real-time transmission of audio signals across the network, ensuring smooth conversations. The UDP protocol header overhead is minimal, at only 8 bytes, making it more efficient in utilizing network bandwidth compared to other protocols such as TCP. Furthermore, the audio encoding algorithm can further reduce data volume and improve transmission efficiency. The audio conversion module effectively converts audio data into network transmission data suitable for communication signal transmission, improving the stability and efficiency of audio conversations.

[0116] Optionally, the speech synthesis module is specifically configured to:

[0117] When the transmission mode of the audio data is the data network transmission, determining the speech rate parameter, intonation parameter, volume parameter and timbre parameter of the speech synthesis model according to the data network signal strength;

[0118] Optimizing preset acoustic parameters of a speech synthesis model according to the speech rate parameter, the intonation parameter, the volume parameter, and the timbre parameter to obtain the optimized speech synthesis model;

[0119] The answer text is dynamically speech synthesized by using the optimized speech synthesis model in combination with forward error correction and redundant coding strategies to obtain the audio stream corresponding to the answer text.

[0120] Specifically, the data network signal strength is obtained in real time through the device's network interface, typically measured as parameters such as the RSSI (Received Signal Strength Indicator) or Received Signal Strength (RSS). These values ​​can be obtained through the operating system's API or network driver. The data network signal strength is compared against a preset strength range and mapped to the speech synthesis model's speaking rate, intonation, volume, and timbre parameters. For example, when the signal strength is high, indicating good network conditions and a quiet and stable user environment, the speaking rate can be increased to achieve smoother speech synthesis. At the same time, the volume can be reduced, the intonation can be stabilized, and the timbre can be softened. Conversely, when the signal strength is low, indicating a noisy or unstable network environment, the speaking rate can be reduced, the volume increased, the intonation clearer, and the timbre brighter to ensure the user can clearly hear the synthesized speech.

[0121] Based on the mapping results, the speech synthesis model's speech rate parameters are adjusted. During speech synthesis, different speech rates are achieved by controlling the playback speed of the speech signal. For example, in deep learning-based speech synthesis models, the model's hyperparameters for controlling speech rate can be adjusted, or the speech signal's time axis can be compressed or expanded during speech generation to change the speech rate. Adjustment of intonation parameters can be achieved by changing the fundamental frequency of the speech signal. In speech synthesis models, the fundamental frequency determines the pitch of speech, thus affecting intonation. Intonation can be altered by adjusting parameters related to the fundamental frequency in the model or by modifying the fundamental frequency after speech generation. For example, the fundamental frequency of a speech signal can be shifted up or down in the frequency domain to achieve different intonation effects. Adjustment of volume parameters primarily controls the amplitude of the speech signal. In speech synthesis models, the volume can be altered by adjusting the amplitude of the model output. Alternatively, the volume can be adjusted by amplifying or reducing the speech signal after speech generation. Adjustment of timbre parameters involves the spectral characteristics of the speech signal. The timbre can be changed by changing the spectrum-related parameters in the speech synthesis model or by filtering the spectrum after speech generation. For example, different vocoders or filters can be used to shape different timbres.

[0122] According to the above-mentioned adjusted speech rate, intonation, volume and timbre parameters, the preset acoustic parameters of the speech synthesis model are updated. Within the model, these parameters may affect various aspects of speech generation, such as spectrum generation, fundamental frequency control, amplitude adjustment, etc. of the speech signal. For example, in the Tacotron-2 speech synthesis model, the speech rate, intonation, volume and timbre can be optimized by adjusting the relevant parameters such as the mel spectrum generation module and the fundamental frequency prediction module in the model. In a preferred embodiment, a small amount of adaptive data is also used to further train the optimized model to better adapt to the current network conditions and user needs. These adaptive data can be some speech data collected in a similar network environment. By fine-tuning the parameters of the model, the model can better generate speech signals that meet the requirements.

[0123] During the speech synthesis process, forward error correction (FEC) coding is used to add redundant information to the generated speech data. For example, using coding schemes such as Reed-Solomon codes allows the receiver to recover the original speech data based on the redundant information, even if some data is lost or damaged during speech data transmission. In the speech synthesis module, during the speech signal encoding stage, the speech data is divided into multiple data blocks. A corresponding error correction code is then generated for each block and appended to the end of the block. In addition to FEC, redundant coding strategies can also be employed, whereby important speech data is encoded multiple times or backed up. For example, key frames or important feature parameters in the speech signal can be redundantly encoded. This allows the receiver to reconstruct the speech signal based on the redundant data, even if some data is lost during transmission. In implementation, the level and method of redundant coding can be determined based on the data network signal strength and the importance of the speech data. The optimized speech synthesis model is used to convert the text response into a speech signal. During the speech synthesis process, the speech data is processed based on FEC and redundant coding strategies. For example, when generating a voice data stream, error correction codes and redundant data are added according to predetermined coding rules. The processed voice data is then packaged according to the communication push transmission protocol to generate an audio stream corresponding to the answer text, ready to be sent to the user end.

[0124] In this embodiment of the present invention, the parameters of the speech synthesis model are dynamically adjusted based on the data network signal strength, allowing speech synthesis to adapt to different network environments and user scenarios. In environments with strong signals and stable networks, this provides smoother and more natural speech synthesis; in environments with weak signals and unstable networks, it ensures clear and understandable speech synthesis, meeting user needs in various situations and improving the adaptability and reliability of the intelligent dialogue system. The adjusted speech rate, intonation, volume, and timbre parameters can better match user auditory needs and usage scenarios. For example, in noisy environments, the volume and clarity are increased, making it easier for users to hear the speech synthesis content; in quiet environments, the speech is softer and more natural, improving user auditory comfort. This optimizes the user experience and makes users more willing to use the intelligent dialogue system. The optimized speech synthesis model can generate speech signals that better meet current network conditions and user needs. Forward error correction and redundant coding strategies effectively mitigate voice data loss and corruption during transmission, ensuring the integrity and accuracy of the voice signal. For example, in the event of network fluctuations or packet loss, the receiver can use error-correcting codes and redundant data to restore the original speech signal, reducing speech interruptions or distortion caused by data loss and improving the overall quality of speech synthesis. Forward error correction and redundant coding strategies increase the fault tolerance of speech data and reduce the requirements for network transmission reliability. Even in poor network conditions, they ensure stable transmission of speech data, reducing conversation interruptions or delays caused by network issues and improving the stability and availability of the intelligent dialogue system.

[0125] Optionally, the intention recognition and answering module is specifically configured to:

[0126] determining network fluctuation data of the user terminal according to the data network signal strength;

[0127] Predicting based on the network fluctuation data to obtain future network fluctuation data during the transmission of the answer text;

[0128] The answer text is sent to the user terminal according to the future network fluctuation data.

[0129] Specifically, the data network signal strength is continuously monitored and its changes over time are recorded. In addition to signal strength, other network-related indicators are also collected, such as packet loss rate and latency. This data can be obtained through the device's network interface API or network diagnostic tools. The monitoring period is divided into multiple small time windows, for example, each 10 seconds as a window. Statistical indicators such as the signal strength average, variance, packet loss rate, and latency within each window are calculated. These indicators constitute network fluctuation data. The rate of change of signal strength between different time windows is calculated. For example, the difference in signal strength between adjacent time windows is divided by the signal strength of the previous window to obtain the relative rate of change of signal strength. At the same time, the variance of signal strength is calculated to measure the stability of signal strength. The packet loss rate and latency changes within each time window are statistically analyzed. For example, statistical indicators such as the standard deviation of packet loss rate and the maximum, minimum, and average values ​​of latency are calculated to fully reflect network fluctuations.

[0130] Select an appropriate time series forecasting model, such as the Autoregressive Integrated Moving Average (ARIMA) model, exponential smoothing, or Long Short-Term Memory (LSTM) network. For example, LSTM can handle long- and short-term dependencies in time series data and is suitable for predicting network fluctuation data. Use collected historical network fluctuation data to train the model. Divide the historical data into training and test sets. Adjust model parameters (such as the size of the LSTM hidden layer and the learning rate) to minimize the difference between predicted and actual values, using loss functions such as mean squared error (MSE). Use the latest network fluctuation data (including statistical indicators such as signal strength, packet loss rate, and latency) as model input. Perform preprocessing operations such as normalization on the input data to ensure it meets the model's input requirements. Based on the input data, the model predicts network fluctuation data for a future period (for example, the next 30 seconds), including predicted values ​​for signal strength trends, packet loss rate, and latency. This predicted data will be used to formulate subsequent response text transmission plans.

[0131] In this embodiment of the present invention, by predicting future network fluctuations and formulating a delivery plan accordingly, it is possible to proactively address the impact of network fluctuations on audio transmission. For example, when an increase in network packet loss is predicted, the transmission frequency is reduced and the delivery plan is adjusted, reducing audio interruptions or freezes caused by packet loss. This adaptive delivery strategy improves the transmission stability of text responses in the event of network fluctuations, ensuring that users can receive audio content smoothly.

[0132] Optionally, the intention recognition and answering module is further configured to:

[0133] determining a transmission frequency of the reply text according to the future network fluctuation data, and dividing the reply text into a plurality of text segments;

[0134] generating a sending plan for the reply text according to the transmission frequency of the reply text and the text fragment;

[0135] The text segment of the answer text is sent to the user terminal according to the sending plan.

[0136] Specifically, the transmission frequency of the reply text is determined based on predicted future network fluctuations. If significant network fluctuations are predicted (e.g., decreased signal strength, increased packet loss rate), the transmission frequency is reduced to minimize data loss and retransmissions caused by network fluctuations. Conversely, if network conditions are predicted to be stable, the transmission frequency can be appropriately increased to speed up reply text transmission. For example, the transmission frequency can be dynamically calculated based on the predicted packet loss rate and latency. Assume that under a stable network environment, the base transmission frequency is 20 text segments per second. If the predicted packet loss rate increases by 10%, the transmission frequency is reduced by 10%, resulting in 18 text segments transmitted per second. If the predicted latency increases by more than a certain threshold, the transmission frequency is similarly reduced. Based on the determined transmission frequency, the reply text is segmented into multiple text segments. A reply text delivery schedule is generated based on the transmission frequency and the sequence of the text segments. The delivery schedule includes information such as the delivery time, delivery order, and data volume of each text segment. For example, each segment is scheduled to be delivered at the corresponding time according to the order of the segments to ensure the continuity and integrity of the reply text. The delivery schedule is optimized based on the predicted network fluctuations. For example, if significant network fluctuations are predicted within a certain timeframe, key text snippets can be sent in advance, or a buffer can be built into the delivery schedule to account for potential network delays. The audio transmission module then sends the text responses to the audio responses to the user according to the generated delivery schedule. During the delivery process, network conditions are monitored in real time. If the actual network conditions deviate significantly from the predicted results, the delivery schedule can be dynamically adjusted. For example, if the actual network fluctuations are smaller than predicted, the transmission frequency can be appropriately increased to speed up the delivery of the text responses.

[0137] In embodiments of the present invention, the order and timing of text segment transmission are rationally arranged based on network fluctuation predictions, reducing the risk of response text loss during transmission. For example, when network conditions are poor, key text segments are prioritized for transmission, or the transmission frequency and buffering time are adjusted to enable the receiving end to better reconstruct the response text. This reduces audio quality issues caused by data loss and improves the reliability of audio transmission. When network conditions are stable, the transmission frequency can be appropriately increased based on the prediction results to accelerate the transmission of the response text. For example, when future network fluctuations are predicted to be minimal and bandwidth is sufficient, the number of text segments sent per second is increased, allowing users to receive the complete audio content more quickly, shortening user wait times and improving the responsiveness of the agent dialogue system. Dynamic adjustment of the transmission frequency and sending schedule enables more efficient utilization of network resources. Under different network environments, the response text sending strategy is adjusted based on actual conditions, avoiding excessive network bandwidth consumption when network conditions are good and resource waste caused by blind transmission during network fluctuations. This optimization strategy improves the efficiency of network resource utilization and makes conversations with the agent smoother and more stable.

[0138] Combine Figure 2 As shown, an agent dialogue method of the present invention is applied to the agent dialogue system as described above, wherein the agent dialogue system includes a user end and a server end, and the output of the user end is connected to the server end in communication;

[0139] The user end includes a collection module, an audio conversion module, a speech recognition processing module and a speech synthesis module which are connected in sequence; the server end includes an intention recognition and answering module;

[0140] The output end of the speech recognition processing module of the user end is connected to the input end of the intention recognition and answering module of the server end, and the output end of the intention recognition and answering module is connected to the input end of the speech synthesis module of the user end;

[0141] The agent dialogue method comprises:

[0142] Acquire the user's audio data through the acquisition module;

[0143] The audio conversion module sets a transmission mode for the audio data according to the data network signal strength of the user terminal. When the transmission mode is communication signal transmission, the audio data is encoded according to the transmission protocol of the communication push stream to obtain an audio stream corresponding to the audio data.

[0144] The speech recognition processing module performs real-time analysis based on the audio stream through deep learning to obtain an analysis result, and optimizes the analysis result through a language acoustic model to obtain text information corresponding to the audio data;

[0145] The intention recognition and answer module determines the domain and intention of the text information according to the text information corresponding to the audio data in combination with context information; and obtains the answer text of the audio data in combination with the logical reasoning algorithm and the language generation algorithm based on the domain and the intention;

[0146] The answer text is speech-synthesized by the speech synthesis module to obtain an audio stream corresponding to the answer text, and the audio stream corresponding to the answer text is encoded according to the transmission protocol of the communication push stream to obtain audio data of the answer text.

[0147] The advantages of the intelligent agent dialogue method of the present invention over the existing technology are the same as the advantages of the above-mentioned intelligent agent dialogue system over the existing technology, and will not be repeated here.

[0148] Although the present invention is disclosed as above, the scope of protection disclosed by the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the scope of protection of the present invention.

Claims

1. An intelligent agent dialogue system, characterized in that: It includes a user side and a server side, and the user side is connected to the server side in communication; The user end includes a collection module, an audio conversion module, a speech recognition processing module and a speech synthesis module which are connected in sequence; the server end includes an intention recognition and answering module; The output end of the speech recognition processing module of the user end is connected to the input end of the intention recognition and answering module of the server end, and the output end of the intention recognition and answering module is connected to the input end of the speech synthesis module of the user end; The acquisition module is used to obtain the user's audio data; The audio conversion module is used to set the transmission mode of the audio data according to the data network signal strength of the user terminal monitored in real time, and the transmission mode includes communication signal transmission and data network transmission; when the transmission mode is communication signal transmission, the audio data is encoded according to the transmission protocol of the communication push stream to obtain an audio stream corresponding to the audio data; The speech recognition processing module is configured to perform real-time analysis of the audio stream through deep learning to obtain an analysis result, and optimize the analysis result through a language acoustic model to obtain text information corresponding to the audio data; when the transmission mode is data network transmission, encode the audio data according to the user datagram protocol to obtain network transmission data of the audio data, and transmit the encoded data to the server; The intention recognition and answering module is used to determine the domain and intention of the text information based on the text information corresponding to the audio data in combination with context information; and obtaining a textual response to the audio data based on the domain and the intention, in combination with a logical reasoning algorithm and a language generation algorithm; The intention recognition and response module is further configured to process the network transmission data, extract semantic information therein, and identify user intentions and needs; generate corresponding response text based on the recognition results; and determine the transmission frequency of the response text based on future network fluctuation data predicted by the data network signal strength; The speech synthesis module is used to perform speech synthesis on the answer text to obtain the audio stream corresponding to the answer text, and encode the audio stream corresponding to the answer text according to the transmission protocol of the communication push stream to obtain the audio data of the answer text; wherein, when the transmission mode of the audio data is the data network transmission, the speech rate parameters, intonation parameters, volume parameters and timbre parameters of the speech synthesis model are determined according to the data network signal strength; the preset acoustic parameters of the speech synthesis model are optimized according to the speech rate parameters, the intonation parameters, the volume parameters and the timbre parameters to obtain the optimized speech synthesis model; through the optimized speech synthesis model, the answer text is dynamically speech synthesized in combination with forward error correction and redundant coding strategies to obtain the audio stream corresponding to the answer text.

2. The agent dialogue system according to claim 1, characterized in that: The audio conversion module is specifically used to: Obtaining the data network signal strength of the user terminal; Setting the transmission mode of the audio data according to a magnitude relationship between the data network signal strength and a preset signal strength threshold; Wherein, when the data network signal strength is less than the preset signal strength threshold, the transmission mode of the audio data is set to the communication signal transmission; When the data network signal strength is greater than or equal to the preset signal strength threshold, the transmission mode of the audio data is set to data network transmission.

3. The agent dialogue system according to claim 1, characterized in that: The speech recognition processing module is specifically used to: Performing frame processing on the audio stream to divide the continuous audio signal in the audio stream into audio frames of multiple time periods; Extracting acoustic features from each of the audio frames to obtain acoustic features of the audio frame; Inputting the acoustic features into a deep learning model for nonlinear mapping to generate preliminary speech recognition results; The preliminary speech recognition result is subjected to grammatical and semantic constraints by means of the language acoustic model to obtain the text information corresponding to the audio data.

4. The agent dialogue system according to claim 1, characterized in that: The intention recognition and answering module is specifically used to: Performing word segmentation and part-of-speech tagging on the text information to obtain the basic structure and grammatical features of the text information; Using an intent classification model, based on the basic structure and the grammatical features, combined with contextual information, semantically understand and classify the text information to obtain the domain and intent of the text information; According to the domain and the intention, combined with the logical reasoning algorithm, generate a logical reasoning result corresponding to the domain and the intention; Then, the language generation algorithm is used to generate the answer text according to the logical reasoning result.

5. The agent dialogue system according to claim 1, characterized in that: The speech synthesis module is specifically used for: Converting the answer text into a speech signal according to preset acoustic parameters using a speech synthesis model; Performing noise suppression and echo cancellation processing on the speech signal to obtain the processed speech signal; The processed voice signal is encoded according to the transmission protocol of the communication push stream to generate the audio stream corresponding to the answer text.

6. The agent dialogue system according to claim 1, characterized in that: The intention recognition and answering module is specifically used to: determining network fluctuation data of the user terminal according to the data network signal strength; Predicting based on the network fluctuation data to obtain future network fluctuation data during the transmission of the answer text; The answer text is sent to the user terminal according to the future network fluctuation data.

7. The agent dialogue system according to claim 6, characterized in that: The intention recognition and answering module is further used to: determining a transmission frequency of the reply text according to the future network fluctuation data, and dividing the reply text into a plurality of text segments; generating a sending plan for the reply text according to the transmission frequency of the reply text and the text fragment; The text segment of the answer text is sent to the user terminal according to the sending plan.

8. An agent dialogue method, characterized in that: Applicable to the intelligent agent dialogue system according to any one of claims 1 to 7, wherein the intelligent agent dialogue system comprises a user end and a server end, and an output of the user end is communicatively connected to the server end; The user end includes a collection module, an audio conversion module, a speech recognition processing module and a speech synthesis module which are connected in sequence; the server end includes an intention recognition and answering module; The output end of the speech recognition processing module of the user end is connected to the input end of the intention recognition and answering module of the server end, and the output end of the intention recognition and answering module is connected to the input end of the speech synthesis module of the user end; The agent dialogue method comprises: Acquire the user's audio data through the acquisition module; The audio conversion module sets a transmission mode for the audio data based on the real-time monitored data network signal strength of the user terminal, wherein the transmission mode includes communication signal transmission and data network transmission; when the transmission mode is communication signal transmission, the audio data is encoded according to the transmission protocol of the communication push stream to obtain an audio stream corresponding to the audio data; The speech recognition processing module performs real-time analysis of the audio stream through deep learning to obtain an analysis result, and optimizes the analysis result through a language acoustic model to obtain text information corresponding to the audio data; when the transmission mode is data network transmission, the audio data is encoded according to the user datagram protocol to obtain network transmission data of the audio data, and the network transmission data of the audio data is transmitted to the server end; The intention recognition and answer module determines the domain and intention of the text information according to the text information corresponding to the audio data in combination with context information; and obtains the answer text of the audio data in combination with the logical reasoning algorithm and the language generation algorithm based on the domain and the intention; The intention recognition and response module processes the network transmission data, extracts semantic information therein, and identifies user intentions and needs; generates corresponding response text based on the recognition results, and determines the transmission frequency of the response text based on future network fluctuation data predicted by the data network signal strength; The answer text is speech-synthesized by the speech synthesis module to obtain an audio stream corresponding to the answer text, and the audio stream corresponding to the answer text is encoded according to the transmission protocol of the communication push stream to obtain audio data of the answer text; wherein, when the transmission mode of the audio data is the data network transmission, the speech rate parameters, intonation parameters, volume parameters and timbre parameters of the speech synthesis model are determined according to the data network signal strength; the preset acoustic parameters of the speech synthesis model are optimized according to the speech rate parameters, the intonation parameters, the volume parameters and the timbre parameters to obtain the optimized speech synthesis model; the answer text is dynamically speech-synthesized by the optimized speech synthesis model in combination with forward error correction and redundant coding strategies to obtain the audio stream corresponding to the answer text.

Images