Multi-channel speech synthesis method, device and equipment based on real-time AI model

Abstract

The present invention provides a multi-channel speech synthesis method, device and equipment based on a real-time AI model, including: frequency slicing and semantically annotating original speech data to obtain frequency slice data with semantic vectors; inputting the frequency slice data with semantic vectors into a coding network in a real-time AI model for coding processing to obtain a set of coding vectors; according to the frequency features, semantic features and pre-set channel allocation rules contained in each coding vector in the coding vector set, the coding vector set is allocated to multiple independent channels to obtain a coding vector group; for the coding vector group of each independent channel, a parameter calculation model based on chaos theory is used to generate speech synthesis parameters suitable for the corresponding channel; audio waveform construction processing is performed based on the speech synthesis parameters of each channel, and each audio waveform is merged to obtain synthesized speech. In the present invention, it is convenient to improve the naturalness and realism of the synthesized speech.

Description

Technical Field

[0001] The present invention relates to the technical field of speech synthesis, and in particular to a multi-channel speech synthesis method, device and equipment based on a real-time AI model. Background Art

[0002] Traditional speech synthesis methods often have some limitations and are difficult to meet the application requirements in complex scenarios. For multi-channel speech synthesis, the current methods lack reasonable rules for channel allocation and do not make effective allocations based on the characteristics of speech data. At the same time, when generating speech synthesis parameters, the calculation model used by traditional methods is relatively simple and cannot adapt well to the needs of different channels, resulting in poor results when the audio waveforms synthesized by each channel are fused, and the sound quality and stereoscopic sense of the synthesized speech are not prominent enough. Moreover, in multi-channel synthesis, the fusion method of the audio waveforms of each channel is also relatively simple, which cannot give full play to the advantages of multi-channels, resulting in insufficient naturalness and realism of the synthesized speech. Summary of the invention

[0003] The main purpose of the present invention is to provide a multi-channel speech synthesis method, device and equipment based on a real-time AI model, aiming to overcome the defect of insufficient realism of current synthesized speech.

[0004] To achieve the above object, the present invention provides a multi-channel speech synthesis method based on a real-time AI model, comprising the following steps:

[0005] Perform frequency slicing and semantic annotation on the original speech data to obtain frequency slicing data with semantic vectors;

[0006] The frequency slice data with semantic vectors is input into the encoding network in the real-time AI model for encoding processing to obtain an encoding vector set;

[0007] According to the frequency features and semantic features contained in each coding vector in the coding vector set and a preset channel allocation rule, the coding vector set is allocated to multiple independent channels to obtain a coding vector group for each channel;

[0008] For each independent channel's coding vector group, a parameter calculation model based on chaos theory is used to generate speech synthesis parameters suitable for the corresponding channel;

[0009] The audio waveform is constructed based on the speech synthesis parameters of each channel, and the audio waveforms constructed by each channel are fused using the waveform superposition principle to obtain synthesized speech.

[0010] Furthermore, the frequency slicing and semantic annotation of the original speech data to obtain frequency slice data with semantic vectors includes:

[0011] Perform frequency slicing processing on the original speech data based on the dynamic frequency to obtain a frequency slice set;

[0012] The semantic information associated with each slice unit in the frequency slice set in the time series is converted into a semantic vector form and embedded therein, so as to obtain frequency slice data with semantic vectors.

[0013] Furthermore, the encoding network is composed of multiple layers of neurons with different parameters and structures. Through layer-by-layer data transmission and conversion, the frequency slice data is converted into a high-dimensional encoding vector to obtain a set of encoding vectors.

[0014] Furthermore, for each independent channel encoding vector group, a parameter calculation model based on chaos theory is used to generate speech synthesis parameters adapted to the corresponding channel, including:

[0015] For each encoding vector group of each independent channel, extract key feature parameters of the encoding vector therein;

[0016] The key feature parameters are input into a pre-built calculation model based on a chaotic mapping function; the chaotic mapping function adopts a Logistic mapping equation; during the iterative operation of the chaotic mapping function, each iteration result is mapped to a corresponding speech synthesis parameter component based on a preset conversion rule, and after multiple iterations, a speech synthesis parameter suitable for the corresponding channel is generated.

[0017] Furthermore, the audio waveform construction process is performed based on the speech synthesis parameters of each channel, and the audio waveforms constructed by each channel are merged using the waveform superposition principle to obtain the synthesized speech, including:

[0018] For the speech synthesis parameters of each channel, an audio waveform construction algorithm based on digital signal processing technology is used to construct the audio waveform of each channel;

[0019] The audio waveforms of each channel are synchronously aligned and weightedly superimposed according to a preset weight coefficient; the weight coefficient is dynamically adjusted according to the importance, frequency characteristics and semantic features of the channel; in the superposition process, the waveform amplitude values ​​of each channel are added point by point to obtain a waveform amplitude value sequence of the synthesized speech; the waveform amplitude value sequence of the synthesized speech is converted from digital to analog to obtain an analog audio signal as the final synthesized speech.

[0020] Furthermore, after obtaining the synthesized speech, the method further comprises:

[0021] Acquiring identification information of a plurality of translation devices that receive the synthesized speech;

[0022] Based on the identification information of each translation device, a common communication code is generated;

[0023] When the synthesized speech is sent to each translation device, the communication code is sent at the same time; when the translation device receives the synthesized speech, it determines whether the communication code is received at the same time. If not, the synthesized speech is deleted; if received, the synthesized speech is read.

[0024] Furthermore, the generation of a common communication code based on the identification information of each translation device includes:

[0025] According to a preset rule, the identification information of each of the translation devices is sorted, and the sorted identification information is combined in sequence to obtain an identification combination;

[0026] Add the characters in the logo combination to the preset binary tree template one by one in order to form a logo binary tree;

[0027] Based on the identification binary tree, the common communication code is generated.

[0028] The present invention also provides a multi-channel speech synthesis device based on a real-time AI model, comprising:

[0029] A slicing unit, used to perform frequency slicing and semantic annotation on the original speech data to obtain frequency slice data with semantic vectors;

[0030] An encoding unit, used for inputting the frequency slice data with the semantic vector into the encoding network in the real-time AI model for encoding processing to obtain an encoding vector set;

[0031] An allocating unit, configured to allocate the coding vector set to a plurality of independent channels according to the frequency characteristics and semantic characteristics contained in each coding vector in the coding vector set and a preset channel allocation rule, so as to obtain a coding vector group for each channel;

[0032] A generating unit, for generating speech synthesis parameters adapted to the corresponding channel by using a parameter calculation model based on chaos theory for the coding vector group of each independent channel;

[0033] The synthesis unit is used to construct and process the audio waveform based on the speech synthesis parameters of each channel, and fuse the audio waveforms constructed by each channel using the waveform superposition principle to obtain synthesized speech.

[0034] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0035] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0036] The multi-channel speech synthesis method, device and equipment based on the real-time AI model provided by the present invention include: performing frequency slicing and semantic annotation on original speech data to obtain frequency slice data with semantic vectors; inputting the frequency slice data with semantic vectors into the coding network in the real-time AI model for coding processing to obtain a set of coding vectors; according to the frequency characteristics, semantic characteristics and pre-set channel allocation rules contained in each coding vector in the coding vector set, the coding vector set is allocated to multiple independent channels to obtain coding vector groups for each channel; for the coding vector group of each independent channel, a parameter calculation model based on chaos theory is used to generate speech synthesis parameters suitable for the corresponding channel; audio waveform construction processing is performed based on the speech synthesis parameters of each channel, and the audio waveforms constructed by each channel are merged using the waveform superposition principle to obtain synthesized speech. In the present invention, the coding vector set is allocated to multiple independent channels according to the frequency characteristics, semantic characteristics and pre-set channel allocation rules contained in each coding vector in the coding vector set, so that the allocation is more reasonable, which is conducive to improving the realism of the speech; a parameter calculation model based on chaos theory is used to generate speech synthesis parameters suitable for the corresponding channel, which can highlight the sound quality and three-dimensional sense of the synthesized speech; the audio waveforms constructed by each channel are fused using the waveform superposition principle to obtain synthesized speech, which is convenient for improving the naturalness and realism of the synthesized speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of a multi-channel speech synthesis method based on a real-time AI model in one embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram of the steps of a multi-channel speech synthesis method based on a real-time AI model in one embodiment of the present invention;

[0039] Figure 3 It is a structural block diagram of a multi-channel speech synthesis device based on a real-time AI model in one embodiment of the present invention;

[0040] Figure 4 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0041] The implementation, functional features and advantages of the present invention will be further described in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0043] Reference Figure 1 as well as Figure 2 In one embodiment of the present invention, a multi-channel speech synthesis method based on a real-time AI model is provided, comprising the following steps:

[0044] Step S1, frequency slicing and semantic annotation of original speech data to obtain frequency slice data with semantic vectors;

[0045] Step S2, inputting the frequency slice data with the semantic vector into the encoding network in the real-time AI model for encoding processing to obtain a set of encoding vectors;

[0046] Step S3, according to the frequency characteristics and semantic characteristics contained in each code vector in the code vector set and the preset channel allocation rule, the code vector set is allocated to multiple independent channels to obtain a code vector group of each channel;

[0047] Step S4, for each coding vector group of the independent channel, using a parameter calculation model based on chaos theory to generate speech synthesis parameters suitable for the corresponding channel;

[0048] Step S5, performing audio waveform construction processing based on the speech synthesis parameters of each channel, and fusing the audio waveforms constructed by each channel using the waveform superposition principle to obtain synthesized speech.

[0049] In this embodiment, as described in step S1 above, the original voice data is first frequency sliced ​​and semantically annotated, wherein frequency slicing uses signal processing technology to finely divide the original voice data according to different frequency ranges. For example, algorithms such as fast Fourier transform (FFT) can be used to convert the voice signal from the time domain to the frequency domain, and then slice it according to a specific frequency interval. In this way, the voice signal can be decomposed into sub-signals of different frequency components for better analysis and processing. The granularity of frequency slicing can be adjusted according to actual needs to achieve the best synthesis effect. Finer slicing can provide richer frequency information, but it will also increase the computational complexity; coarser slicing will lose some details.

[0050] The above semantic annotation uses natural language processing technology and machine learning algorithms to semantically annotate each frequency slice. This can be achieved by training a semantic annotation model, which can assign a corresponding semantic vector to each frequency slice based on the content and context of the speech. The semantic vector can represent the semantic information corresponding to the frequency slice, such as words, phrases, emotions, etc. This can better preserve the semantic content of the original speech in the subsequent synthesis process, and improve the intelligibility and naturalness of the synthesized speech.

[0051] As described in step S2 above, the encoding network in the above real-time AI model is usually composed of a multi-layer neural network, such as a convolutional neural network (CNN), a recurrent neural network (RNN) or a long short-term memory network (LSTM). The above network can effectively extract frequency features and semantic features in speech data. Each layer of the encoding network can learn different levels of feature representation, from low-level frequency features to high-level semantic features. In a layer-by-layer progressive manner, the frequency slice data with semantic vectors is finally converted into a set of encoding vectors.

[0052] The encoding process includes: first, the frequency slice data with semantic vectors is input into the first layer of the encoding network. This layer is usually a convolutional layer or a recurrent layer, which is used to extract low-level frequency features. Then, higher-level features are gradually extracted through subsequent layers, while integrating the information of semantic vectors. In this process, the network will continuously adjust weights and parameters to optimize the effect of feature extraction. Finally, the encoding network outputs a set of encoding vectors, each of which contains rich frequency features and semantic features. These encoding vectors can be used as input for subsequent steps to provide a basis for multi-channel speech synthesis.

[0053] As described in step S3 above, for each coding vector in the coding vector set, the frequency features and semantic features contained therein are further analyzed. Frequency features may include frequency range, frequency intensity, etc.; semantic features may include words, phrases, emotions, etc. By analyzing these features, the speech content and characteristics represented by each coding vector can be understood, providing a basis for channel allocation.

[0054] The above-mentioned pre-set channel allocation rules are formulated according to the characteristics of speech and synthesis requirements. For example, the coding vectors can be allocated to different channels according to the frequency range, with the low-frequency part allocated to one channel and the high-frequency part allocated to another channel. It can also be allocated according to semantic features, for example, the coding vectors related to a specific topic are allocated to one channel, and the coding vectors related to other topics are allocated to another channel. The channel allocation rules can be adjusted and optimized in advance according to the actual situation to achieve the best synthesis effect.

[0055] The channel allocation process includes: allocating the set of coding vectors to multiple independent channels based on the analyzed frequency features and semantic features, as well as the pre-set channel allocation rules. Each channel has its own set of coding vectors, representing different frequency and semantic information. The channel allocation process can be implemented by an allocation function or algorithm, which allocates the coding vectors to the corresponding channels based on their features and rules.

[0056] As described in step S4 above, chaos theory is a theory that studies the behavior of nonlinear systems, and it has the characteristics of randomness, sensitivity, and determinism. In speech synthesis, chaos theory can be used to generate more flexible and accurate speech synthesis parameters. The parameter calculation model based on chaos theory can utilize the randomness and sensitivity of the chaotic system to generate parameters with diversity and adaptability to better adapt to the characteristics and requirements of different channels.

[0057] For each coding vector group of an independent channel, a parameter calculation model based on chaos theory is used to generate speech synthesis parameters adapted to the corresponding channel. First, the initial state and parameters of the chaotic system are determined according to the coding vector group of the channel. Then, a series of parameter values ​​are generated by iterating the chaotic system. The above parameter values ​​may include frequency, amplitude, phase, etc., which will be used for audio waveform construction processing. In the process of generating parameters, adjustments and optimizations can be made according to actual needs to achieve the best synthesis effect.

[0058] As described in step S5 above, audio waveform construction processing is performed based on the speech synthesis parameters of each channel. Each channel generates a specific audio waveform according to its own parameter value. Among them, the audio waveform construction can use various signal processing technologies, such as sine wave synthesis, pulse code modulation (PCM), etc. By converting the parameter value into an audio signal, the audio waveform of each channel can be obtained.

[0059] The audio waveforms constructed by each channel are fused using the waveform superposition principle. The waveform superposition principle refers to superimposing multiple audio waveforms according to certain rules to obtain a synthesized audio waveform. During the fusion process, the weights and phases of different channels can also be adjusted according to their importance and characteristics to achieve the best fusion effect. The fused audio waveform will be used as the final synthesized speech output, with better sound quality and stereoscopic effect.

[0060] In summary, by allocating the coding vector set to multiple independent channels according to the frequency characteristics, semantic characteristics and pre-set channel allocation rules contained in each coding vector in the coding vector set, the allocation is made more reasonable, which is conducive to improving the realism of the speech; a parameter calculation model based on chaos theory is used to generate speech synthesis parameters suitable for the corresponding channel, which can highlight the sound quality and three-dimensional sense of the synthesized speech; the audio waveforms constructed by each channel are fused using the waveform superposition principle to obtain synthesized speech, which is convenient for improving the naturalness and realism of the synthesized speech.

[0061] In one embodiment, the frequency slicing and semantic annotation of the original speech data to obtain frequency slice data with semantic vectors includes:

[0062] Perform frequency slicing processing on the original speech data based on the dynamic frequency to obtain a frequency slice set;

[0063] The semantic information associated with each slice unit in the frequency slice set in the time series is converted into a semantic vector form and embedded therein, so as to obtain frequency slice data with semantic vectors.

[0064] In this embodiment, first, the original speech data is subjected to frequency slicing processing in a dynamic frequency-based manner. This method can dynamically adjust the range and accuracy of frequency slicing according to the characteristics of the speech signal, thereby more accurately capturing the different frequency components in the speech. For example, by analyzing the spectral characteristics of the speech signal, the appropriate frequency slicing interval can be automatically determined so that each slicing unit can better reflect the characteristics of the speech within a specific frequency range. The resulting frequency slicing set contains the subdivision information of the original speech data at different frequencies, providing richer basic data for subsequent processing.

[0065] For each slice unit in the frequency slice set, the semantic information associated with it in the time series is converted into a semantic vector form and embedded therein. Specifically, first, the semantic content in the speech signal is analyzed through natural language processing technology and machine learning algorithms, and the semantic information corresponding to each slice unit is determined. Then, these semantic information are converted into semantic vectors so that they can be processed and transmitted in digital form. Finally, the semantic vectors are embedded into the corresponding slice units, so that each frequency slice not only contains frequency features, but also carries semantic information related to it. The advantage of this is that in the subsequent speech synthesis process, the semantic content of the original speech can be better preserved, and the intelligibility and naturalness of the synthesized speech can be improved.

[0066] In this embodiment, the dynamic frequency slicing method can be adjusted according to the actual situation of the speech signal, avoiding the inaccuracy problem that may be caused by fixed frequency slicing, and improving the accuracy and adaptability of frequency slicing. The semantic vector is embedded in the frequency slice, so that the semantic information can be better utilized in the speech synthesis process, thereby enhancing the semantic coherence and intelligibility of the synthesized speech. The frequency slice data with semantic vectors provides a richer information basis for subsequent multi-channel speech synthesis, so that each channel can better process the speech content of specific frequency and semantics, and improve the quality and three-dimensional sense of the synthesized speech.

[0067] In one embodiment, the encoding network is composed of multiple layers of neurons with different parameters and structures. Through layer-by-layer data transmission and conversion, the frequency slice data is converted into a high-dimensional encoding vector to obtain a set of encoding vectors.

[0068] In this embodiment, the above-mentioned coding network is composed of multiple layers of neuron layers with different parameters and structures. This design can make full use of different types of neuron layers to extract and process information at different levels. For example, the coding network may include convolutional layers, recurrent layers, fully connected layers, etc. The convolutional layer can effectively extract local features in frequency slice data; the recurrent layer can process time series information and capture the dynamic changes of speech signals; the fully connected layer can integrate and transform the extracted features to obtain a high-dimensional coding vector. The parameters and structures of different layers are adjusted according to specific task requirements and data characteristics. By continuously optimizing these parameters and structures, the performance and effect of the coding network can be improved.

[0069] Through layer-by-layer data transmission and conversion, the frequency slice data is converted into a high-dimensional coding vector. In this process, each layer of neurons will perform specific processing and transformation on the input data. First, the frequency slice data enters the first layer of the coding network as input. The first layer of neurons processes the input data according to their specific parameters and structure to obtain the output of the first layer. Then, the output of the first layer is used as the input of the second layer, and the second layer of neurons continues to process and transform it. And so on, after processing by multiple layers of neurons, a high-dimensional coding vector is finally obtained. This layer-by-layer data transmission and conversion method allows the coding network to gradually extract and integrate features at different levels in the speech data, thereby obtaining a more representative and expressive set of coding vectors. The obtained set of coding vectors contains the high-dimensional feature representation of the frequency slice data after being processed by the coding network. These coding vectors can be used as important inputs in the subsequent multi-channel speech synthesis process. In the subsequent steps of channel allocation, parameter generation, and audio waveform construction, the frequency features, semantic features, and other information contained in each coding vector in the coding vector set will be fully utilized to achieve high-quality speech synthesis.

[0070] In one embodiment, for each independent channel encoding vector group, a parameter calculation model based on chaos theory is used to generate speech synthesis parameters adapted to the corresponding channel, including:

[0071] For each encoding vector group of each independent channel, extract key feature parameters of the encoding vector therein;

[0072] The key feature parameters are input into a pre-built calculation model based on a chaotic mapping function; the chaotic mapping function adopts a Logistic mapping equation; during the iterative operation of the chaotic mapping function, each iteration result is mapped to a corresponding speech synthesis parameter component based on a preset conversion rule, and after multiple iterations, a speech synthesis parameter suitable for the corresponding channel is generated.

[0073] In this embodiment, for each encoding vector group of an independent channel, key feature parameters are first extracted. This step is intended to filter out the parameters that best represent the speech characteristics of the channel from the encoding vector. A variety of methods can be used to extract key feature parameters, such as principal component analysis (PCA), independent component analysis (ICA), etc. The above method can effectively reduce the data dimension and extract the most representative features. The extracted key feature parameters will serve as the input of the subsequent parameter calculation model to provide a basis for generating speech synthesis parameters adapted to the corresponding channel.

[0074] A computational model based on a chaos mapping function is pre-built, wherein the chaos mapping function adopts the Logistic mapping equation. Chaos theory has the characteristics of randomness, sensitivity and determinism, and can provide more flexible and diverse results for parameter generation. The Logistic mapping equation is a classic chaos mapping function with a simple form and rich dynamic behavior. By adjusting the parameters of the Logistic mapping equation, the behavior of the chaotic system can be controlled to generate different speech synthesis parameters. After the key characteristic parameters are input into the computational model, the iterative operation of the chaos mapping function begins. In each iteration process, the iteration result is mapped to the corresponding speech synthesis parameter component based on the preset conversion rule. The preset conversion rule can be designed according to specific task requirements and data characteristics. For example, the numerical range of the iteration result can be mapped to a specific speech synthesis parameter interval, or the positive and negative of the parameter can be determined according to the sign of the iteration result. After multiple iterations, speech synthesis parameters suitable for the corresponding channel are gradually generated. Multiple iterations can increase the diversity and accuracy of the parameters, making the synthesized speech more natural and realistic.

[0075] In this embodiment, the parameter calculation model based on chaos theory can generate more flexible and diverse speech synthesis parameters, avoiding the limitation of fixed parameters in traditional methods. The randomness and sensitivity of the chaotic system can make the synthesized speech have more natural changes and dynamics, improve its intelligibility and naturalness. Generating parameters for each independent channel can better adapt to the speech characteristics and synthesis requirements of different channels, and improve the quality of multi-channel speech synthesis.

[0076] In one embodiment, the audio waveform construction process is performed based on the speech synthesis parameters of each channel, and the audio waveforms constructed by each channel are merged using the waveform superposition principle to obtain the synthesized speech, including:

[0077] For the speech synthesis parameters of each channel, an audio waveform construction algorithm based on digital signal processing technology is used to construct the audio waveform of each channel;

[0078] The audio waveforms of each channel are synchronously aligned and weightedly superimposed according to a preset weight coefficient; the weight coefficient is dynamically adjusted according to the importance, frequency characteristics and semantic features of the channel; in the superposition process, the waveform amplitude values ​​of each channel are added point by point to obtain a waveform amplitude value sequence of the synthesized speech; the waveform amplitude value sequence of the synthesized speech is converted from digital to analog to obtain an analog audio signal as the final synthesized speech.

[0079] In this embodiment, first, for the speech synthesis parameters of each channel, an audio waveform construction algorithm based on digital signal processing technology is used to construct the audio waveform of each channel. Among them, digital signal processing technology can efficiently process and generate audio signals. Through a specific algorithm, the speech synthesis parameters are converted into an audio waveform so that each channel has an independent audio waveform representation. Different audio waveform construction algorithms can be selected according to specific needs, such as sine wave synthesis, pulse code modulation (PCM), etc. These algorithms can accurately construct an audio waveform with a specific frequency, amplitude and phase according to the speech synthesis parameters.

[0080] Then, the audio waveforms of each channel are synchronized and aligned. This step ensures that the audio waveforms of each channel are consistent in time for subsequent superposition operations. Synchronous alignment can be achieved by adjusting the start time and sampling rate of the audio waveform. Ensure that the corresponding positions of the audio waveforms of each channel on the time axis are accurately matched to avoid distortion of the synthesized speech caused by time misalignment.

[0081] Then, the audio waveforms of each channel are weighted and superimposed according to the preset weight coefficients. The weight coefficients are dynamically adjusted according to the importance, frequency characteristics and semantic features of the channels. The importance of the channels can be determined according to their role and contribution in speech synthesis. For example, some channels may contain the main speech information and therefore be given a higher weight. Frequency characteristics and semantic features can also affect the allocation of weight coefficients. For example, for high-frequency channels or channels containing specific semantic information, their weights can be appropriately increased to highlight the performance of these parts in the synthesized speech. During the superposition process, the waveform amplitude values ​​of each channel are added point by point to obtain a waveform amplitude value sequence of the synthesized speech. Point by point addition ensures that the amplitude value of each sampling point is reasonably integrated, thereby generating a complete waveform of the synthesized speech.

[0082] The waveform amplitude value sequence of the synthesized speech is converted into an analog audio signal as the final synthesized speech. The digital-to-analog conversion converts the waveform amplitude value sequence in digital form into an analog audio signal so that it can be played by a playback device or used in practical applications. The analog audio signal after the digital-to-analog conversion has a continuous waveform and can more realistically restore the sound characteristics of the synthesized speech.

[0083] In this embodiment, the audio waveform construction algorithm using digital signal processing technology can accurately generate the audio waveform of each channel and improve the quality of synthesized speech. The weight coefficient is dynamically adjusted according to the importance, frequency characteristics and semantic features of the channel, so that the synthesized speech is more in line with actual needs and highlights key information. Ensure that the audio waveforms of each channel are synchronized in time to avoid distortion and incoordination in the synthesized speech.

[0084] In one embodiment, after obtaining the synthesized speech, the following steps are included:

[0085] Acquiring identification information of a plurality of translation devices that receive the synthesized speech;

[0086] Based on the identification information of each translation device, a common communication code is generated;

[0087] When the synthesized speech is sent to each translation device, the communication code is sent at the same time; when the translation device receives the synthesized speech, it determines whether the communication code is received at the same time. If not, the synthesized speech is deleted; if received, the synthesized speech is read.

[0088] In this embodiment, after obtaining the synthesized speech, the identification information of the multiple translation devices receiving the synthesized speech is first obtained. The above identification information can be a unique serial number, MAC address, etc. of the device, which is used to distinguish different translation devices. These identification information can be obtained by communicating with the translation device or querying from the system database, etc., which provides a basis for the subsequent generation of a common communication code and ensuring the accurate transmission of the synthesized speech.

[0089] Furthermore, based on the identification information of each translation device, a common communication code is generated. The communication code can be a randomly generated number, letter combination or other specific format code. The process of generating the communication code can use an encryption algorithm or other security mechanism to ensure the uniqueness and security of the communication code. The purpose of the communication code is to serve as a verification mark for the transmission of the synthesized speech to ensure that the synthesized speech is received and processed by the correct translation device.

[0090] Finally, when the synthesized speech is sent to each translation device, the generated communication code is sent at the same time. In this way, when the translation device receives the synthesized speech, it can also determine whether the communication code has been received. By sending the synthesized speech and the communication code together, the reliability and security of the transmission can be improved. If the synthesized speech is partially lost or damaged during the transmission process, but the communication code is not lost, the translation device can determine that the synthesized speech is incomplete based on the communication code, and take corresponding processing measures.

[0091] When the translation device receives the synthesized speech, it will determine whether the communication code is received at the same time. If the communication code is not received, it means that there may be a problem with the transmission of the synthesized speech. In order to ensure the accuracy and security of the data, the translation device will delete the received synthesized speech. If the communication code is received, it means that the synthesized speech is complete and correctly transmitted. At this point, the translation device can read the synthesized speech and perform subsequent translation processing and other operations.

[0092] In this embodiment, through the common communication code verification mechanism, it is ensured that the synthesized speech is received by the correct translation device, invalid data processing caused by transmission errors is avoided, and the reliability of transmission is improved. The generation and verification process of the communication code can adopt security mechanisms such as encryption, which enhances the security of the synthesized speech during transmission and prevents data from being illegally obtained or tampered with. The translation device can automatically determine the integrity of the synthesized speech and perform deletion or reading operations according to the situation, reducing the need for manual intervention and improving the degree of automation.

[0093] In one embodiment, generating a common communication code based on the identification information of each translation device includes:

[0094] According to a preset rule, the identification information of each of the translation devices is sorted, and the sorted identification information is combined in sequence to obtain an identification combination;

[0095] Add the characters in the logo combination to the preset binary tree template one by one in order to form a logo binary tree;

[0096] Based on the identification binary tree, the common communication code is generated.

[0097] In this embodiment, first, the identification information of each translation device is sorted according to a preset rule. The preset rule can be to sort according to the numerical value of the identification information, the alphabetical order, etc., to ensure the orderliness of the identification information. The sorted identification information is combined in order to obtain an identification combination. For example, if there are three translation devices whose identification information is A, B, and C, respectively, and they are sorted into B, A, and C according to a certain rule, then the combined identification combination is "BAC".

[0098] Then, the characters in the logo combination are added one by one in sequence to the preset binary tree template to form a logo binary tree. A binary tree is a data structure with efficient insertion, search and traversal characteristics. The preset binary tree template can be an empty binary tree structure, waiting for the characters in the logo combination to be added one by one. For example, for the logo combination "BAC", first add "B" to the root node of the binary tree, then add "A" to the left child node of "B", and then add "C" to the right child node of "B", and gradually build the logo binary tree. The above binary tree template is a pre-set binary tree structure, which can be a left-right symmetrical structure.

[0099] Finally, a common communication code is generated based on the constructed identification binary tree. There are many ways to generate a communication code. For example, the binary tree can be traversed to generate a code in a specific format based on information such as the node position and character value. The communication code generation process can also use encryption algorithms or other security mechanisms to ensure the uniqueness and security of the communication code. The generated communication code will serve as a verification mark for the transmission of the synthesized speech to ensure that the synthesized speech is received and processed by the correct translation device.

[0100] In this embodiment, by sorting and combining the identification information and constructing a binary tree, the generated communication code is ensured to have a certain degree of orderliness and uniqueness. This helps to improve the reliability and recognition of the communication code and reduce the risk of erroneous reception. The use of a binary tree data structure to generate communication codes can efficiently process a large amount of identification information and can be flexibly adjusted and expanded according to different needs and scenarios.

[0101] In one embodiment, generating the common communication code based on the identification binary tree includes:

[0102] The identification binary tree is standardized so that it has a bilaterally symmetrical structure along the vertical direction;

[0103] Connecting all the English characters in the identification binary tree in sequence to form a character curve;

[0104] Calculating the shortest distance between all the numeric characters in the identification binary tree and the character curve, and taking the numeric character whose shortest distance is less than a preset value as the target numeric character;

[0105] The target digital characters are combined in sequence to obtain the common communication code.

[0106] In this embodiment, the identification binary tree is standardized so that it is a left-right symmetrical structure along the vertical direction. The purpose of this step is to make the binary tree have certain regularity and symmetry, which is convenient for subsequent processing and analysis. The standardized processing can be achieved by adjusting the node position of the binary tree, rotating the nodes, etc. For example, starting from the root node, the structure of the left and right subtrees can be gradually adjusted so that the entire binary tree presents a left-right symmetrical form in the vertical direction.

[0107] Then, all the English characters in the binary tree are connected in sequence to form a character curve. This step is to extract the English characters in the binary tree and connect them into a curve in sequence. The character curve can be formed by traversing the binary tree, finding all the English character nodes, and connecting these characters in a specific order (such as pre-order traversal, in-order traversal or post-order traversal). The character curve can be regarded as a visual representation of the character information in the binary tree.

[0108] Then, calculate the shortest distance between all the numeric characters in the binary tree and the character curve. For each numeric character node, calculate its shortest distance to the character curve. The numeric character whose shortest distance is less than the preset value is taken as the target numeric character. The preset value can be set according to actual needs to filter out numeric characters that are closer to the character curve. The purpose of this step is to select numeric characters that have a certain correlation with the English character curve from the numeric characters as an important part of generating the communication code.

[0109] Finally, the target digital characters are combined in sequence to obtain a common communication code. The selected target digital characters are connected in a preset order to form the final communication code. The combination method can be designed according to actual needs. For example, the digital characters can be directly arranged in sequence, or combined using specific encoding rules. The generated communication code will serve as a verification mark for the transmission of the synthesized speech to ensure that the synthesized speech is received and processed by the correct translation device.

[0110] In this embodiment, a novel communication code generation method is provided by standardizing the identification binary tree, forming a character curve, and screening and combining digital characters. This method has high complexity and security and is difficult to crack and forge. The formation of the character curve makes the character information in the binary tree more intuitive and visible. At the same time, by calculating the closest distance between the digital character and the character curve, the correlation between the digital character and the English character is established. This correlation can improve the reliability and recognition of the communication code. The technical solution can also be adjusted and optimized according to different identification information and requirements. For example, the size of the preset value can be changed to adjust the screening criteria of the target digital character, or different character connection orders can be used to generate different communication codes.

[0111] Reference Figure 3 In another embodiment of the present invention, a multi-channel speech synthesis device based on a real-time AI model is provided, comprising:

[0112] A slicing unit, used to perform frequency slicing and semantic annotation on the original speech data to obtain frequency slice data with semantic vectors;

[0113] An encoding unit, used for inputting the frequency slice data with the semantic vector into the encoding network in the real-time AI model for encoding processing to obtain an encoding vector set;

[0114] An allocating unit, configured to allocate the coding vector set to a plurality of independent channels according to the frequency characteristics and semantic characteristics contained in each coding vector in the coding vector set and a preset channel allocation rule, so as to obtain a coding vector group for each channel;

[0115] A generating unit, for generating speech synthesis parameters adapted to the corresponding channel by using a parameter calculation model based on chaos theory for the coding vector group of each independent channel;

[0116] The synthesis unit is used to construct and process the audio waveform based on the speech synthesis parameters of each channel, and fuse the audio waveforms constructed by each channel using the waveform superposition principle to obtain synthesized speech.

[0117] In this embodiment, for the specific implementation of each unit in the above device embodiment, please refer to the above method embodiment, which will not be repeated here.

[0118] Reference Figure 4 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 4As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0119] Those skilled in the art will understand that Figure 4 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0120] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0121] In summary, the multi-channel speech synthesis method, device and equipment based on the real-time AI model provided in the embodiments of the present invention include: frequency slicing and semantic annotation of the original speech data to obtain frequency slice data with semantic vectors; inputting the frequency slice data with semantic vectors into the coding network in the real-time AI model for encoding processing to obtain a set of coding vectors; according to the frequency characteristics, semantic characteristics and pre-set channel allocation rules contained in each coding vector in the coding vector set, the coding vector set is allocated to multiple independent channels to obtain coding vector groups for each channel; for each independent channel coding vector group, a parameter calculation model based on chaos theory is used to generate speech synthesis parameters suitable for the corresponding channel; audio waveform construction processing is performed based on the speech synthesis parameters of each channel, and the audio waveforms constructed by each channel are fused using the waveform superposition principle to obtain synthesized speech. In the present invention, the coding vector set is allocated to multiple independent channels according to the frequency characteristics, semantic characteristics and pre-set channel allocation rules contained in each coding vector in the coding vector set, so that the allocation is more reasonable, which is conducive to improving the realism of the speech; a parameter calculation model based on chaos theory is used to generate speech synthesis parameters suitable for the corresponding channel, which can highlight the sound quality and three-dimensional sense of the synthesized speech; the audio waveforms constructed by each channel are fused using the waveform superposition principle to obtain synthesized speech, which is convenient for improving the naturalness and realism of the synthesized speech.

[0122] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0123] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0124] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A multi-channel speech synthesis method based on a real-time AI model, characterized in that: The following steps are involved: Perform frequency slicing and semantic annotation on the original speech data to obtain frequency slicing data with semantic vectors; Inputting the frequency slice data with the semantic vector into the encoding network in the real-time AI model for encoding processing to obtain a set of encoding vectors; the encoding network is composed of multiple layers of neurons with different parameters and structures, and the frequency slice data is converted into a high-dimensional encoding vector through layer-by-layer data transmission and conversion to obtain a set of encoding vectors; According to the frequency features and semantic features contained in each coding vector in the coding vector set and a preset channel allocation rule, the coding vector set is allocated to multiple independent channels to obtain a coding vector group for each channel; For each independent channel's coding vector group, a parameter calculation model based on chaos theory is used to generate speech synthesis parameters adapted to the corresponding channel; The audio waveform is constructed based on the speech synthesis parameters of each channel, and the audio waveforms constructed by each channel are fused using the waveform superposition principle to obtain synthesized speech.

2. The multi-channel speech synthesis method based on real-time AI model according to claim 1, characterized in that: The frequency slicing and semantic annotation of the original speech data to obtain frequency slicing data with semantic vectors includes: Perform frequency slicing processing on the original voice data based on the dynamic frequency to obtain a frequency slice set; The semantic information associated with each slice unit in the frequency slice set in the time series is converted into a semantic vector form and embedded therein, so as to obtain frequency slice data with semantic vectors.

3. The multi-channel speech synthesis method based on real-time AI model according to claim 1, characterized in that: The encoding vector group for each independent channel uses a parameter calculation model based on chaos theory to generate speech synthesis parameters adapted to the corresponding channel, including: For each encoding vector group of each independent channel, extract key feature parameters of the encoding vector therein; The key feature parameters are input into a pre-built calculation model based on a chaotic mapping function; the chaotic mapping function adopts a Logistic mapping equation; during the iterative operation of the chaotic mapping function, each iteration result is mapped to a corresponding speech synthesis parameter component based on a preset conversion rule, and after multiple iterations, a speech synthesis parameter suitable for the corresponding channel is generated.

4. The multi-channel speech synthesis method based on real-time AI model according to claim 1, characterized in that: The audio waveform construction process is performed based on the speech synthesis parameters of each channel, and the audio waveforms constructed by each channel are merged by using the waveform superposition principle to obtain the synthesized speech, including: For the speech synthesis parameters of each channel, an audio waveform construction algorithm based on digital signal processing technology is used to construct the audio waveform of each channel; The audio waveforms of each channel are synchronously aligned and weightedly superimposed according to a preset weight coefficient; the weight coefficient is dynamically adjusted according to the importance, frequency characteristics and semantic features of the channel; in the superposition process, the waveform amplitude values ​​of each channel are added point by point to obtain a waveform amplitude value sequence of the synthesized speech; the waveform amplitude value sequence of the synthesized speech is converted from digital to analog to obtain an analog audio signal as the final synthesized speech.

5. The multi-channel speech synthesis method based on real-time AI model according to claim 1, characterized in that: After obtaining the synthesized speech, the method comprises: Acquiring identification information of a plurality of translation devices that receive the synthesized speech; Based on the identification information of each translation device, a common communication code is generated; When the synthesized speech is sent to each translation device, the communication code is sent at the same time; when the translation device receives the synthesized speech, it determines whether the communication code is received at the same time. If not, the synthesized speech is deleted; if received, the synthesized speech is read.

6. The multi-channel speech synthesis method based on real-time AI model according to claim 5, characterized in that: The method of generating a common communication code based on the identification information of each translation device includes: According to a preset rule, the identification information of each of the translation devices is sorted, and the sorted identification information is combined in sequence to obtain an identification combination; Add the characters in the logo combination to the preset binary tree template one by one in order to form a logo binary tree; Based on the identification binary tree, the common communication code is generated.

7. A multi-channel speech synthesis device based on a real-time AI model, characterized in that: include: A slicing unit, used to perform frequency slicing and semantic annotation on the original speech data to obtain frequency slice data with semantic vectors; An encoding unit is used to input the frequency slice data with semantic vectors into the encoding network in the real-time AI model for encoding processing to obtain a set of encoding vectors; the encoding network is composed of multiple layers of neurons with different parameters and structures, and the frequency slice data is converted into a high-dimensional encoding vector through layer-by-layer data transmission and conversion to obtain a set of encoding vectors; An allocating unit, configured to allocate the coding vector set to a plurality of independent channels according to the frequency characteristics and semantic characteristics contained in each coding vector in the coding vector set and a preset channel allocation rule, so as to obtain a coding vector group for each channel; A generating unit, for generating speech synthesis parameters adapted to the corresponding channel by using a parameter calculation model based on chaos theory for the coding vector group of each independent channel; The synthesis unit is used to construct and process the audio waveform based on the speech synthesis parameters of each channel, and fuse the audio waveforms constructed by each channel using the waveform superposition principle to obtain synthesized speech.

8. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images