Voice generation method and device based on artificial intelligence, computer equipment and medium

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to an artificial intelligence-based voice generation method, which comprises the following steps of: performing layer-by-layer calculation on noise data by using a diffusion model, and calculating an error corresponding to a specified encoder-decoder structure of a current time step when the current time step is detected to reach a cache node; if the error is smaller than a cache threshold value, acquiring a specified output result of a specified encoder-decoder structure cached in the previous time step corresponding to the current time step as an output result of the current time step; if the error is greater than or equal to the cache threshold value, recalculating the specified encoder-decoder structure of the current time step; the cache node judgment and dynamic cache and multiplexing decision process are repeatedly executed until calculation of all time steps is completed, and the output of the last time step is obtained to serve as a voice output result to be output; the method can be applied to voice generation scenes in the financial science and technology field and the digital medical field, and the voice generation efficiency is effectively improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology and can be applied to fields such as fintech and digital healthcare, particularly to artificial intelligence-based speech generation methods, devices, computer equipment, and storage media. Background Technology

[0002] In the field of text-to-speech (TTS) generation technology, existing technologies are mainly divided into two categories: autoregressive (AR) and non-autoregressive (NAR). Autoregressive methods, such as Tacotron 2, while modeling time dependencies well, suffer from slow inference speed and are prone to cumulative errors when generating long audio clips. For example, in the financial insurance field, when generating long audio interpretations of insurance terms, autoregressive methods may experience stuttering and inaccurate information delivery due to cumulative errors, affecting customers' understanding of the insurance terms. Non-autoregressive methods, like FastPitch, offer significant improvements in generation efficiency, but face challenges in long-range dependency modeling and text alignment, often requiring additional alignment mechanisms. Taking the medical field as an example, when generating long audio clips for medical knowledge dissemination, non-autoregressive methods may suffer from insufficient long-range dependency modeling, resulting in chaotic speech rhythm and unclear highlighting of key information, affecting patients' reception of medical knowledge.

[0003] In recent years, diffusion models have been introduced into TTS tasks, with representative works including Voicebox, the NaturalSpeech series, and F5-TTS. These methods generate high-fidelity speech through progressive denoising, avoiding explicit alignment problems and generating natural and fluent speech under zero-sample conditions. However, diffusion-based TTS models suffer from a major drawback: enormous computational overhead. The generation process requires repeated execution of self-attention and feedforward computations in the Transformer layer across multiple iterations, making the inference speed insufficient for real-time or low-latency applications, resulting in low speech generation efficiency. For example, in emergency claims voice notification scenarios in the financial insurance field, slow inference speed may prevent timely delivery of claims information to customers, hindering the claims process; similarly, in emergency medical instruction voice communication scenarios in the medical field, slow inference speed may delay patient treatment.

[0004] Therefore, there is an urgent need to provide an efficient TTS generation technology to improve speech generation efficiency and meet the needs of real-time or low-latency applications. Summary of the Invention

[0005] The purpose of this application is to propose a speech generation method, apparatus, computer device, and storage medium based on artificial intelligence, so as to solve the technical problem that the existing speech generation methods based on diffusion models are inefficient.

[0006] Firstly, an artificial intelligence-based speech generation method is provided, including: Acquire noise data to be processed and input the noise data into a preset diffusion model; wherein, the diffusion model is a speech generation model composed of a specified number of encoder-decoder structures stacked together; Based on the preset cache node judgment and dynamic caching and reuse decision strategy, the diffusion model is used to perform layer-by-layer calculation processing on the noise data corresponding to the specified number, and when the current time step reaches the preset cache node, the error corresponding to the specified encoder-decoder structure of the current time step is calculated. If the error is less than a preset cache threshold, then the specified output result of the specified encoder-decoder structure cached in the previous time step corresponding to the current time step is obtained from the preset cache data structure and used as the output result of the current time step. If the error is greater than or equal to the cache threshold, the specified encoder-decoder structure of the current time step is recalculated, and the calculation result is used for subsequent processing of the current time step. Continue the calculation and processing for the next time step, and repeat the cache node judgment and dynamic caching and reuse decision process until the calculation of all time steps corresponding to the total number of time steps of the diffusion model is completed, and obtain the output of the last time step as the speech output result. The voice output result is processed for output.

[0007] Secondly, an artificial intelligence-based speech generation device is provided, comprising: The first processing module is used to acquire noise data to be processed and input the noise data into a preset diffusion model; wherein, the diffusion model is a speech generation model composed of a specified number of encoder-decoder structures stacked together; The second processing module is used to perform layer-by-layer calculation processing on the noise data corresponding to the specified number using the diffusion model based on the preset cache node judgment and dynamic caching and reuse decision strategy, and to calculate the error corresponding to the specified encoder-decoder structure of the current time step when the current time step is detected to have reached the preset cache node. The acquisition module is used to acquire, if the error is less than a preset cache threshold, a specified output result of the specified encoder-decoder structure cached in the previous time step corresponding to the current time step from a preset cache data structure and use it as the output result of the current time step. The first calculation module is used to recalculate the specified encoder-decoder structure of the current time step if the error is greater than or equal to the cache threshold, and use the calculation result for subsequent processing of the current time step. The third processing module is used to continue the calculation and processing of the next time step, and repeatedly execute the cache node judgment and dynamic caching and reuse decision process until the calculation of all time steps corresponding to the total number of time steps of the diffusion model is completed, and the output of the last time step is obtained as the speech output result. The output module is used to process the voice output results.

[0008] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described artificial intelligence-based speech generation method.

[0009] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described artificial intelligence-based speech generation method.

[0010] In the aforementioned scheme implemented by the artificial intelligence-based speech generation method, apparatus, computer equipment, and storage medium, noise data to be processed is first acquired and input into a preset diffusion model; wherein, the diffusion model is a speech generation model composed of a specified number of encoder-decoder structures stacked together; then, based on a preset cache node judgment and dynamic caching and reuse decision strategy, the diffusion model is used to perform layer-by-layer calculation processing on the noise data corresponding to the specified number, and when the current time step reaches a preset cache node, the error corresponding to the specified encoder-decoder structure of the current time step is calculated; if the error is less than a preset cache threshold, then the noise data is processed from a preset number of cache nodes. The specified output result of the specified encoder-decoder structure cached in the previous time step corresponding to the current time step is obtained from the structure and used as the output result of the current time step. If the error is greater than or equal to the cache threshold, the specified encoder-decoder structure of the current time step is recalculated, and the obtained calculation result is used for the subsequent processing of the current time step. The calculation processing of the next time step continues, and the cache node judgment and dynamic caching and reuse decision process are repeatedly executed until the calculation of all time steps corresponding to the total number of time steps of the diffusion model is completed, and the output of the last time step is obtained as the speech output result. Finally, the speech output result is processed. In this way, this application adopts a diffusion-based TTS acceleration method based on selective caching. By caching and reusing part of the output of the encoder-decoder structure in the inference stage of speech generation, significant computational acceleration is achieved, which effectively improves the efficiency of speech generation, while maintaining the naturalness and clarity of the speech and ensuring the quality of the generated speech. Attached Figure Description

[0011] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is an exemplary system architecture diagram to which this application can be applied; Figure 2 This is a flowchart of an embodiment of the AI-based speech generation method according to this application; Figure 3 This is a schematic diagram of the structure of an embodiment of the artificial intelligence-based speech generation device according to this application; Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0014] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0015] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0016] like Figure 1 As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0017] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0018] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptops 1011, tablets 1012, or mobile phones 1013, terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, and a desktop computer, etc.

[0019] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0020] It should be noted that the AI-based speech generation method provided in this application is generally executed by a server / terminal device, and correspondingly, the AI-based speech generation device is generally located in the server / terminal device.

[0021] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0022] Continue to refer to Figure 2 This document illustrates a flowchart of an embodiment of the AI-based speech generation method according to this application. The order of steps in the flowchart can be changed, and some steps can be omitted, depending on different requirements. The AI-based speech generation method provided in this application can be applied to any scenario requiring speech generation, and therefore can be applied to products in these scenarios, such as speech generation products in the financial insurance field. The AI-based speech generation method includes the following steps: Step S201: Obtain the noise data to be processed and input the noise data into a preset diffusion model; wherein, the diffusion model is a speech generation model composed of a specified number of encoder-decoder structures stacked together.

[0023] In this embodiment, the AI-based speech generation method runs on an electronic device (e.g., Figure 1The server / terminal device shown can acquire the noise data to be processed via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future wireless connection methods. The executing entity of this application is specifically a speech generation system, which can be simply referred to as the system. The aforementioned diffusion model is a speech generation model composed of a specified number (N layers) of encoder-decoder structures (Transformers) stacked together. When speech generation begins, following the diffusion model, the input noise data is used as initial data, and the input noise is gradually converted into speech through M time steps (total number of time steps). At the beginning of each time step, the relevant parameters and input data for the current time step are prepared. Furthermore, the specific implementation process of acquiring the noise data to be processed will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated upon here.

[0024] Furthermore, this application can be applied to speech generation scenarios in the fintech and digital healthcare fields. For example, in the insurance product recommendation speech generation scenario within the financial insurance sector, the aforementioned noise data can include: noise related to random operations and points of interest generated by customers browsing insurance product pages. For instance, customers may randomly click on different types of insurance product introductions (such as life insurance, health insurance, car insurance, etc.), the duration of their stay on each product page is random, and their level of attention to different clauses in the product terms (such as deductibles, payout ratios, waiting periods, etc.) also fluctuates randomly. These random behaviors and points of interest can be converted into noise data. For example, the customer's clicks on different insurance product types can be considered as a discrete random variable, with different product types corresponding to different values; the customer's stay on the product page can be considered as a continuous random variable, following a certain distribution (such as a normal distribution); the level of attention to product clauses can be represented by a random number between 0 and 1, where 0 represents no attention and 1 represents high attention.

[0025] In the context of health status reminder voice generation in the digital healthcare field, the aforementioned noise data can include: simulating random fluctuations and uncertainties in a patient's daily health monitoring data as noise. For example, data such as heart rate, blood pressure, and sleep quality collected by smart health devices worn by patients (such as smart bracelets and smart blood pressure monitors) will have certain random errors and fluctuations. Heart rate may fluctuate randomly within the normal range due to factors such as the patient's activity level and emotional state that day; blood pressure may also fluctuate slightly due to factors such as diet and ambient temperature; sleep quality data (such as sleep duration and the proportion of deep sleep) will also vary depending on the patient's sleep environment and psychological state that night. These random fluctuations and uncertainties in health data can constitute input noise. For example, heart rate data can be viewed as a random sequence that changes over time, with the heart rate value at each time point fluctuating around a baseline value; similarly, blood pressure data has its own random fluctuation range for both systolic and diastolic blood pressure.

[0026] Step S202: Based on the preset cache node judgment and dynamic caching and reuse decision strategy, the diffusion model is used to perform layer-by-layer calculation processing on the noise data corresponding to the specified number, and when the current time step reaches the preset cache node, the error corresponding to the specified encoder-decoder structure of the current time step is calculated.

[0027] In this embodiment, the layer-by-layer computation process includes: at each time step, the data is processed sequentially through N Transformer layers. For each Transformer layer, according to the order defined in the model, it first passes through the self-attention (Attn) layer, then through the feedforward network (FFN) layer, and so on. For example, for the first Transformer layer, the output of the previous time step (or the initial input noise if it is the first time step) is used as input, and the outputs of the Attn and FFN layers are calculated sequentially. The cache node determination includes: when a cache node is reached (i.e., the Attn and FFN layers corresponding to the current time step are recorded as cache nodes during the calibration phase), the error at the current time step is checked. The error here can be obtained by calculating the L1 relative error between the outputs of the corresponding layers at the current time step and the previous time step, using the same calculation method as in the calibration phase.

[0028] Step S203: If the error is less than a preset cache threshold, then the specified output result of the specified encoder-decoder structure cached in the previous time step corresponding to the current time step is obtained from the preset cache data structure and used as the output result of the current time step.

[0029] In this embodiment, if the detected error is less than the cache threshold α, the output results of the Attn layer and FFN layer cached in the previous time step are directly reused without recalculation. This avoids redundant calculations and saves computational resources and time. For example, if the error of a certain Attn layer in the current time step is lower than α, the output of that Attn layer in the previous time step is directly used as the output of the current time step.

[0030] Step S204: If the error is greater than or equal to the buffer threshold, the specified encoder-decoder structure of the current time step is recalculated, and the calculation result is used for subsequent processing of the current time step.

[0031] In this embodiment, if the detected error is greater than or equal to the buffer threshold α, the Attn layer and FFN layer are recalculated, and the calculation result is used for subsequent processing at the current time step, without buffering. This is to ensure accurate calculation results even with large errors, thereby guaranteeing speech quality. For example, if the error of a certain FFN layer at the current time step exceeds α, the output of that FFN layer is recalculated and used for the calculation of subsequent layers.

[0032] Step S205: Continue the calculation process for the next time step, and repeat the cache node judgment and dynamic caching and reuse decision process until the calculation of all time steps corresponding to the total number of time steps of the diffusion model is completed, and obtain the output of the last time step as the speech output result.

[0033] In this embodiment, the calculation process continues for the next time step, repeating the above-described cache node judgment and dynamic caching and reuse decision process, until all M time steps (i.e., all time steps corresponding to the total number of time steps in the diffusion model) are calculated. Finally, the output of the last time step is used as the generated speech output.

[0034] Dynamic caching and reuse during speech generation are crucial for accelerating the entire data inference process. By utilizing the cache nodes determined during the calibration phase and setting cache thresholds, the system dynamically decides whether to reuse cached results based on the actual situation during speech generation. This dynamic strategy minimizes redundant calculations and improves inference efficiency while ensuring speech quality. When the error is small, reusing cached results avoids unnecessary calculations and speeds up inference; when the error is large, recalculation ensures that speech quality remains unaffected. Through this dynamic balance, a balance is achieved between efficiency, sound quality, and deployability, providing an effective solution for scenarios with extremely high response speed requirements, such as real-time speech synthesis, intelligent voice assistants, and interactive dialogue systems.

[0035] Step S206: Perform output processing on the voice output result.

[0036] In this embodiment, the specific implementation process of outputting the voice output result described above will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0037] Based on the above automated processing flow, this application first acquires the noise data to be processed and inputs the noise data into a preset diffusion model; wherein, the diffusion model is a speech generation model composed of a specified number of encoder-decoder structures stacked together; then, based on a preset cache node judgment and dynamic caching and reuse decision strategy, the diffusion model is used to perform layer-by-layer calculation processing on the noise data corresponding to the specified number, and when the current time step reaches a preset cache node, the error corresponding to the specified encoder-decoder structure of the current time step is calculated; if the error is less than a preset cache threshold, the noise data corresponding to the specified encoder-decoder structure of the current time step is obtained from the preset cache data structure. The specified output result of the specified encoder-decoder structure cached in the previous time step corresponding to the previous time step is used as the output result of the current time step; if the error is greater than or equal to the cache threshold, the specified encoder-decoder structure of the current time step is recalculated, and the obtained calculation result is used for the subsequent processing of the current time step; the calculation processing of the next time step continues, and the cache node judgment and dynamic caching and reuse decision process are repeatedly executed until the calculation of all time steps corresponding to the total number of time steps of the diffusion model is completed, and the output of the last time step is obtained as the speech output result; finally, the speech output result is processed. Thus, this application adopts a diffusion-based TTS acceleration method based on selective caching. By caching and reusing part of the output of the encoder-decoder structure in the inference stage of speech generation, significant computational acceleration is achieved, effectively improving the efficiency of speech generation, while maintaining the naturalness and clarity of the speech, ensuring the quality of the generated speech.

[0038] In some alternative implementations, prior to step S202, the electronic device may also perform the following steps: Select speech samples from a pre-set speech dataset.

[0039] In this embodiment, the specific implementation process of selecting voice samples from the preset voice dataset will be described in further detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0040] The speech samples are input into the diffusion model, and inference calculations are performed at different time steps based on each encoder-decoder structure in the diffusion model to obtain the corresponding inference output results.

[0041] In this embodiment, inference computation includes: Model loading and initialization: The diffusion-based TTS model (diffusion model) is loaded into the computing environment and initialized. This includes setting model parameters, such as learning rate, batch size, etc., and loading pre-trained weights. This ensures the model is in a state where it can perform inference computation normally. Sample input: Selected speech samples are input into the diffusion model one by one. For each speech sample, it is processed according to the input format required by the model, for example, converting the speech signal into a feature representation such as a Mel spectrogram, as input data for the diffusion model. Time step division: Based on the generation process of the diffusion-based TTS model, the number of time steps M (total time steps) for inference computation is determined. The entire speech generation process is divided into M consecutive time steps, each time step corresponding to an intermediate state in the model generation process. Layer-by-layer computation process: For each time step, according to the structure of the diffusion model, each Transformer (encoder-decoder structure) layer is computed sequentially. In each Transformer layer, the output of the previous layer is first used as the input of the current layer, and then processed sequentially through modules such as the self-attention (Attn) layer and the feedforward network (FFN) layer. Self-attention layer computation: In the self-attention layer, the correlation weights between different positions in the input sequence are calculated, and the input sequence is weighted and summed according to these weights to obtain a new representation. This process involves calculating the query, key, and value matrices, as well as the attention weight matrix. Feedforward network layer computation: The output of the self-attention layer is input into the feedforward network layer for nonlinear transformation. The feedforward network layer typically consists of multiple fully connected layers, introducing nonlinearity through activation functions (such as ReLU) to extract and transform features from the input data. Result recording: After each time step and each Transformer layer has been computed, the output result of that layer at that time step is recorded. These output results will be used for subsequent error calculation and cache node determination.

[0042] Based on a preset error calculation strategy, the relative error between the outputs of the self-attention layer and the feedforward network layer in each encoder-decoder structure in adjacent time steps is calculated.

[0043] In this embodiment, the error calculation strategy includes the following: 1. Error calculation preparation: Before performing error calculation, ensure that the output results of the Attn and FFN layers in each Transformer module in adjacent time steps have been obtained. These output results are usually stored in the form of matrices or vectors. 2. L1 relative error calculation: Vector or matrix difference calculation: For adjacent time steps t and t + 1, the outputs of the Attn or FFN layer of a Transformer are A_t and A_{t+1}, respectively. First, calculate the absolute difference between these two outputs, i.e., |A_{t+1} - A_t|. If the output is a matrix, calculate the absolute difference for each element in the matrix; if it is a vector, calculate the absolute difference for each component in the vector. Reference value determination: Select a suitable reference value to calculate the relative error. Usually, the norm ‖A_t‖ of the output of the previous time step is selected as the reference value. The method of norm calculation can be selected according to the specific situation, such as L1 norm (sum of absolute values ​​of vector elements), L2 norm (square root of the sum of squares of vector elements), etc. Relative error calculation: The sum of absolute differences is divided by the reference value to obtain the L1 relative error. That is, ∑|A_{t+1} - A_t| / ‖A_t‖. If the output is a matrix, the sum of all elements of the absolute difference matrix is ​​first calculated, and then divided by the norm of the output matrix from the previous time step; if it is a vector, the sum of all components of the absolute difference vector is calculated, and then divided by the norm of the output vector from the previous time step. 3. Error result storage: The calculated L1 relative error results of each Transformer layer between adjacent time steps are stored for subsequent cache node determination.

[0044] Specifically, based on the aforementioned error calculation strategy, the relative error between the outputs of the self-attention layer and the feedforward network layer in each encoder-decoder structure at adjacent time steps can be calculated.

[0045] The relative error is compared with the cache threshold to filter out target time steps with a relative error less than the cache threshold, as well as target self-attention layers and target feedforward network layers corresponding to the target time steps.

[0046] In this embodiment, the error is compared with the cache threshold, and the time steps that are finally selected below the cache threshold α, along with the corresponding Attn layer and FFN layer, are recorded as cache nodes.

[0047] The process of determining the buffer threshold may include: 1. Error curve analysis: Based on the inter-layer error curves calculated during the calibration phase, carefully observe the changes in error with time steps and layers. Analyze the trend of the error curves to identify areas where the error is relatively stable and areas with large fluctuations. For example, the errors at some time steps and layers may remain at a low level, while the errors at other time steps and layers may fluctuate significantly. 2. Quality and efficiency considerations: Set the buffer threshold α by combining comprehensive considerations of speech quality and computational efficiency. From the perspective of speech quality, it is desirable to have a threshold as small as possible to ensure that no significant errors are introduced when reusing buffered results, thereby maintaining the naturalness, clarity, and speaker similarity of the speech. From the perspective of computational efficiency, it is desirable to have a threshold as large as possible to increase the number of nodes that can be buffered and improve inference speed. For example, the threshold can be gradually adjusted experimentally to observe the changes in speech quality and the improvement in inference speed under different thresholds. 3. Threshold determination: After comprehensively considering the above factors, determine a suitable buffer threshold α. This threshold will serve as the key criterion for determining whether to perform buffering in subsequent speech generation processes. For example, after multiple experiments and evaluations, it was found that when the threshold is set to 0.1, the inference speed can be significantly improved while ensuring that the speech quality is basically unaffected. Therefore, α can be set to 0.1.

[0048] Based on the target time step, the target self-attention layer, and the target feedforward network layer, a cache node corresponding to the diffusion model is generated.

[0049] In this embodiment, data structures (such as lists, dictionaries, etc.) can be used to store information about these cache nodes, including time step numbers, layer numbers, and corresponding errors. These records will serve as cache nodes in the subsequent speech generation process, guiding dynamic caching and reuse operations.

[0050] Based on the above processing flow, this application selects speech samples from a pre-set speech dataset; then, the speech samples are input into a diffusion model, and inference calculations are performed at different time steps based on each layer of the encoder-decoder structure in the diffusion model to obtain the corresponding inference output results; then, based on a pre-set error calculation strategy, the relative error between the outputs of the self-attention layer and the feedforward network layer in each layer of the encoder-decoder structure in adjacent time steps is calculated; subsequently, the relative error is compared with a buffer threshold to filter out the target time step with a relative error less than the buffer threshold, as well as the target self-attention layer and the target feedforward network layer corresponding to the target time step; finally, a buffer node corresponding to the diffusion model is generated based on the target time step, the target self-attention layer, and the target feedforward network layer. The calibration stage of this application is the cornerstone of the entire data inference process. Its purpose is to identify those parts with relatively stable and less changing calculation results by analyzing the calculation errors of the diffusion model under different conditions. Since the diffusion model needs to repeatedly execute the calculation of the Transformer layer at multiple time steps during speech generation, the calculation results of some time steps and layers may have high similarity. Furthermore, by identifying cache nodes, this application allows for the direct reuse of previous calculation results for highly similar parts during subsequent speech generation, thereby avoiding redundant calculations and improving inference efficiency. Simultaneously, setting a reasonable error threshold ensures that while accelerating computation, it does not excessively impact speech quality.

[0051] In some optional implementations of this embodiment, selecting speech samples from a preset speech dataset includes the following steps: Obtain the scene information corresponding to the diffusion model.

[0052] In this embodiment, the above-mentioned scenario information may refer to the above-mentioned diffusion model, the application scenario and target of the diffusion-type TTS model.

[0053] The corresponding speech feature dimensions are determined based on the scene information.

[0054] In this embodiment, the key speech feature dimensions to be covered, namely the aforementioned speech feature dimensions, can be determined based on the application scenario and objectives of the diffusion model. For example, if the diffusion model is mainly used in intelligent customer service scenarios, then the speaker's speaking speed, clarity of tone, and common expressions are more important; if the diffusion model is used for audiobook production, then pronunciation style, emotional expression, and other aspects need to be given priority consideration.

[0055] A corresponding sample selection strategy is constructed based on the aforementioned speech feature dimensions.

[0056] In this embodiment, a specific sample selection strategy can be formulated based on determined speech feature dimensions. Specifically, a combination of random sampling and targeted selection can be used. First, a certain proportion of samples are randomly selected from the speech dataset. Then, representative speech samples are further selected based on preset feature requirements, such as different genders, age groups, and speech rate ranges. For example, the ratio of male to female speakers is set to 1:1, the age groups cover three intervals: 20-30 years old, 30-40 years old, and 40-50 years old, and the speech rate is between 120 and 180 words per minute. Samples meeting these criteria are then selected.

[0057] Based on the sample selection strategy, the preset speech dataset is selected to obtain the corresponding initial speech samples, and the initial speech samples are then verified.

[0058] In this embodiment, an existing speech dataset can be comprehensively evaluated to understand its size, diversity, and distribution of speech features. This includes examining the number of speakers, gender ratio, age distribution, and the type of speech content (e.g., news broadcasts, everyday conversations, poetry recitations) and language style (formal, informal, technical terms, etc.). Then, based on the constructed sample selection strategy, the aforementioned speech dataset is used to select initial speech samples.

[0059] The initial speech samples selected can be further verified to ensure they indeed cover the required speech features. This can be done through manual listening to check if the samples meet the preset feature requirements, or by using speech feature analysis tools to extract speech feature parameters such as fundamental frequency, duration, and energy from the initial speech samples for quantitative analysis. If the initial speech sample is found to meet the preset feature requirements, it is considered to have passed the verification; otherwise, it is considered to have failed the verification.

[0060] If the initial speech sample passes the verification, then the initial speech sample is used as the speech sample.

[0061] Based on the above processing flow, this application obtains scene information corresponding to the diffusion model; then determines the corresponding speech feature dimensions based on the scene information; subsequently, it constructs a corresponding sample selection strategy based on the speech feature dimensions; next, it filters the preset speech dataset based on the sample selection strategy to obtain corresponding initial speech samples, and verifies the initial speech samples; if the initial speech samples pass the verification, they are used as speech samples. Thus, by using the sample selection strategy, this application can automatically and accurately select representative speech samples from the speech dataset, improving the intelligence of speech sample selection and ensuring the accuracy and diversity of the obtained speech samples.

[0062] In some alternative implementations, obtaining the noise data to be processed in step S203 includes the following steps: Obtain the preset noise generation strategy.

[0063] In this embodiment, the source and type of input noise can be determined. A suitable noise generation method (noise generation strategy), such as a random noise generation algorithm, can be selected based on the requirements of the diffusion model. Parameters such as the noise's dimension and distribution can be set to meet the model's input requirements. For example, in a diffusion model for speech synthesis, the noise's dimension typically matches the dimension of the speech signal, and the distribution can be a Gaussian distribution, etc. The noise generation strategy may include using a random number generator to generate noise, or utilizing physical noise sources to generate noise.

[0064] Specifically, generating noise using a random number generator includes: Uniform noise generation: If uniform noise is chosen as the input noise, a uniform random number generation function can be used. The generation range of uniform random numbers can be set as needed. For example, when generating a one-dimensional uniform noise sequence of length N, a minimum and maximum value can be specified, and then random numbers within this range can be generated. Gaussian noise generation: For Gaussian noise, a Gaussian random number generation function is used. Based on the set mean and standard deviation, a random number sequence conforming to a Gaussian distribution is generated. When generating a two-dimensional noise matrix (such as a noise matrix the size of a Mel spectrogram), Gaussian random numbers can be generated for each element of the matrix separately.

[0065] Noise generation using physical noise sources includes: Hardware acquisition: If using physical noise sources, corresponding hardware devices, such as noise generators or radio telescopes, need to be connected. The acquired analog noise signal is converted into a digital signal through the hardware device's interface. Signal processing: The acquired digital noise signal is preprocessed, such as through filtering and amplification, to remove unwanted interference signals and adjust the signal's amplitude and frequency range to meet the requirements of the diffusion model input.

[0066] Initial noise is generated based on the noise generation strategy.

[0067] In this embodiment, noise generation processing can be performed according to the content of the selected noise generation strategy to generate corresponding initial noise.

[0068] The initial noise is preprocessed based on a preset processing strategy to obtain the corresponding processed noise.

[0069] In this embodiment, the above-mentioned preprocessing of the initial noise based on the preset processing strategy to obtain the corresponding processed noise is described in more detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0070] The processed noise is used as the noise data.

[0071] Based on the above processing flow, this application obtains a preset noise generation strategy; then generates initial noise based on the noise generation strategy; subsequently, it preprocesses the initial noise based on a preset processing strategy to obtain corresponding processed noise; and finally, it uses the processed noise as noise data. This application effectively improves the data quality of the generated noise data by generating initial noise based on the noise generation strategy and then preprocessing the initial noise based on the processing strategy to generate the final noise data.

[0072] In some optional implementations, the preprocessing of the initial noise based on a preset processing strategy to obtain the corresponding processed noise includes the following steps: The initial noise is normalized to obtain the corresponding first noise.

[0073] In this embodiment, the normalization operation includes: determining the normalization range: determining the normalization range of noise values ​​according to the input requirements of the diffusion model. Common normalization ranges include , , etc. For example, if the model requires the input noise values ​​to be between , then the generated noise needs to be normalized accordingly. Calculating normalization parameters: for a one-dimensional noise sequence, calculate its maximum and minimum values. Then, calculate the normalization parameters according to the normalization range. For example, to normalize the noise sequence to between , the formula can be used, where is the normalized noise sequence, and and are the minimum and maximum values ​​of the original noise sequence, respectively. Applying normalization: applying the calculated normalization parameters to the noise sequence or matrix, performing normalization processing on each element. For a two-dimensional noise matrix, it is also necessary to perform normalization operations on each element of the matrix to ensure that the numerical range of the entire matrix meets the requirements of the model.

[0074] The first noise is filtered to obtain the corresponding second noise.

[0075] In this embodiment, the filtering process includes: if the generated noise contains high-frequency interference or low-frequency noise, a filter can be used for filtering. For example, a low-pass filter can be used to remove high-frequency interference, and a high-pass filter can be used to remove low-frequency noise. The filter parameters can be set according to the characteristics of the noise and the requirements of the model.

[0076] The second noise is whitened to obtain the corresponding third noise.

[0077] In this embodiment, the whitening process includes: in some cases, further whitening can be performed to improve the statistical properties of the noise. Whitening can make the various components of the noise uncorrelated and have the same variance. Whitening methods include principal component analysis (PCA) whitening and independent component analysis (ICA) whitening. Appropriate whitening methods can be selected based on actual needs to improve the training effect and generation quality of the diffusion model.

[0078] The third noise is used as the processing noise.

[0079] Based on the above processing flow, this application normalizes the initial noise to obtain a first noise; then filters the first noise to obtain a second noise; subsequently, it whitens the second noise to obtain a third noise; and finally, the third noise is used as the processed noise. Thus, by normalizing, filtering, and whitening the initial noise, this application can automatically and accurately preprocess the initial noise, thereby improving the data quality of the generated noise data.

[0080] In some optional implementations of this embodiment, step S206 includes the following steps: The speech output is optimized based on a preset speech optimization strategy to obtain the corresponding target speech data.

[0081] In this embodiment, the specific implementation process of optimizing the speech output result based on the preset speech optimization strategy to obtain the corresponding target speech data will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0082] Get the preset output method.

[0083] In this embodiment, the selection of the above output method is not specifically limited and can be determined according to the actual business needs. For example, any one of the output methods such as email sending, information sending, and interface display can be used.

[0084] The target speech data is output and processed based on the aforementioned output method.

[0085] In this embodiment, the target speech data can be processed according to the selected output method to complete the corresponding output processing.

[0086] Based on the above processing flow, this application optimizes the speech output result using a preset speech optimization strategy to obtain the corresponding target speech data; then, it obtains a preset output method; subsequently, it outputs the target speech data based on the output method. Thus, this application effectively improves the speech quality of the generated target speech data by optimizing the speech output result using a speech optimization strategy. Furthermore, by using the output method to output the target speech data, it improves the accuracy and intelligence of the target speech data output.

[0087] In some optional implementations of this embodiment, before step S201, the electronic device may further perform the following steps: The voice output result is processed by format conversion to obtain the corresponding first voice data.

[0088] In this embodiment, the above format conversion process includes: 1. Sampling rate adjustment: Determining the sampling rate of the target audio format. If the sampling rate of the generated audio (audio is the speech output) differs from the target sampling rate, sampling rate conversion is required. For example, if the generated audio sampling rate is 22050Hz, while the target sampling rate is 44100Hz, interpolation methods (such as linear interpolation, spline interpolation, etc.) are used to increase the number of sampling points to improve the frequency range and clarity of the audio. Conversely, if the target sampling rate is lower than the sampling rate of the generated audio, decimation methods (such as simple decimation, weighted decimation, etc.) can be used to reduce the number of sampling points, but it is necessary to ensure that excessive aliasing distortion is not introduced during the decimation process. 2. Bit depth conversion: Understanding the bit depth requirements of the target audio format. Common bit depths include 16-bit and 24-bit. If the bit depth of the generated audio differs from the target bit depth, quantization processing is required. For example, when quantizing a 32-bit floating-point audio signal into a 16-bit integer, appropriate scaling and rounding operations are required based on the range of the audio signal to avoid excessive quantization errors. 3. Channel Count Adjustment: Determine the number of channels in the target audio format (e.g., mono, stereo, etc.). If the generated audio is mono and the target format is stereo, a stereo effect can be generated by copying the mono signal to the left and right channels. If the generated audio is stereo and the target format is mono, the left and right channel signals can be averaged or weighted averaged to obtain a mono signal.

[0089] The volume of the first voice data is adjusted to obtain the corresponding second voice data.

[0090] In this embodiment, the volume adjustment process includes: 1. Volume analysis: Calculating the volume level of the generated audio. Several volume measurement methods can be used, such as root mean square (RMS) value, peak level, etc. The RMS value reflects the average energy of the audio signal, while the peak level represents the maximum instantaneous amplitude of the audio signal. The required adjustment amplitude is determined based on the target volume requirement. For example, if the target volume is -10dB, and the current audio's RMS value corresponds to a decibel value lower than -10dB, the volume needs to be increased; conversely, if it is higher than -10dB, the volume needs to be decreased. 2. Volume adjustment operation: For simple volume adjustments, linear amplification or reduction methods can be used. Each sample point of the audio signal is multiplied by a gain coefficient, which is calculated based on the difference between the target volume and the current volume. For example, if the volume needs to be increased by 3dB, the gain coefficient is approximately 1.414 (because 3dB corresponds to a doubling of power, or a √2-fold increase in amplitude). When adjusting the volume, care is taken to avoid clipping distortion. If the gain factor is too large, the amplitude of the audio signal may exceed the maximum value that the audio format can represent, resulting in clipping distortion. Therefore, when adjusting the volume, the adjusted signal is further limited to ensure that the signal amplitude is within a reasonable range.

[0091] The second speech data is subjected to quality optimization processing to obtain the corresponding third speech data.

[0092] In this embodiment, the above-mentioned quality optimization process includes: 1. Quality assessment: Using objective evaluation metrics, such as signal-to-noise ratio (SNR) and perceived speech quality assessment (PESQ), the quality of the processed audio is evaluated. SNR measures the ratio of effective signal to noise in an audio signal, while PESQ simulates the human auditory system's perception of speech quality. Subjective listening tests are conducted. A group of listeners is invited to rate the processed audio, and the audio quality is evaluated based on their feedback. Subjective listening tests provide more intuitive evaluation results that better reflect human auditory perception. 2. Optimization processing: Based on the quality assessment results, the audio is further optimized. If noise is found in the audio, noise reduction algorithms (such as spectral subtraction, Wiener filtering, etc.) can be used to reduce noise. If the audio clarity is insufficient, speech enhancement algorithms (such as short-time spectral estimation enhancement, harmonic enhancement, etc.) can be used to improve speech clarity. Furthermore, based on listener feedback, adjustments can be made to volume, timbre, etc., to achieve a better auditory effect.

[0093] The third voice data is used as the target voice data.

[0094] Based on the above processing flow, this application performs format conversion on the speech output to obtain the corresponding first speech data; then, it adjusts the volume of the first speech data to obtain the corresponding second speech data; subsequently, it optimizes the quality of the second speech data to obtain the corresponding third speech data; and finally, it uses the third speech data as the target speech data. Thus, by performing format conversion, volume adjustment, and quality optimization on the speech output, this application can automatically and accurately optimize the speech output, effectively improving the speech quality and accuracy of the final output target speech data.

[0095] In some alternative implementations, the user information obtained is subject to user consent and complies with relevant laws and policies.

[0096] Furthermore, any software tools or components not belonging to our company that appear in the embodiments of this application are merely illustrative examples and do not represent actual use.

[0097] Furthermore, unlike knowledge distillation or structural pruning methods, this application operates entirely during the inference phase without altering the original model parameters, thus possessing plug-and-play characteristics and being directly applicable to existing diffusion TTS frameworks. This significantly reduces deployment costs and improves the practical feasibility of the solution. Moreover, this application can effectively accelerate the inference process of diffusion-based TTS systems without changing the model structure or requiring retraining. By selectively caching the Transformer layer output, the system significantly reduces redundant computation, thereby lowering latency and hardware power consumption. This is of great significance for scenarios with extremely high response speed requirements, such as real-time speech synthesis, intelligent voice assistants, and interactive dialogue systems. Compared to methods that simply reduce denoising steps, this application maintains the naturalness, clarity, and speaker similarity of speech while ensuring inference speed, without introducing significant distortion or sound quality degradation due to excessive compression computation. Furthermore, because this solution is based on calibration-driven dynamic scheduling, it can automatically adjust the cache range according to different data distributions and scenarios, exhibiting stronger adaptability and robustness. Overall, this application achieves a balance in terms of efficiency, sound quality, and deployability, meeting the needs of efficient reasoning while ensuring high-quality voice output, and providing solid support for the next generation of intelligent voice interaction technology.

[0098] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0099] It should be emphasized that, in order to further ensure the privacy and security of the above voice output results, the above voice output results can also be stored in a blockchain node.

[0100] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0101] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0102] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0103] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0104] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of an artificial intelligence-based speech generation device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0105] like Figure 3 As shown, the AI-based speech generation device 300 described in this embodiment includes: a first processing module 301, a second processing module 302, an acquisition module 303, a first calculation module 304, a third processing module 305, and an output module 306. Wherein: The first processing module 301 is used to acquire noise data to be processed and input the noise data into a preset diffusion model; wherein, the diffusion model is a speech generation model composed of a specified number of encoder-decoder structures stacked together. The second processing module 302 is used to perform layer-by-layer calculation processing on the noise data corresponding to the specified number using the diffusion model based on the preset cache node judgment and dynamic caching and reuse decision strategy, and to calculate the error corresponding to the specified encoder-decoder structure of the current time step when the current time step is detected to have reached the preset cache node. The acquisition module 303 is used to, if the error is less than a preset cache threshold, acquire from a preset cache data structure the specified output result of the specified encoder-decoder structure cached in the previous time step corresponding to the current time step as the output result of the current time step. The first calculation module 304 is used to recalculate the specified encoder-decoder structure of the current time step if the error is greater than or equal to the cache threshold, and use the obtained calculation result for the subsequent processing of the current time step. The third processing module 305 is used to continue the calculation processing of the next time step, and repeatedly execute the cache node judgment and dynamic caching and reuse decision process until the calculation of all time steps corresponding to the total number of time steps of the diffusion model is completed, and the output of the last time step is obtained as the speech output result. The output module 306 is used to process the voice output result.

[0106] In some optional implementations of this embodiment, the AI-based speech generation device further includes: The selection module is used to select speech samples from a preset speech dataset; The inference module is used to input the speech samples into the diffusion model, perform inference calculations based on each encoder-decoder structure in the diffusion model at different time steps, and obtain the corresponding inference output results. The second calculation module is used to calculate the relative error between the outputs of the self-attention layer and the feedforward network layer in each encoder-decoder structure in adjacent time steps based on a preset error calculation strategy. The filtering module is used to compare the relative error with the cache threshold to filter out the target time step whose relative error is less than the cache threshold, as well as the target self-attention layer and the target feedforward network layer corresponding to the target time step; The generation module is used to generate cache nodes corresponding to the diffusion model based on the target time step, the target self-attention layer, and the target feedforward network layer.

[0107] In some optional implementations of this embodiment, the selected modules include: The first acquisition submodule is used to acquire scene information corresponding to the diffusion model; The first determining submodule is used to determine the corresponding speech feature dimension based on the scene information; A submodule is constructed to build a corresponding sample selection strategy based on the speech feature dimension. The filtering submodule is used to filter the preset speech dataset based on the sample filtering strategy to obtain the corresponding initial speech samples, and to verify the initial speech samples. The second determining submodule is used to use the initial speech sample as the speech sample if the initial speech sample passes the verification.

[0108] In some optional implementations of this embodiment, the first processing module 301 includes: The second acquisition submodule is used to acquire the preset noise generation strategy; A generation submodule is used to generate initial noise based on the noise generation strategy; The preprocessing submodule is used to preprocess the initial noise based on a preset processing strategy to obtain the corresponding processed noise; The third determining submodule is used to treat the processed noise as the noise data.

[0109] In some optional implementations of this embodiment, the preprocessing submodule includes: An operation unit is used to perform a normalization operation on the initial noise to obtain the corresponding first noise; The first processing unit is used to filter the first noise to obtain the corresponding second noise; The second processing unit is used to whiten the second noise to obtain the corresponding third noise; The first determining unit is used to determine the third noise as the processing noise.

[0110] In some optional implementations of this embodiment, the output module 306 includes: The optimization submodule is used to optimize the speech output result based on a preset speech optimization strategy to obtain the corresponding target speech data. The third acquisition submodule is used to acquire preset output methods; The output submodule is used to process the target speech data based on the output method.

[0111] In some optional implementations of this embodiment, the optimized submodule includes: The third processing unit is used to perform format conversion processing on the voice output result to obtain the corresponding first voice data. The fourth processing unit is used to perform volume adjustment processing on the first voice data to obtain the corresponding second voice data; The fifth processing unit is used to perform quality optimization processing on the second speech data to obtain the corresponding third speech data; The second determining unit is used to use the third speech data as the target speech data.

[0112] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0113] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0114] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0115] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for speech generation methods based on artificial intelligence. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0116] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions of the artificial intelligence-based speech generation method.

[0117] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0118] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the artificial intelligence-based speech generation method described above.

[0119] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0120] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A speech generation method based on artificial intelligence, characterized in that, Includes the following steps: Acquire noise data to be processed and input the noise data into a preset diffusion model; wherein, the diffusion model is a speech generation model composed of a specified number of encoder-decoder structures stacked together; Based on the preset cache node judgment and dynamic caching and reuse decision strategy, the diffusion model is used to perform layer-by-layer calculation processing on the noise data corresponding to the specified number, and when the current time step reaches the preset cache node, the error corresponding to the specified encoder-decoder structure of the current time step is calculated. If the error is less than a preset cache threshold, then the specified output result of the specified encoder-decoder structure cached in the previous time step corresponding to the current time step is obtained from the preset cache data structure and used as the output result of the current time step. If the error is greater than or equal to the cache threshold, the specified encoder-decoder structure of the current time step is recalculated, and the calculation result is used for subsequent processing of the current time step. Continue the calculation and processing for the next time step, and repeat the cache node judgment and dynamic caching and reuse decision process until the calculation of all time steps corresponding to the total number of time steps of the diffusion model is completed, and obtain the output of the last time step as the speech output result. The voice output result is processed for output.

2. The artificial intelligence-based speech generation method according to claim 1, characterized in that, Before the steps of using the diffusion model to perform layer-by-layer calculation processing on the noise data corresponding to the specified number of times based on the preset cache node judgment and dynamic caching and reuse decision strategy, and calculating the error corresponding to the specified encoder-decoder structure of the current time step when the preset cache node is detected, the method further includes: Select speech samples from a pre-set speech dataset; The speech sample is input into the diffusion model, and inference calculations are performed at different time steps based on each encoder-decoder structure in the diffusion model to obtain the corresponding inference output results. Based on a preset error calculation strategy, the relative error between the outputs of the self-attention layer and the feedforward network layer in each encoder-decoder structure in adjacent time steps is calculated. The relative error is compared with the cache threshold to filter out target time steps with a relative error less than the cache threshold, as well as target self-attention layers and target feedforward network layers corresponding to the target time steps; Based on the target time step, the target self-attention layer, and the target feedforward network layer, a cache node corresponding to the diffusion model is generated.

3. The artificial intelligence-based speech generation method according to claim 2, characterized in that, The step of selecting speech samples from a preset speech dataset specifically includes: Obtain scene information corresponding to the diffusion model; Determine the corresponding speech feature dimensions based on the scene information; A corresponding sample selection strategy is constructed based on the aforementioned speech feature dimensions; Based on the sample selection strategy, the preset speech dataset is selected to obtain the corresponding initial speech samples, and the initial speech samples are then verified. If the initial speech sample passes the verification, then the initial speech sample is used as the speech sample.

4. The artificial intelligence-based speech generation method according to claim 1, characterized in that, The step of acquiring the noise data to be processed specifically includes: Obtain the preset noise generation strategy; Initial noise is generated based on the aforementioned noise generation strategy; The initial noise is preprocessed based on a preset processing strategy to obtain the corresponding processed noise; The processed noise is used as the noise data.

5. The artificial intelligence-based speech generation method according to claim 4, characterized in that, The step of preprocessing the initial noise based on a preset processing strategy to obtain the corresponding processed noise specifically includes: The initial noise is normalized to obtain the corresponding first noise; The first noise is filtered to obtain the corresponding second noise. The second noise is whitened to obtain the corresponding third noise; The third noise is used as the processing noise.

6. The artificial intelligence-based speech generation method according to claim 1, characterized in that, The step of outputting the voice output result specifically includes: The speech output is optimized based on a preset speech optimization strategy to obtain the corresponding target speech data. Get the preset output mode; The target speech data is output and processed based on the aforementioned output method.

7. The artificial intelligence-based speech generation method according to claim 6, characterized in that, The step of optimizing the speech output based on a preset speech optimization strategy to obtain the corresponding target speech data specifically includes: The voice output result is processed by format conversion to obtain the corresponding first voice data; The volume of the first voice data is adjusted to obtain the corresponding second voice data; The second speech data is subjected to quality optimization processing to obtain the corresponding third speech data; The third voice data is used as the target voice data.

8. A speech generation device based on artificial intelligence, characterized in that, include: The first processing module is used to acquire noise data to be processed and input the noise data into a preset diffusion model; wherein, the diffusion model is a speech generation model composed of a specified number of encoder-decoder structures stacked together; The second processing module is used to perform layer-by-layer calculation processing on the noise data corresponding to the specified number using the diffusion model based on the preset cache node judgment and dynamic caching and reuse decision strategy, and to calculate the error corresponding to the specified encoder-decoder structure of the current time step when the current time step is detected to have reached the preset cache node. The acquisition module is used to acquire, if the error is less than a preset cache threshold, a specified output result of the specified encoder-decoder structure cached in the previous time step corresponding to the current time step from a preset cache data structure and use it as the output result of the current time step. The first calculation module is used to recalculate the specified encoder-decoder structure of the current time step if the error is greater than or equal to the cache threshold, and use the calculation result for subsequent processing of the current time step. The third processing module is used to continue the calculation and processing of the next time step, and repeatedly execute the cache node judgment and dynamic caching and reuse decision process until the calculation of all time steps corresponding to the total number of time steps of the diffusion model is completed, and the output of the last time step is obtained as the speech output result. The output module is used to process the voice output results.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the artificial intelligence-based speech generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the artificial intelligence-based speech generation method as described in any one of claims 1 to 7.

Images