A data reasoning method and a data reasoning device

Abstract

This application provides a data inference method and a data inference apparatus. The method includes: acquiring data to be processed; inputting the data to be processed into a pre-trained encoder to obtain a first implicit feature output by the pre-trained encoder; wherein the pre-trained encoder is jointly trained by a data distribution transformation model and a discriminant model, and the first implicit feature follows a first continuous probability distribution; inputting the first implicit feature into a decoder to restore the first implicit feature through the decoder to obtain first generated data. Through this method and apparatus, instead of relying on traditional Mel-spectrum as an intermediate representation, implicit features are used, which can better preserve the feature information of the original data, thereby improving the model's generation capability, achieving higher accuracy and more diverse generation effects, and solving the information loss problem existing in traditional generation models.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data reasoning method and a data reasoning device. Background Technology

[0002] With the widespread application of deep learning technology in audio processing, intermediate representations based on Mel-spectrum data are commonly used for sound or music generation. Mel-spectrum analysis is a frequency analysis method that aligns with human auditory characteristics, converting raw audio signals into frequency domain features more suitable for machine learning models. A typical audio generation process involves first converting the raw audio into a Mel-spectrum using a short-time Fourier transform; then, using a deep learning model to model and generate the Mel-spectrum; and finally, using a vocoder to reconstruct the time-domain audio signal from the generated Mel-spectrum. This intermediate representation-based approach has become the mainstream technology in audio generation, effectively reducing computational complexity and achieving significant results in applications such as speech synthesis and music generation.

[0003] However, during the conversion between audio signals and Mel spectra, as well as further transformations based on Mel spectra, information loss is inevitably introduced, while data processing overhead is increased. This loss, under the cumulative effect of multiple conversions, will significantly reduce audio quality, thereby affecting the final output of the generative model. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a data inference method and a data inference device that uses implicit features instead of traditional Mel spectra as an intermediate representation. Implicit features can better preserve the feature information of the original data and avoid the error effects of extracting Mel spectra from data and restoring data from Mel spectra. Furthermore, the pre-trained encoder adds a data distribution transformation model during the training phase, so that the implicit features generated by the model do not have to directly conform to the standard data distribution, thereby increasing the amount of information carried by the implicit features and thus improving the model's generation capability. This achieves higher accuracy and more diverse generation effects, solving the information loss problem existing in traditional generation models.

[0005] In a first aspect, embodiments of this application provide a data reasoning method, the data reasoning method comprising:

[0006] The data to be processed is acquired and input into a pre-trained encoder to obtain the first implicit feature output by the pre-trained encoder; wherein the pre-trained encoder is trained by a data distribution transformation model and a discrimination model, and the first implicit feature follows a first continuous probability distribution.

[0007] The first implicit feature is input into the decoder so that the first implicit feature can be restored by the decoder to obtain the first generated data.

[0008] Furthermore, the pre-trained encoder is obtained through the following steps:

[0009] The sample data is input into the encoder to be trained to obtain the implicit feature samples output by the encoder to be trained;

[0010] The implicit feature samples are input into the data distribution transformation model to convert the implicit feature samples into second implicit features that follow a second continuous probability distribution.

[0011] The discriminative model establishes a loss function based on the difference between the second implicit feature and the standard probability distribution, and adjusts the parameters of the encoder to be trained based on the result of the loss function until the training conditions are met, thus obtaining the pre-trained encoder.

[0012] Furthermore, when the first continuous probability distribution is a Gaussian distribution, the second continuous probability distribution is a standard Gaussian distribution, and the data distribution transformation model is used to convert the implicit feature samples that conform to the Gaussian distribution into second implicit features that conform to the standard Gaussian distribution.

[0013] Furthermore, the discriminant model is based on a generative adversarial network architecture, and the discriminant model is used to determine whether the second implicit feature belongs to the standard distribution data; wherein the standard distribution data is generated by the generator in the generative adversarial network, or sampled from the standard distribution data.

[0014] Furthermore, the discriminant model is a generative adversarial network (GAN), used to determine the probability that the second continuous probability distribution is standard distribution data, and the loss function is KL divergence or JS divergence; or,

[0015] The discriminant model is a Wasserstein generative adversarial network, which is used to determine the distribution gap between the second continuous probability distribution and the standard distribution data. The loss function is the Wasserstein distance.

[0016] Furthermore, after obtaining the first implicit feature output by the pre-trained encoder, the data inference method further includes:

[0017] Obtain at least one target implicit feature corresponding to the data to be processed;

[0018] The at least one target implicit feature and the first implicit feature are simultaneously input into the generative model to obtain the third implicit feature output by the generative model.

[0019] The third implicit feature is input into the decoder so that the decoder can restore the third implicit feature to obtain the second generated data.

[0020] Furthermore, the data to be processed is audio data, the decoder is a neural vocoder, and the implicit feature is a latent space audio vector.

[0021] Furthermore, the data inference method is implemented based on a streaming model, and the pre-trained encoder is based on a continuous flow matching architecture, which maps the data to be processed to the latent space through reversible mapping to obtain the first implicit features.

[0022] Furthermore, the latent space includes at least one latent subspace, each of which independently represents audio latent features of different dimensions.

[0023] Secondly, embodiments of this application also provide a data inference apparatus, the data inference apparatus comprising:

[0024] An implicit feature generation module is used to acquire data to be processed, input the data to be processed into a pre-trained encoder, and obtain the first implicit feature output by the pre-trained encoder; wherein, the pre-trained encoder is jointly trained by a data distribution transformation model and a discrimination model, and the first implicit feature follows a first continuous probability distribution;

[0025] The first data restoration module is used to input the first implicit feature into the decoder so that the first implicit feature can be restored by the decoder to obtain the first generated data.

[0026] This application provides a data inference method and data inference device. First, data to be processed is acquired and input into a pre-trained encoder to obtain a first implicit feature output by the pre-trained encoder. The pre-trained encoder is trained by a data distribution transformation model and a discriminant model, and the first implicit feature follows a first continuous probability distribution. Then, the first implicit feature is input into a decoder to restore the first implicit feature and obtain first generated data.

[0027] This application uses implicit features instead of traditional Mel spectra as an intermediate representation, which better preserves the feature information of the original data. Furthermore, the implicit features generated by the pre-trained encoder follow a first continuous probability distribution, rather than directly making the latent space variables follow a standard continuous probability distribution, thus carrying more information and avoiding the error effects of extracting Mel spectra from data and reconstructing data from Mel spectra. In addition, the pre-trained encoder incorporates a data distribution transformation model during the training phase, so that the implicit features generated by the model do not necessarily directly follow a standard data distribution, increasing the amount of information carried by the latent space vectors, thereby improving the model's generation capability and achieving higher accuracy and more diverse generation effects, solving the information loss problem existing in traditional generative models.

[0028] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A flowchart illustrating a data reasoning method provided in an embodiment of this application;

[0031] Figure 2 This is one of the structural schematic diagrams of a data inference device provided in the embodiments of this application;

[0032] Figure 3 This is a second schematic diagram of the structure of a data inference device provided in an embodiment of this application;

[0033] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0035] First, the applicable scenarios for this application will be introduced. This application can be applied to the field of data processing technology.

[0036] With the widespread application of deep learning technology in audio processing, intermediate representations based on Mel-spectrum data are commonly used for sound or music generation. Mel-spectrum analysis is a frequency analysis method that aligns with human auditory characteristics, converting raw audio signals into frequency domain features more suitable for machine learning models. A typical audio generation process involves first converting the raw audio into a Mel-spectrum using a short-time Fourier transform; then, using a deep learning model to model and generate the Mel-spectrum; and finally, using a vocoder to reconstruct the time-domain audio signal from the generated Mel-spectrum. This intermediate representation-based approach has become the mainstream technology in audio generation, effectively reducing computational complexity and achieving significant results in applications such as speech synthesis and music generation.

[0037] Research has found that information loss is inevitably introduced during the conversion between audio signals and Mel spectra, as well as further transformations based on Mel spectra, while increasing data processing overhead. This loss, under the cumulative effect of multiple conversions, will significantly reduce audio quality, thereby affecting the final output of the generative model.

[0038] Based on this, the embodiments of this application provide a data reasoning method to improve the model's generation capability, achieve higher accuracy and more diverse generation effects, and solve the information loss problem existing in traditional generation models.

[0039] Please see Figure 1 , Figure 1 This is a flowchart illustrating a data reasoning method provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the data reasoning method includes:

[0040] S101, Obtain the data to be processed, input the data to be processed into the pre-trained encoder, and obtain the first implicit feature output by the pre-trained encoder.

[0041] Regarding step S101 above, in specific implementation, the data to be processed is acquired and input into a pre-trained encoder, which generates implicit features. Here, the implicit features can be latent space variables, and they follow a first continuous probability distribution, thus obtaining the first implicit features output by the pre-trained encoder. For example, taking data inference methods applied to the fields of sound and music, the implicit features are not traditional explicit features (such as Mel spectrum, MFCC, etc.) directly extracted from audio files, but rather an intermediate representation after being encoded by a neural network, which can more comprehensively capture the complex patterns and potential information in the audio data. The pre-trained encoder is obtained through joint training of a data distribution transformation model and a discriminative model, and the first implicit features follow a first continuous probability distribution.

[0042] Here, the data reasoning method provided in this application can be applied to multiple data fields such as sound, music, images, videos, multimodal data and text. The following embodiment takes audio data as the data to be processed as an example, and the implicit feature is the latent space audio vector.

[0043] Here, the data inference method provided in this application embodiment is based on a streaming model, specifically, the streaming model can be flowVAE. Here, the FlowVAE streaming model is a generative model combining variational autoencoders (VAE) and normalizing flows, specifically designed for continuous data streaming scenarios. It achieves real-time modeling and updating of the latent space by dynamically adapting to incrementally arriving data. The pre-trained encoder is based on the Flow Matching architecture, which maps the data to be processed to the latent space through invertible mapping to obtain the first latent features. Further, the latent space includes at least one latent subspace, each of which independently represents audio latent features of different dimensions. Here, Flow Matching is a generative model framework based on Continuous Normalizing Flows (CNF), which achieves efficient probabilistic path modeling by directly regressing the target vector field rather than the density function. Its core idea is to view the data generation process as a continuous dynamic evolution from a simple prior distribution to the target distribution. In implicit feature transformation, each implicit feature corresponds to a subspace. Audio dimensions include timbre, prosody, pitch, and other data, each represented independently by its own latent subspace. FlowVAE requires a decoupling mechanism to implement this functionality. The advantage of this design is that it enhances the flexibility and expressiveness of the latent space, making latent space interpolation, arithmetic, and attribute editing easier. For example, it allows for smooth changes to the speaker's timbre or adjustment of emotional intensity while preserving the speech content.

[0044] In this way, the FlowVAE streaming model and the Flow Matching architecture jointly provide a generative modeling framework based on continuous transformation, which can construct more complex latent distributions and make the probability density of the latent space accurately computable. This feature allows FlowVAE to directly maximize the log-likelihood of the data, rather than relying on the variational lower bound (ELBO) of the VAE, thereby reducing the information loss of the latent space. At the same time, traditional VAEs may lead to excessive compression of the latent distribution due to KL divergence constraints, while the FlowVAE streaming model preserves data details through invertible mapping, which is significantly better than traditional VAE models.

[0045] Based on the data inference method provided in this application, the pre-trained encoder is obtained through the following steps:

[0046] A: Input the sample data into the encoder to be trained to obtain the implicit feature samples output by the encoder to be trained.

[0047] In specific implementation of step A above, sample data is input into the encoder to be trained to obtain the implicit feature samples output by the encoder to be trained.

[0048] B: Input the implicit feature sample into the data distribution transformation model, so as to transform the implicit feature sample into a second implicit feature that follows a second continuous probability distribution through the data distribution transformation model.

[0049] Here, the data distribution transformation model is mainly used to calculate the distribution transformation of implicit features. For example, if the implicit feature follows a Gaussian distribution, then the latent space variable is mapped to an implicit feature that follows a standard Gaussian distribution.

[0050] Regarding step B above, during the specific training process, the implicit feature samples output by the encoder to be trained are input into the data distribution transformation model, so as to transform the implicit feature samples into second implicit features that follow the second continuous probability distribution through the data distribution transformation model.

[0051] Specifically, when the first continuous probability distribution is a Gaussian distribution, the second continuous probability distribution is a standard Gaussian distribution. The implicit feature samples output by the encoder to be trained follow a Gaussian distribution. The data distribution transformation model is used to transform the implicit feature samples following a Gaussian distribution into second implicit features that follow a standard Gaussian distribution. As an example, when performing the implicit feature transformation to a standard Gaussian distribution, the data distribution transformation model first estimates the mean and standard deviation (or variance) of the implicit feature samples, and then uses the estimated mean and standard deviation to standardize the implicit feature samples (i.e., subtract the mean and then divide by the standard deviation) to obtain the second implicit features that conform to the standard Gaussian distribution.

[0052] Alternatively, when the first continuous probability distribution is a Laplace or gamma distribution, the second continuous probability distribution is a standard Laplace or standard gamma distribution. The implicit feature samples output by the encoder to be trained follow a Laplace or gamma distribution. The data distribution transformation model is used to convert the implicit feature samples following the Laplace or gamma distribution into second implicit features following the standard Laplace or standard gamma distribution. As an example, when the data distribution transformation model performs implicit feature transformation from a standard Laplace distribution, it first estimates the location parameter μ (i.e., the median) of the implicit feature samples. Since the Laplace distribution is symmetric, the location parameter μ is the median of the distribution. Therefore, μ can be estimated using the sample median. Then, the scale parameter b is estimated. In the Laplace distribution, the scale parameter b can be estimated using the median absolute deviation (MAD) of the samples, or the mean of the absolute deviations of the samples. Finally, the location parameter μ and the scale parameter b are used to standardize each sample point in the implicit feature samples to obtain the second implicit feature conforming to the standard Laplace distribution. The standard gamma distribution typically refers to a gamma distribution with a scale parameter θ=1, thus requiring a scale transformation. The distribution transformation described in this application is merely illustrative; other methods capable of achieving distribution transformation are also within the scope of protection of this application.

[0053] C: The discriminative model establishes a loss function based on the difference between the second implicit feature and the standard probability distribution, and adjusts the parameters of the encoder to be trained based on the result of the loss function until the training conditions are met, thus obtaining the pre-trained encoder.

[0054] Here, the discriminant model is mainly used to calculate the difference between the second implicit feature mapping result and the standard normal distribution.

[0055] Regarding step C above, in specific implementation, the discriminant model calculates the difference between the second implicit feature and the standard probability distribution, uses this difference to establish a loss function, and adjusts the parameters of the encoder to be trained based on the result of the loss function until the training conditions are met. Here, the training conditions can be that the training reaches a preset number of times, or the loss function result converges or is minimized. At this point, training is stopped to obtain the pre-trained encoder.

[0056] In existing technologies, traditional methods based on latent space variables are typically based on VAEs (Variational Autoencoders). This structure usually directly constrains the latent space variables to follow a standard normal distribution through a discriminative model. However, latent space variables following a standard normal distribution carry relatively little information, resulting in poor data quality when decoded from these variables. In contrast, this application introduces a data distribution transformation model that supports distribution transformation. This model maps latent space variables following a first continuous distribution to a standard first continuous distribution. By calculating the difference between the mapping result and the generated standard distribution, the parameters of the encoder to be trained are adjusted. This allows the implicit features output by the pre-trained encoder to follow a non-standard first continuous distribution, thereby carrying more information. Taking Gaussian distribution as an example, this application, by introducing a data distribution transformation model, ensures that the latent space variables generated by the model can follow an ordinary Gaussian distribution with arbitrary mean and variance, rather than directly constraining the latent space variables generated by the model to a standard Gaussian distribution. Latent space variables with an ordinary Gaussian distribution can carry more audio information than those with a standard Gaussian distribution. Thus, this application uses implicit features instead of the traditional Mel spectrum as an intermediate representation. Instead of directly making the latent space variable follow a standard normal distribution, it allows it to follow other distributions (such as Gaussian distribution, Laplace distribution, etc.), thereby carrying more information. The richer the feature information carried by the latent space variable, the better the quality of the information recovered by decoding based on the latent space variable.

[0057] Preferably, the discriminative model is based on a generative adversarial network (GAN) architecture and is used to determine whether the second implicit feature belongs to the standard distribution data. During training, the training data for the discriminative model needs to include the actual distribution data of the latent space vectors (such as the Laplace distribution) and the generated standard distribution data. The standard distribution data can be generated by the generator in the GAN or sampled from standard distribution data. If generated by the generator, the generation quality needs to be guaranteed, i.e., the generated data must conform to the standard distribution.

[0058] Furthermore, when the discriminant model is a generative adversarial network (GAN), it is used to determine the probability that the second continuous probability distribution is the standard distribution, and the loss function is KL divergence or JS divergence. Specifically, KL divergence (Kullback-Leibler Divergence), also known as relative entropy, is an important tool in information theory for measuring the difference between two probability distributions. It is widely used in machine learning, statistics, natural language processing, and other fields to quantify the difference between the model's predicted distribution and the true distribution, or to evaluate the efficiency of information compression. JS divergence (Jensen-Shannon Divergence) is a smoothed version of KL divergence; it lies between two probability distributions and helps to address undefined problems that KL divergence may encounter in certain situations (e.g., when one distribution is 0). When the discriminant model is a Wasserstein GAN, it is used to determine the distributional difference between the second continuous probability distribution and the standard distribution, and the loss function is Wasserstein distance. Specifically, Wasserstein distance (Earth Mover's Distance - EMD) is a method for measuring the difference between two probability distributions. It quantifies this dissimilarity by measuring the minimum cost required to transfer one distribution to another. The calculation of the Wasserstein distance involves an optimization problem, the goal of which is to find the optimal mapping to transfer one distribution to another in order to minimize the cost of the transfer.

[0059] S102, the first implicit feature is input into the decoder so that the first implicit feature is restored by the decoder to obtain the first generated data.

[0060] Here, the decoder reconstructs the data based on latent space variables to obtain the generated data. As an example, when the data to be processed is audio data (such as a WAV audio file), the generated data is also the corresponding audio. In this case, the decoder is a vocoder, for example, a neural vocoder.

[0061] In specific implementation of step S102, the first implicit feature output by the pre-trained encoder is input into the decoder so that the first implicit feature can be restored by the decoder to obtain the first generated data.

[0062] According to the data inference method provided in this application, after obtaining the first implicit feature output by the pre-trained encoder, the data inference method further includes:

[0063] I: Obtain at least one target implicit feature corresponding to the data to be processed.

[0064] Regarding step I above, in specific implementation, at least one target implicit feature corresponding to the data to be processed is obtained. Here, the target implicit feature may include characteristic information such as timbre, pitch, speech rate, and prosody of the data to be processed. This characteristic information can also be obtained from the data to be processed through a pre-trained encoder, or from the Mel spectrum of the data to be processed through a pre-trained encoder.

[0065] II: Input the at least one target implicit feature and the first implicit feature into the generative model simultaneously to obtain the third implicit feature output by the generative model.

[0066] Regarding step II above, in specific implementation, at least one target implicit feature and the first implicit feature obtained in step I are simultaneously input into the generative model to obtain the third implicit feature output by the generative model. Here, the third implicit feature is obtained by transforming the first implicit feature based on the generative model. The input data of the generative model are the target implicit feature and the first implicit feature, and the output data of the generative model are the latent space variables that need to be input into the decoder. Preferably, the generative model can be a transformer. In this way, an additional optimization step is added in the encoding stage, which further processes the latent space variables through the generative model. The generative model can not only input implicit features, but also combine characteristic information such as timbre, pitch, speech rate, and prosody, which can better capture the latent patterns and details in the audio data and generate richer latent space variables.

[0067] III: Input the third implicit feature into the decoder so that the decoder can restore the third implicit feature to obtain the second generated data.

[0068] In step III above, the third implicit feature output by the generative model is input into the decoder so that the decoder can restore the third implicit feature and obtain the second generated data.

[0069] The data reasoning method provided in this application firstly acquires data to be processed and inputs the data to be processed into a pre-trained encoder to obtain a first implicit feature output by the pre-trained encoder; wherein the pre-trained encoder is jointly trained by a data distribution transformation model and a discriminant model, and the first implicit feature follows a first continuous probability distribution; then, the first implicit feature is input into a decoder to restore the first implicit feature through the decoder to obtain first generated data.

[0070] This application uses implicit features instead of traditional Mel spectra as an intermediate representation, which better preserves the feature information of the original data. Furthermore, the implicit features generated by the pre-trained encoder follow a first continuous probability distribution, rather than directly making the latent space variables follow a standard continuous probability distribution, thus carrying more information and avoiding the error effects of extracting Mel spectra from data and reconstructing data from Mel spectra. In addition, the pre-trained encoder incorporates a data distribution transformation model during the training phase, so that the implicit features generated by the model do not necessarily directly follow a standard data distribution, increasing the amount of information carried by the latent space vectors, thereby improving the model's generation capability and achieving higher accuracy and more diverse generation effects, solving the information loss problem existing in traditional generative models.

[0071] Please see Figure 2 , Figure 3 , Figure 2 This is one of the structural schematic diagrams of a data inference device provided in an embodiment of this application. Figure 3 This is a second schematic diagram of a data inference device provided in an embodiment of this application. Figure 2 As shown, the data inference device 200 includes:

[0072] The implicit feature generation module 201 is used to acquire the data to be processed, input the data to be processed into the pre-trained encoder, and obtain the first implicit feature output by the pre-trained encoder; wherein, the pre-trained encoder is jointly trained by the data distribution transformation model and the discrimination model, and the first implicit feature follows a first continuous probability distribution.

[0073] The first data restoration module 202 is used to input the first implicit feature into the decoder so that the first implicit feature can be restored by the decoder to obtain the first generated data.

[0074] Furthermore, such as Figure 3 As shown, the data inference device 200 further includes an encoder training module 203, which is used to obtain the pre-trained encoder through the following steps:

[0075] The sample data is input into the encoder to be trained to obtain the implicit feature samples output by the encoder to be trained;

[0076] The implicit feature samples are input into the data distribution transformation model to convert the implicit feature samples into second implicit features that follow a second continuous probability distribution.

[0077] The discriminative model establishes a loss function based on the difference between the second implicit feature and the standard probability distribution, and adjusts the parameters of the encoder to be trained based on the result of the loss function until the training conditions are met, thus obtaining the pre-trained encoder.

[0078] Furthermore, when the first continuous probability distribution is a Gaussian distribution, the second continuous probability distribution is a standard Gaussian distribution, and the data distribution transformation model is used to convert the implicit feature samples that conform to the Gaussian distribution into second implicit features that conform to the standard Gaussian distribution.

[0079] Furthermore, the discriminant model is based on a generative adversarial network architecture, and the discriminant model is used to determine whether the second implicit feature belongs to the standard distribution data; wherein the standard distribution data is generated by the generator in the generative adversarial network, or sampled from the standard distribution data.

[0080] Furthermore, the discriminant model is a generative adversarial network (GAN), used to determine the probability that the second continuous probability distribution is standard distribution data, and the loss function is KL divergence or JS divergence; or,

[0081] The discriminant model is a Wasserstein generative adversarial network, which is used to determine the distribution gap between the second continuous probability distribution and the standard distribution data. The loss function is the Wasserstein distance.

[0082] Furthermore, such as Figure 3 As shown, the data inference device 200 further includes a second data restoration module 204. After obtaining the first implicit feature output by the pre-trained encoder, the second data restoration module 204 is used to:

[0083] Obtain at least one target implicit feature corresponding to the data to be processed;

[0084] The at least one target implicit feature and the first implicit feature are simultaneously input into the generative model to obtain the third implicit feature output by the generative model.

[0085] The third implicit feature is input into the decoder so that the decoder can restore the third implicit feature to obtain the second generated data.

[0086] Furthermore, the data to be processed is audio data, the decoder is a neural vocoder, and the implicit feature is a latent space audio vector.

[0087] Furthermore, the data inference method is implemented based on a streaming model, and the pre-trained encoder is based on a continuous flow matching architecture, which maps the data to be processed to the latent space through reversible mapping to obtain the first implicit features.

[0088] Furthermore, the latent space includes at least one latent subspace, each of which independently represents audio latent features of different dimensions.

[0089] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0090] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 The steps of the data reasoning method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0091] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the data reasoning method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0092] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0094] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0095] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0096] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data inference method, characterized by, The data reasoning method comprises: obtaining to-be-processed data, inputting the to-be-processed data into a pre-trained encoder, and obtaining first implicit features output by the pre-trained encoder; wherein the pre-trained encoder is obtained by jointly training a data distribution transformation model and a discriminator model, the first implicit features follow a first continuous probability distribution, and the to-be-processed data is audio data; inputting the first implicit features into a decoder to restore the first implicit features by the decoder, and obtaining first generated data; the pre-trained encoder is obtained by the following steps: inputting sample data into a to-be-trained encoder to obtain implicit feature samples output by the to-be-trained encoder; inputting the implicit feature samples into the data distribution transformation model to convert the implicit feature samples into second implicit features following a second continuous probability distribution by the data distribution transformation model; the discriminator model establishes a loss function based on the difference between the second implicit features and standard probability distribution, and adjusts the parameters of the to-be-trained encoder based on the result of the loss function until a training condition is reached to obtain the pre-trained encoder; wherein the discriminator model is used to determine whether the second implicit features belong to standard distribution data.

2. The data reasoning method of claim 1, wherein, When the first continuous probability distribution is a Gaussian distribution, the second continuous probability distribution is a standard Gaussian distribution, and the data distribution transformation model is used to convert implicit feature samples following the Gaussian distribution into second implicit features following the standard Gaussian distribution.

3. The data reasoning method of claim 1, wherein, The discriminator model is based on a generative adversarial network architecture, and the standard distribution data is generated by a generator in the generative adversarial network or is sampled from standard distribution data.

4. The data reasoning method of claim 3, wherein, The discriminator model is a generative adversarial network, the discriminator model is used to determine the probability that the second continuous probability distribution is standard distribution data, and the loss function is KL divergence or JS divergence; or The discriminator model is a Wasserstein generative adversarial network, the discriminator model is used to determine the distribution gap between the second continuous probability distribution and the standard distribution data, and the loss function is the Wasserstein distance.

5. The data reasoning method of claim 1, wherein, After obtaining the first implicit features output by the pre-trained encoder, the data reasoning method further comprises: obtaining at least one target implicit feature corresponding to the to-be-processed data; inputting the at least one target implicit feature and the first implicit feature into a generative model at the same time to obtain third implicit features output by the generative model; inputting the third implicit features into the decoder to restore the third implicit features by the decoder to obtain second generated data.

6. The data reasoning method according to any one of claims 1-5, characterized in that, The decoder is a neural vocoder, and the implicit features are implicit space audio vectors.

7. The data reasoning method according to any one of claims 1-5, characterized in that, The data reasoning method is implemented based on a streaming model, the pre-trained encoder is based on a continuous flow matching (Flow Matching) architecture, and the to-be-processed data is mapped to a latent space by reversible mapping to obtain the first implicit features.

8. The data reasoning method of claim 7, wherein, The latent space comprises at least one latent subspace, and each latent subspace independently represents audio latent features of different dimensions.

9. A data inference apparatus, comprising: The data reasoning apparatus comprises: An implicit feature generation module, configured to obtain to-be-processed data, input the to-be-processed data into a pre-training encoder, and obtain first implicit features output by the pre-training encoder; wherein the pre-training encoder is obtained through common training of a data distribution transformation model and a discriminant model, the first implicit features are subject to a first continuous probability distribution, and the to-be-processed data are audio data; A first data restoration module, configured to input the first implicit features into a decoder, restore the first implicit features through the decoder, and obtain first generated data; The data reasoning apparatus further comprises an encoder training module, configured to obtain the pre-training encoder through the following steps: Input sample data into a to-be-trained encoder, and obtain implicit feature samples output by the to-be-trained encoder; Input the implicit feature samples into the data distribution transformation model, and convert the implicit feature samples into second implicit features subject to a second continuous probability distribution through the data distribution transformation model; The discriminant model establishes a loss function based on a difference between the second implicit features and a standard probability distribution, adjusts parameters of the to-be-trained encoder based on a result of the loss function, until a training condition is reached, and obtains the pre-training encoder; wherein the discriminant model is configured to determine whether the second implicit features belong to standard distribution data.

Images