Data reasoning method and data reasoning device

Abstract

The invention provides a data reasoning method and a data reasoning device, and the method comprises the steps: obtaining to-be-processed data, inputting the to-be-processed data into a pre-training encoder, and obtaining a first implicit feature outputted by the pre-training encoder; wherein the pre-training encoder is obtained by jointly training a data distribution transformation model and a discrimination model, and the first implicit feature obeys first continuous probability distribution; and inputting the first implicit feature into a decoder so as to restore the first implicit feature through the decoder to obtain first generated data. According to the method and the device, the traditional Mel spectrum is no longer used as an intermediate representation form, and the implicit features are used, so that the feature information of the original data can be better reserved, the generation capability of the model is further improved, the higher-precision and more diversified generation effects are realized, and the problem of information loss in the traditional generation model is solved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data inference method and a data inference device. Background Art

[0002] With the widespread application of deep learning technology in the field of audio processing, an intermediate representation based on Mel spectrum is often used to generate sound or music. Mel spectrum is a frequency analysis method that conforms to the characteristics of human hearing and can convert raw audio signals into frequency domain features that are more suitable for processing by machine learning models. The common audio generation process is: first, the raw audio is converted into a Mel spectrum through a short-time Fourier transform, then the Mel spectrum is modeled and generated using a deep learning model, and finally, the generated Mel spectrum is restored to a time-domain audio signal through a vocoder. This intermediate representation-based method has become the mainstream technical solution in the current field of audio generation, which can effectively reduce computational complexity and has achieved remarkable results in applications such as speech synthesis and music generation.

[0003] However, in the process of converting audio signals to and from Mel spectrograms and further transforming based on Mel spectrograms, information loss is inevitably introduced, while increasing the overhead of data processing. This loss will significantly reduce the audio quality under the cumulative effect of multiple conversions, thereby affecting the final output effect of the generative model. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a data inference method and data inference device that uses implicit features instead of traditional mel-spectrograms as an intermediate representation. Implicit features can better preserve the characteristic information of the original data and avoid the influence of errors in extracting mel-spectrograms from data and restoring them from mel-spectrograms. In addition, the pre-trained encoder adds a data distribution transformation model during the training phase, so that the implicit features generated by the model do not need to directly follow the standard data distribution, which increases the amount of information carried by the implicit features and thus improves the model's generation capability, achieving higher precision and more diverse generation effects, and solving the information loss problem existing in traditional generation models.

[0005] In a first aspect, an embodiment of the present application provides a data reasoning method, the data reasoning method comprising: Obtaining data to be processed, inputting the data to be processed into a pre-trained encoder, and obtaining a first implicit feature output by the pre-trained encoder; wherein the pre-trained encoder is obtained by jointly training a data distribution transformation model and a discriminant model, and the first implicit feature obeys a first continuous probability distribution; The first implicit feature is input into a decoder so as to restore the first implicit feature through the decoder to obtain first generated data.

[0006] Furthermore, the pre-trained encoder is obtained by the following steps: Inputting sample data into the encoder to be trained to obtain implicit feature samples output by the encoder to be trained; Inputting the implicit feature sample into the data distribution transformation model to transform the implicit feature sample into a second implicit feature obeying a second continuous probability distribution through the data distribution transformation model; The discriminant model establishes a loss function based on the difference between the second implicit feature and the standard probability distribution, and adjusts the parameters of the encoder to be trained based on the result of the loss function until the training conditions are met, thereby obtaining the pre-trained encoder.

[0007] Furthermore, when the first continuous probability distribution is a Gaussian distribution, the second continuous probability distribution is a standard Gaussian distribution, and the data distribution transformation model is used to convert the implicit feature samples that obey the Gaussian distribution into second implicit features that obey the standard Gaussian distribution.

[0008] Furthermore, the discriminant model is based on a generative adversarial network architecture, and the discriminant model is used to determine whether the second implicit feature belongs to standard distribution data; wherein, the standard distribution data is generated by the generator in the generative adversarial network, or sampled from the standard distribution data.

[0009] Furthermore, the discriminant model is a generative adversarial network, the discriminant model is used to determine the probability that the second continuous probability distribution is standard distribution data, and the loss function is KL divergence or JS divergence; or, The discriminant model is a Wasserstein generative adversarial network, which is used to determine the distribution gap between the second continuous probability distribution and the standard distribution data, and the loss function is the Wasserstein distance.

[0010] Furthermore, after obtaining the first implicit feature output by the pre-trained encoder, the data inference method further includes: Acquire at least one target implicit feature corresponding to the data to be processed; Inputting the at least one target implicit feature and the first implicit feature into a generative model simultaneously to obtain a third implicit feature output by the generative model; The third implicit feature is input into the decoder, so that the third implicit feature is restored by the decoder to obtain second generated data.

[0011] Furthermore, the data to be processed is audio data, the decoder is a neural vocoder, and the implicit feature is a latent space audio vector.

[0012] Furthermore, the data inference method is implemented based on a streaming model, and the pre-trained encoder is based on a continuous flow matching Flow Matching architecture, which maps the data to be processed to a latent space through a reversible mapping to obtain a first implicit feature.

[0013] Furthermore, the latent space includes at least one latent subspace, and each latent subspace independently represents audio latent features of different dimensions.

[0014] In a second aspect, an embodiment of the present application further provides a data inference device, the data inference device comprising: an implicit feature generation module, configured to obtain data to be processed, input the data to be processed into a pre-trained encoder, and obtain a first implicit feature output by the pre-trained encoder; wherein the pre-trained encoder is trained by a data distribution transformation model and a discriminant model, and the first implicit feature obeys a first continuous probability distribution; The first data restoration module is configured to input the first implicit feature into a decoder so as to restore the first implicit feature through the decoder to obtain first generated data.

[0015] An embodiment of the present application provides a data inference method and a data inference device. First, data to be processed is obtained, and the data to be processed is input into a pre-trained encoder to obtain a first implicit feature output by the pre-trained encoder; wherein the pre-trained encoder is obtained by jointly training a data distribution transformation model and a discriminant model, and the first implicit feature obeys a first continuous probability distribution; then, the first implicit feature is input into a decoder so that the first implicit feature is restored by the decoder to obtain first generated data.

[0016] This application uses implicit features instead of traditional Mel spectrum as an intermediate representation. Implicit features can better retain the characteristic information of the original data. Moreover, the implicit features generated by the pre-trained encoder obey the first continuous probability distribution, rather than directly making the latent space variables obey the standard continuous probability distribution, so that they carry more information and avoid the error influence of extracting Mel spectrum from data and restoring Mel spectrum to data. In addition, the pre-trained encoder adds a data distribution transformation model in the training stage, so that the implicit features generated by the model do not have to directly obey the standard data distribution, which improves the amount of information carried by the latent space vector, thereby improving the generation ability of the model, achieving higher precision and more diverse generation effects, and solving the information loss problem existing in the traditional generation model.

[0017] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flowchart of a data reasoning method provided in an embodiment of the present application; Figure 2 This is one of the structural diagrams of a data inference device provided in an embodiment of the present application; Figure 3 This is a second structural diagram of a data inference device provided in an embodiment of the present application; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work falls within the scope of protection of the present application.

[0021] First, the application scenarios to which this application is applicable are introduced. This application can be applied in the field of data processing technology.

[0022] With the widespread application of deep learning technology in the field of audio processing, an intermediate representation based on Mel spectrum is often used to generate sound or music. Mel spectrum is a frequency analysis method that conforms to the characteristics of human hearing and can convert raw audio signals into frequency domain features that are more suitable for processing by machine learning models. The common audio generation process is: first, the raw audio is converted into a Mel spectrum through a short-time Fourier transform, then the Mel spectrum is modeled and generated using a deep learning model, and finally, the generated Mel spectrum is restored to a time-domain audio signal through a vocoder. This intermediate representation-based method has become the mainstream technical solution in the current field of audio generation, which can effectively reduce computational complexity and has achieved remarkable results in applications such as speech synthesis and music generation.

[0023] Research has found that in the process of converting audio signals to and from Mel spectra, as well as further transformations based on Mel spectra, information loss is inevitably introduced, while increasing data processing overhead. This loss, through the cumulative effect of multiple conversions, will significantly reduce audio quality, thereby affecting the final output effect of the generative model.

[0024] Based on this, an embodiment of the present application provides a data inference method to improve the generation capability of the model, achieve higher precision and more diverse generation effects, and solve the information loss problem existing in traditional generation models.

[0025] See also Figure 1 , Figure 1 This is a flow chart of a data inference method provided in an embodiment of the present application. Figure 1 As shown in , the data reasoning method provided by the embodiment of the present application includes: S101, obtaining data to be processed, inputting the data to be processed into a pre-trained encoder, and obtaining a first implicit feature output by the pre-trained encoder.

[0026] Regarding step S101 above, during implementation, data to be processed is obtained and input into a pre-trained encoder, which then generates implicit features. Here, the implicit features can be latent space variables, and the implicit features obey a first continuous probability distribution, i.e., the first implicit features output by the pre-trained encoder are obtained. For example, when applying data inference methods to the fields of sound and music, implicit features are not traditional explicit features extracted directly from audio files (such as Mel-spectrograms, MFCCs, etc.), but rather an intermediate representation encoded by a neural network that more comprehensively captures the complex patterns and underlying information in the audio data. The pre-trained encoder is trained jointly by a data distribution transformation model and a discriminant model, and the first implicit features obey a first continuous probability distribution.

[0027] Here, the data inference method provided in this application can be applied to multiple data fields such as sound, music, images, videos, multimodal data and text. The following embodiment takes the data to be processed as audio data as an example, and the implicit feature is the latent space audio vector.

[0028] Here, the data inference method provided in the embodiment of the present application is implemented based on a streaming model. Specifically, the streaming model can be flowVAE. Here, the FlowVAE streaming model is a generative model that combines a variational autoencoder (VAE) and normalizing flows, and is designed for continuous data streaming scenarios. It achieves real-time modeling and updating of the latent space by dynamically adapting to incrementally arriving data. The pre-trained encoder is based on the continuous flow matching Flow Matching architecture, and maps the data to be processed to the latent space through a reversible mapping to obtain the first implicit feature. Furthermore, the latent space includes at least one latent subspace, each latent subspace independently represents the audio latent features of different dimensions. Here, Flow Matching is a generative model framework based on continuous normalizing flows (CNF), which achieves efficient probabilistic path modeling by directly regressing the target vector field rather than the density function. Its core idea is to regard the data generation process as a continuous dynamic evolution from a simple prior distribution to a target distribution. When performing implicit feature conversion, each implicit feature corresponds to a subspace. Audio dimensions, including timbre, rhythm, pitch, and other data, are represented separately by each latent subspace, and each latent subspace is independent of each other. A decoupling mechanism is required in flowVAE to implement this functionality. The benefit of this design is that it improves the flexibility and expressiveness of the latent space, making it easier to perform latent space interpolation, latent space arithmetic, and attribute editing. For example, while preserving the content of the speech, it is possible to smoothly change the speaker's timbre or adjust the emotional intensity.

[0029] In this way, the FlowVAE streaming model and the Flow Matching architecture jointly provide a generative modeling framework based on continuous transformations, which can construct more complex latent distributions and make the probability density of the latent space accurately calculated. This feature allows FlowVAE to directly maximize the log-likelihood of the data rather than relying on the variational lower bound (ELBO) of VAE, thereby reducing the information loss of the latent space; at the same time, traditional VAE may cause excessive compression of the latent distribution due to the KL divergence constraint, while FlowVAE's flow model retains data details through reversible mapping, which is significantly better than the traditional VAE model.

[0030] According to the data inference method provided in this application, the pre-trained encoder is obtained through the following steps: A: Input sample data into the encoder to be trained to obtain implicit feature samples output by the encoder to be trained.

[0031] Regarding the above step A, during specific implementation, the sample data is input into the encoder to be trained to obtain the implicit feature samples output by the encoder to be trained.

[0032] B: Inputting the implicit feature sample into the data distribution transformation model to transform the implicit feature sample into a second implicit feature obeying a second continuous probability distribution through the data distribution transformation model.

[0033] Here, the data distribution transformation model is mainly used to calculate the distribution transformation of implicit features. For example, if the implicit feature obeys Gaussian distribution, the implicit space variable is mapped into an implicit feature that obeys standard Gaussian distribution.

[0034] Regarding step B above, when implementing the training, the implicit feature samples output by the encoder to be trained are input into the data distribution transformation model to transform the implicit feature samples into second implicit features that obey the second continuous probability distribution through the data distribution transformation model.

[0035] Specifically, when the first continuous probability distribution is a Gaussian distribution, the second continuous probability distribution is a standard Gaussian distribution, and the implicit feature samples output by the encoder to be trained follow the Gaussian distribution, the data distribution transformation model is used to convert the implicit feature samples that follow the Gaussian distribution into a second implicit feature that follows the standard Gaussian distribution. As an example, when performing implicit feature conversion from a standard Gaussian distribution, the data distribution transformation model first estimates the mean and standard deviation (or variance) of the implicit feature samples, then uses the estimated mean and standard deviation to standardize the implicit feature samples (i.e., subtract the mean and divide by the standard deviation), thereby obtaining a second implicit feature that conforms to the standard Gaussian distribution.

[0036] Alternatively, when the first continuous probability distribution is a Laplace distribution or a Gamma distribution, and the second continuous probability distribution is a standard Laplace distribution or a standard Gamma distribution, the implicit feature samples output by the encoder to be trained follow the Laplace distribution or the Gamma distribution. The data distribution transformation model is used to transform the implicit feature samples following the Laplace distribution or the Gamma distribution into a second implicit feature following the standard Laplace distribution or the Gamma distribution. For example, when the data distribution transformation model transforms the implicit feature samples from the standard Laplace distribution, it first estimates the location parameter μ (i.e., the median) of the implicit feature samples. Since the Laplace distribution is symmetric, the location parameter μ is the median of the distribution. Therefore, μ can be estimated using the sample median. The scale parameter b is then estimated. In the Laplace distribution, the scale parameter b can be estimated using the median absolute deviation (MAD) of the samples or the mean of the sample absolute deviations. Finally, each sample point in the implicit feature samples is normalized using the location parameter μ and the scale parameter b to obtain the second implicit feature that follows the standard Laplace distribution. The standard gamma distribution usually refers to the gamma distribution with a scale parameter θ = 1, so a scale transformation is required. The content of the distribution transformation described in this application is only an example, and other methods that can achieve distribution transformation are also within the scope of protection of this application.

[0037] C: The discriminant model establishes a loss function based on the difference between the second implicit feature and the standard probability distribution, and adjusts the parameters of the encoder to be trained based on the result of the loss function until the training conditions are met, thereby obtaining the pre-trained encoder.

[0038] Here, the discriminant model is mainly used to calculate the difference between the second implicit feature mapping result and the standard normal distribution.

[0039] For the above step C, during the specific implementation, the discriminant model calculates the difference between the second implicit feature and the standard probability distribution, and uses the difference to establish a loss function, and adjusts the parameters of the encoder to be trained based on the result of the loss function until the training conditions are met. The training conditions here can make the training reach a preset number of times, or the loss function result converges or is minimized. At this time, the training is stopped to obtain a pre-trained encoder.

[0040] In the prior art, traditional latent space variable-based methods are typically based on VAEs (Variation Auto-Encoders). This structure typically directly constrains latent space variables to follow a standard normal distribution through a discriminant model. However, latent space variables following a standard normal distribution can carry less information, resulting in poor data quality from decoding these latent space variables. In contrast, the present application introduces a data distribution transformation model that supports distribution transformation. This model maps latent space variables following a first continuous distribution to a standard first continuous distribution. The parameters of the encoder to be trained are adjusted by calculating the difference between the mapping result and the generated standard distribution. This allows the implicit features output by the pre-trained encoder to follow a non-standard first continuous distribution, thereby carrying more information. Taking the Gaussian distribution as an example, the present application introduces a data distribution transformation model to ensure that the latent space variables generated by the model can follow a standard Gaussian distribution with any mean and variance, rather than directly constraining the latent space variables generated by the model to follow a standard Gaussian distribution. Latent space variables with a standard Gaussian distribution can carry more audio information than those with a standard Gaussian distribution. In this way, this application uses implicit features instead of traditional Mel spectrum as an intermediate representation. It does not directly allow the latent space variables to obey the standard normal distribution, but allows it to obey other distributions (such as Gaussian distribution, Laplace distribution, etc.), thereby carrying more information. The richer the feature information carried by the latent space variables, the better the quality of the information recovered by decoding based on the latent space variables.

[0041] Preferably, the discriminant model is based on a generative adversarial network architecture and is used to determine whether the second implicit feature belongs to standard distribution data. During training, the discriminant model's training data must include the actual distribution data of latent space vectors (e.g., Laplace distribution) and generated standard distribution data. The standard distribution data can be standard distribution data generated by the generator in the generative adversarial network, or sampled from standard distribution data. If generated by the generator, the quality of the generated data must be guaranteed, i.e., the generated data must conform to the standard distribution.

[0042] Furthermore, when the discriminant model is a generative adversarial network, the discriminant model determines the probability that the second continuous probability distribution is a standard distribution data, and the loss function is KL divergence or JS divergence. Specifically, KL divergence (Kullback-Leibler Divergence), also known as relative entropy, is an important tool in information theory for measuring the difference between two probability distributions. It is widely used in fields such as machine learning, statistics, and natural language processing to quantify the difference between a model's predicted distribution and the true distribution, or to evaluate the efficiency of information compression. JS divergence (Jensen-Shannon Divergence) is a smoothed version of KL divergence. It lies between two probability distributions and helps resolve the undefined nature of KL divergence in certain situations (for example, when one of the distributions is zero). When the discriminant model is a Wasserstein generative adversarial network, the discriminant model determines the distribution gap between the second continuous probability distribution and the standard distribution data, and the loss function is the Wasserstein distance. Specifically, the Wasserstein distance (Earth Mover's Distance - EMD) is a method for measuring the difference between two probability distributions. It quantifies this difference by measuring the minimum cost required to transfer one distribution to another. The calculation of Wasserstein distance involves an optimization problem, whose goal is to find the best mapping from one distribution to another to minimize the cost of the transfer.

[0043] S102: Input the first implicit feature into a decoder, so as to restore the first implicit feature through the decoder to obtain first generated data.

[0044] Here, the decoder restores the data based on the latent space variables to obtain the generated data. Here, as an example, when the data to be processed is audio data (such as an audio file such as WAV), the generated data is also the corresponding audio. In this case, the decoder is a vocoder, such as a neural vocoder.

[0045] Regarding the above step S102, in a specific implementation, the first implicit feature output by the pre-trained encoder is input into the decoder, so that the first implicit feature is restored by the decoder to obtain the first generated data.

[0046] According to the data inference method provided in this application, after obtaining the first implicit feature output by the pre-trained encoder, the data inference method further includes: I: Obtain at least one target implicit feature corresponding to the data to be processed.

[0047] Regarding step I above, during the specific implementation, at least one target implicit feature corresponding to the data to be processed is obtained. Here, the target implicit feature may include characteristic information of the data to be processed, such as timbre, pitch, speaking rate, and rhythm. This characteristic information may also be obtained from the data to be processed by a pre-trained encoder, or obtained from the mel-spectrogram of the data to be processed by a pre-trained encoder.

[0048] II: Inputting the at least one target implicit feature and the first implicit feature into a generative model simultaneously to obtain a third implicit feature output by the generative model.

[0049] For the above step II, in the specific implementation, at least one target implicit feature and the first implicit feature obtained in the above step I are simultaneously input into the generative model to obtain a third implicit feature output by the generative model. Here, the first implicit feature is transformed based on the generative model to obtain the third implicit feature. The input data of the generative model are the target implicit feature and the first implicit feature, and the output data of the generative model are the latent space variables that need to be input into the decoder. Preferably, the generative model can be a transformer. In this way, an additional optimization step is added in the encoding stage, and the latent space variables are further processed by the generative model. The generative model can not only input implicit features, but also combine characteristic information such as timbre, pitch, speaking speed, rhythm, etc., which can better capture the potential patterns and detail information in the audio data and generate richer latent space variables.

[0050] III: Inputting the third implicit feature into the decoder so as to restore the third implicit feature through the decoder to obtain second generated data.

[0051] Regarding the above step III, during specific implementation, the third implicit feature output by the generative model is input into the decoder, so that the third implicit feature is restored by the decoder to obtain the second generated data.

[0052] The data inference method provided in an embodiment of the present application first obtains data to be processed, inputs the data to be processed into a pre-trained encoder, and obtains a first implicit feature output by the pre-trained encoder; wherein the pre-trained encoder is obtained by jointly training a data distribution transformation model and a discriminant model, and the first implicit feature obeys a first continuous probability distribution; then, the first implicit feature is input into a decoder so that the first implicit feature is restored by the decoder to obtain first generated data.

[0053] This application uses implicit features instead of traditional Mel spectrum as an intermediate representation. Implicit features can better retain the characteristic information of the original data. Moreover, the implicit features generated by the pre-trained encoder obey the first continuous probability distribution, rather than directly making the latent space variables obey the standard continuous probability distribution, so that they carry more information and avoid the error influence of extracting Mel spectrum from data and restoring Mel spectrum to data. In addition, the pre-trained encoder adds a data distribution transformation model in the training stage, so that the implicit features generated by the model do not have to directly obey the standard data distribution, which improves the amount of information carried by the latent space vector, thereby improving the generation ability of the model, achieving higher precision and more diverse generation effects, and solving the information loss problem existing in the traditional generation model.

[0054] See also Figure 2 、 Figure 3 , Figure 2 This is one of the structural diagrams of a data inference device provided in an embodiment of the present application. Figure 3 This is a second structural diagram of a data inference device provided in an embodiment of the present application. Figure 2 As shown in , the data inference device 200 includes: An implicit feature generation module 201 is configured to obtain data to be processed, input the data to be processed into a pre-trained encoder, and obtain a first implicit feature output by the pre-trained encoder; wherein the pre-trained encoder is trained by jointly training a data distribution transformation model and a discriminant model, and the first implicit feature obeys a first continuous probability distribution; The first data restoration module 202 is configured to input the first implicit feature into a decoder so as to restore the first implicit feature through the decoder to obtain first generated data.

[0055] Further, such as Figure 3 As shown, the data inference device 200 further includes an encoder training module 203, and the encoder training module 203 is used to obtain the pre-trained encoder through the following steps: Inputting sample data into the encoder to be trained to obtain implicit feature samples output by the encoder to be trained; Inputting the implicit feature sample into the data distribution transformation model to transform the implicit feature sample into a second implicit feature obeying a second continuous probability distribution through the data distribution transformation model; The discriminant model establishes a loss function based on the difference between the second implicit feature and the standard probability distribution, and adjusts the parameters of the encoder to be trained based on the result of the loss function until the training conditions are met, thereby obtaining the pre-trained encoder.

[0056] Furthermore, when the first continuous probability distribution is a Gaussian distribution, the second continuous probability distribution is a standard Gaussian distribution, and the data distribution transformation model is used to convert the implicit feature samples that obey the Gaussian distribution into second implicit features that obey the standard Gaussian distribution.

[0057] Furthermore, the discriminant model is based on a generative adversarial network architecture, and the discriminant model is used to determine whether the second implicit feature belongs to standard distribution data; wherein, the standard distribution data is generated by the generator in the generative adversarial network, or sampled from the standard distribution data.

[0058] Furthermore, the discriminant model is a generative adversarial network, the discriminant model is used to determine the probability that the second continuous probability distribution is standard distribution data, and the loss function is KL divergence or JS divergence; or, The discriminant model is a Wasserstein generative adversarial network, which is used to determine the distribution gap between the second continuous probability distribution and the standard distribution data, and the loss function is the Wasserstein distance.

[0059] Further, such as Figure 3 As shown, the data inference device 200 further includes a second data restoration module 204. After obtaining the first implicit feature output by the pre-trained encoder, the second data restoration module 204 is configured to: Acquire at least one target implicit feature corresponding to the data to be processed; Inputting the at least one target implicit feature and the first implicit feature into a generative model simultaneously to obtain a third implicit feature output by the generative model; The third implicit feature is input into the decoder, so that the third implicit feature is restored by the decoder to obtain second generated data.

[0060] Furthermore, the data to be processed is audio data, the decoder is a neural vocoder, and the implicit feature is a latent space audio vector.

[0061] Furthermore, the data inference method is implemented based on a streaming model, and the pre-trained encoder is based on a continuous flow matching Flow Matching architecture, which maps the data to be processed to a latent space through a reversible mapping to obtain a first implicit feature.

[0062] Furthermore, the latent space includes at least one latent subspace, and each latent subspace independently represents audio latent features of different dimensions.

[0063] See also Figure 4 , Figure 4This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in FIG, the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0064] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 The steps of the data inference method in the method embodiment shown and the specific implementation methods can be found in the method embodiment, which will not be repeated here.

[0065] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The steps of the data inference method in the method embodiment shown and the specific implementation methods can be found in the method embodiment, which will not be repeated here.

[0066] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0067] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. There may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0068] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0069] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0070] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0071] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A data reasoning method, characterized in that: The data reasoning method includes: Obtaining data to be processed, inputting the data to be processed into a pre-trained encoder, and obtaining a first implicit feature output by the pre-trained encoder; wherein the pre-trained encoder is obtained by jointly training a data distribution transformation model and a discriminant model, and the first implicit feature obeys a first continuous probability distribution; The first implicit feature is input into a decoder so as to restore the first implicit feature through the decoder to obtain first generated data.

2. The data inference method according to claim 1, characterized in that: The pre-trained encoder is obtained by the following steps: Inputting sample data into the encoder to be trained to obtain implicit feature samples output by the encoder to be trained; Inputting the implicit feature sample into the data distribution transformation model to transform the implicit feature sample into a second implicit feature obeying a second continuous probability distribution through the data distribution transformation model; The discriminant model establishes a loss function based on the difference between the second implicit feature and the standard probability distribution, and adjusts the parameters of the encoder to be trained based on the result of the loss function until the training conditions are met, thereby obtaining the pre-trained encoder.

3. The data inference method according to claim 2, characterized in that: When the first continuous probability distribution is a Gaussian distribution, the second continuous probability distribution is a standard Gaussian distribution, and the data distribution transformation model is used to convert the implicit feature samples that obey the Gaussian distribution into second implicit features that obey the standard Gaussian distribution.

4. The data inference method according to claim 2, characterized in that: The discriminant model is based on a generative adversarial network architecture, and the discriminant model is used to determine whether the second implicit feature belongs to standard distribution data; wherein, the standard distribution data is generated by a generator in the generative adversarial network, or sampled from standard distribution data.

5. The data inference method according to claim 4, characterized in that: The discriminant model is a generative adversarial network, the discriminant model is used to determine the probability that the second continuous probability distribution is standard distribution data, and the loss function is KL divergence or JS divergence; or, The discriminant model is a Wasserstein generative adversarial network, which is used to determine the distribution gap between the second continuous probability distribution and the standard distribution data, and the loss function is the Wasserstein distance.

6. The data inference method according to claim 1, characterized in that: After obtaining the first implicit feature output by the pre-trained encoder, the data inference method further includes: Acquire at least one target implicit feature corresponding to the data to be processed; Inputting the at least one target implicit feature and the first implicit feature into a generative model simultaneously to obtain a third implicit feature output by the generative model; The third implicit feature is input into the decoder, so that the third implicit feature is restored by the decoder to obtain second generated data.

7. The data inference method according to any one of claims 1 to 6, characterized in that: The data to be processed is audio data, the decoder is a neural vocoder, and the implicit feature is a latent space audio vector.

8. The data inference method according to any one of claims 1 to 6, characterized in that: The data inference method is implemented based on a streaming model, and the pre-trained encoder is based on a continuous flow matching Flow Matching architecture, which maps the data to be processed to a latent space through a reversible mapping to obtain a first implicit feature.

9. The data inference method according to claim 8, characterized in that: The latent space includes at least one latent subspace, and each latent subspace independently represents audio latent features of different dimensions.

10. A data inference device, characterized in that: The data inference device comprises: an implicit feature generation module, configured to obtain data to be processed, input the data to be processed into a pre-trained encoder, and obtain a first implicit feature output by the pre-trained encoder; wherein the pre-trained encoder is trained by a data distribution transformation model and a discriminant model, and the first implicit feature obeys a first continuous probability distribution; The first data restoration module is configured to input the first implicit feature into a decoder so as to restore the first implicit feature through the decoder to obtain first generated data.

Images