A data denoising method and related device

Abstract

The embodiment of the application discloses a data denoising method and related equipment, which can use artificial intelligence technology to denoise data. Any target denoising operation in at least one denoising operation performed on noisy data includes: generating distribution information corresponding to the target denoising operation according to first prediction information and second prediction information, the first prediction information indicating the predicted noise between the second noisy data and the clean data, the second prediction information indicating the square of the predicted noise between the second noisy data and the clean data, or indicating the square of the predicted distance between the first prediction information and the actual noise, the actual noise including the actual noise between the second noisy data and the clean data; sampling the denoised data from the distribution space pointed by the distribution information. Since the distribution information corresponding to the denoising operation is not directly learned, the number of denoising operations performed on the noisy data is not constrained by the training stage, and the flexibility of the inference stage is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a data denoising method and related equipment. BACKGROUND

[0002] Diffusion probabilistic model (DPM) has achieved good results on many generation tasks, such as generating high-resolution images, converting text to images, converting text to speech, generating speech, etc. The foregoing processes can be achieved by performing denoising operations on noisy data using diffusion probabilistic model.

[0003] Currently, in the process of denoising using diffusion probabilistic model, the noisy data is input into the diffusion probabilistic model, the mean value corresponding to the current denoising operation is determined based on the data output by the diffusion probabilistic model; the noisy data is input into the target neural network to obtain the covariance corresponding to the current denoising operation output by the target neural network; a denoised data corresponding to the noisy data is sampled from the distribution space pointed to by the mean value and the covariance corresponding to the current denoising operation.

[0004] In the training process of the diffusion probabilistic model and the above neural network, Gaussian noise is added to the clean data one by one, and after performing M-step noise adding operation, the clean data is gradually transformed into noisy data. The value of s is generally several hundred to several thousand. In order to improve the efficiency of the denoising process, the number of denoising operations can be less than the number of noise adding operations, but since the target neural network outputs the covariance corresponding to the denoising operation, the number of denoising operations performed by the target neural network in the inference stage needs to be consistent with the number of denoising operations in the training stage, that is, the flexibility of the trained target neural network is poor; as an example, if the target neural network learns to gradually transform the noisy data into clean data after performing 10 denoising operations in the training stage, it must also perform 10 denoising operations in the denoising stage.

[0005] In addition, performing 10 denoising operations may not necessarily achieve good denoising effect, but if you want to gradually transform the noisy data into clean data after performing 20 denoising operations, you need to retrain a new target neural network, which is a huge waste of computing resources. SUMMARY

[0006] The embodiment of the present application provides a data denoising method and related equipment. Distribution information corresponding to a denoising operation is generated based on first prediction information and second prediction information. Since the first feature processing network and the second feature processing network do not directly learn the distribution information corresponding to the denoising operation, the number of times of performing the denoising operation on the first noisy data is not constrained by the training stage. When the number of times of the denoising operation needs to be adjusted, the first feature processing network and the second feature processing network do not need to be retrained. The denoising scheme provided by the embodiment of the present application avoids the computer resource overhead caused by repeated training of the neural network, and improves the flexibility of the inference stage.

[0007] To solve the above technical problems, the embodiment of the present application provides the following technical solutions:

[0008] In a first aspect, the embodiment of the present application provides a data denoising method, which can use artificial intelligence technology to denoise data. The method is used to perform at least one denoising operation on the first noisy data by the execution device to obtain clean data corresponding to the first noisy data. Any target denoising operation in the at least one denoising operation includes:

[0009] The execution device obtains first prediction information, wherein the first prediction information indicates the predicted noise between the second noisy data and the clean data. The first prediction information is generated by the first feature processing network. If the target denoising operation is the first denoising operation, the second noisy data is the first noisy data. If the at least one denoising operation is at least two denoising operations, and the target denoising operation is not the first denoising operation, the second noisy data is the noisy data after at least one denoising operation is performed on the first noisy data.

[0010] The execution device obtains second prediction information, wherein the second prediction information indicates the square of the predicted noise between the second noisy data and the clean data. The square of the predicted noise between the second noisy data and the clean data represents the square of each value in the predicted noise between the second noisy data and the clean data. Alternatively, the second prediction information indicates the square of the predicted distance between the first prediction information and the actual noise. The actual noise includes the actual noise between the second noisy data and the clean data. The second prediction information is generated by the second feature processing network. The feature information processed by the first feature processing network and the second feature processing network is obtained based on the second noisy data.

[0011] The execution device generates distribution information corresponding to the target denoising operation according to the first prediction information and the second prediction information. Sampling operation is performed on the distribution space pointed by the distribution information to obtain denoised data corresponding to the second noisy data, that is, the denoised data after the target denoising operation is performed.

[0012] In the implementation, in the process of performing the denoising operation, the first feature processing network is used to generate first prediction information, i.e., predicted noise between the noisy data and the clean data; the second feature processing network is used to generate second prediction information, which indicates the square of the predicted noise between the noisy data and the clean data, or indicates the square of the distance between the first prediction information and the actual noise; and then, based on the first prediction information and the second prediction information, distribution information corresponding to the denoising operation is generated, and the denoised data corresponding to the noisy data is sampled from a distribution space pointed by the distribution information, i.e., one denoising operation in the M-step denoising operation is completed. Since the first feature processing network and the second feature processing network learn data between the noisy data and the clean data, i.e., the first feature processing network and the second feature processing network do not directly learn the distribution information corresponding to the denoising operation, the number of times of performing the denoising operation on the first noisy data is not restricted by the training phase. When the number of times of the denoising operation needs to be adjusted, the first feature processing network and the second feature processing network do not need to be retrained. The denoising scheme provided in the embodiment avoids the computer resource consumption caused by repeated training of the neural network, and improves the flexibility in the inference phase.

[0013] In a possible implementation of the first aspect, the at least one denoising operation is N denoising operations, and N is an integer greater than or equal to 1. Since, in the training phase, the pure noise is obtained by performing the M-step noise adding process on the clean data, the corresponding clean data can be obtained by performing the M-step denoising operation on the pure noise, and the most ideal state is that the denoised data obtained by each denoising operation is the complete inverse process of the corresponding noise adding operation. In order to improve the efficiency in the inference phase, only N denoising operations are performed in the inference phase in the present scheme, and the value of N is much smaller than s. Therefore, the purpose of each denoising operation in the N denoising operations is to obtain the noisy data corresponding to the s-step noise adding operation in the M-step denoising operation based on the noisy data corresponding to the n-step noise adding operation in the M-step noise adding operation, where n and s are positive integers, the value of n is greater than the value of s, and the difference between n and s is greater than or equal to 2. That is, each denoising operation in the N denoising operations includes the denoising effect of multiple denoising operations in the M-step denoising operation in the ideal state. The feature information processed by the first feature processing network and the second feature processing network is obtained based on the second noisy data and the value of n corresponding to the second noisy data,

[0014] If the first noisy data is the pure noise, the value of n can be divided by M. If the first noisy data is not the pure noise, the execution device can estimate the values of n and N corresponding to the first noisy data based on the first noisy data. The more noise carried in the first noisy data, the closer the value of n is to M, and the greater the value of N is. The more noise carried in the first noisy data, the smaller the values of n and N are.

[0015] In a possible implementation of the first aspect, the first feature processing network and the second feature processing network are configured to process the same feature information. Specifically, in one case, the first feature processing network and the second feature processing network adopt the same feature extraction network; in another case, the first feature extraction network included in the first neural network and the second feature extraction network included in the second neural network can be the same neural network.

[0016] In the implementation, the first feature processing network and the second feature processing network process the same feature information, that is, the first feature processing network and the second feature processing network correspond to the same feature extraction network, or the first feature processing network and the second feature processing network correspond to the same feature extraction network, that is, only one feature extraction network needs to be trained in the training phase, which is beneficial to improving the efficiency of the training phase.

[0017] In a possible implementation of the first aspect, the at least one denoising operation is N times of denoising operation, N is an integer greater than or equal to 1, and if the first noisy data is pure noise, N is a hyperparameter. The feature information processed by the first feature processing network includes feature information of the second noisy data and feature information of constraint information, and the clean data satisfies the constraint of the constraint information. In the implementation, the denoising operation can be continuously performed on the pure noise (that is, not carrying information in the clean data) based on the constraint information to obtain the clean data satisfying the constraint information, that is, the first neural network and the second neural network can generate the clean data satisfying the constraint information from nothing, which expands the channel for obtaining specific clean data.

[0018] In a possible implementation of the first aspect, if the distribution space corresponding to the denoising operation satisfies a Gaussian distribution or a Gamma distribution, the execution device generates the distribution information corresponding to the target denoising operation according to the first prediction information and the second prediction information, including: the execution device generates the mean value corresponding to the target denoising operation according to the first prediction information; and generates the covariance corresponding to the target denoising operation according to the second prediction information.

[0019] In the implementation, the details of generating the distribution information of each denoising operation according to the first prediction information and the second prediction information are further disclosed, which improves the realizability of the scheme.

[0020] In a possible implementation of the first aspect, the performing device generates the distribution information corresponding to the target denoising operation according to the first prediction information and the second prediction information, including: the performing device can determine the values of n and s corresponding to the Hth denoising operation (i.e., the target denoising operation) in the N times of denoising operations, and then generate the mean value corresponding to the Hth denoising operation (i.e., the target denoising operation) in the N times of denoising operations according to the first prediction information corresponding to the second noisy data, the value of n and the value of s, where the value of H is greater than or equal to 1. Correspondingly, the performing device can generate the covariance corresponding to the Hth denoising operation (i.e., the target denoising operation) in the N times of denoising operations according to the second prediction information corresponding to the second noisy data, the value of n and the value of s.

[0021] In a possible implementation of the first aspect, the distribution space pointed to by the distribution information is subject to a Gaussian distribution or a gamma distribution. In this implementation, the denoised data can be sampled in the distribution space subject to the Gaussian distribution or the gamma distribution, that is, whether the noise carried in the noisy data is Gaussian noise or gamma noise, the data denoising method provided in this application can be used for denoising, which expands the application scenarios of the present solution and improves the implementation flexibility of the present solution.

[0022] In a possible implementation of the first aspect, the clean data can be specifically expressed as any one of the following types of data: image data, sound data, text data, or other types of data, and the like. Further, the data sizes of the clean data, the first noisy data, and the second noisy data are the same, that is, the clean data, the noisy data, and the second noisy data are specifically H-dimensional tensors, and the lengths of each dimension in the H dimensions are the same.

[0023] In a possible implementation of the first aspect, the clean data can be specifically expressed as any one of the following types of data: image data, sound data, text data, or other types of data, and the like. Further, the data sizes of the clean data, the first noisy data, and the second noisy data are the same, that is, the clean data, the noisy data, and the second noisy data are specifically H-dimensional tensors, and the lengths of each dimension in the H dimensions are the same.

[0024] In the second aspect of the present application, the training device can also be used to perform the steps performed by the device in the first aspect and the various possible implementation manners of the first aspect. The specific implementation manners of the steps in the various possible implementation manners of the second aspect, the meanings of the terms, and the beneficial effects brought about can be referred to the first aspect, and will not be described here again.

[0025] In the third aspect, the embodiments of the present application provide a neural network training method, which can use artificial intelligence technology to denoise data. The method comprises the following steps: a training device inputs third noisy data into a feature extraction network to obtain first feature information generated by the feature extraction network, wherein the first feature information at least comprises feature information of the third noisy data;

[0026] The training device processes the first feature information through a second feature processing network to obtain second prediction information output by the second feature processing network, wherein the third noisy data is obtained after performing multiple noise adding operations on the first clean data, the second prediction information indicates the square of the predicted distance between the first prediction information and the actual noise, the first prediction information comprises the predicted noise between the third noisy data and the first clean data generated by the first neural network, the first neural network is used to generate the first prediction information based on the input third noisy data, and the actual noise comprises the actual noise between the third noisy data and the first clean data.

[0027] The training device trains the second feature processing network according to the second expected information corresponding to the third noisy data, the second prediction information, and a second loss function until a convergence condition is met, wherein the second loss function indicates the similarity between the second prediction information and the second expected information, and the second expected information indicates the square of the actual distance between the first prediction information and the actual noise.

[0028] In the third aspect of the present application, the training device can also be used to perform the steps performed by the device in the first aspect and the various possible implementation manners of the first aspect. The specific implementation manners of the steps in the various possible implementation manners of the third aspect, the meanings of the terms, and the beneficial effects brought about can be referred to the first aspect, and will not be described here again.

[0029] In the fourth aspect, the embodiments of the present application provide a data denoising device, which can use artificial intelligence technology to denoise data. The data denoising device is used to perform at least one denoising operation on the first noisy data to obtain clean data corresponding to the first noisy data. The data denoising device comprises an acquisition module configured to acquire first prediction information, wherein the first prediction information indicates the predicted noise between the second noisy data and the clean data, the first prediction information is generated by a first feature processing network, and the second noisy data is the first noisy data or the second noisy data is the noisy data obtained after performing at least one denoising operation on the first noisy data.

[0030] The acquisition module is further configured to acquire second prediction information, where the second prediction information indicates a square of a predicted noise between the second noisy data and the clean data, or the second prediction information indicates a square of a predicted distance between the first prediction information and an actual noise, the actual noise including an actual noise between the second noisy data and the clean data, the second prediction information is generated by a second feature processing network, and the feature information processed by the first feature processing network and the second feature processing network is both based on the second noisy data.

[0031] The generation module is configured to generate distribution information corresponding to a target denoising operation according to the first prediction information and the second prediction information, the target denoising operation being any one of the at least one denoising operation; and the sampling module is configured to sample the denoised data corresponding to the second noisy data from a distribution space pointed to by the distribution information.

[0032] In the fourth aspect of the present application, the data denoising device can also be used to perform the steps performed by the device in the first aspect and the possible implementation manners of the first aspect. The specific implementation manners of the steps in the possible implementation manners of the fourth aspect, the meanings of the terms, and the beneficial effects brought about can be referred to the first aspect, and will not be described here.

[0033] In the fifth aspect, the present application provides a neural network training device. The artificial intelligence technology can be used for data denoising. The neural network training device comprises: a feature extraction module configured to input third noisy data into a feature extraction network to obtain first feature information generated by the feature extraction network, the first feature information at least including feature information of the third noisy data; a feature processing module configured to process the first feature information through a second feature processing network to obtain second prediction information output by the second feature processing network, the second prediction information indicating a square of a predicted noise between the third noisy data and first clean data, the third noisy data being obtained by performing multiple noise adding operations on the first clean data.

[0034] The training module is configured to train the second feature processing network according to first expected information corresponding to the third noisy data, the second prediction information, and a first loss function until a convergence condition is met, where the first loss function indicates a similarity between the second prediction information and the first expected information, and the first expected information indicates a square of an actual noise between the third noisy data and the first clean data.

[0035] In the fifth aspect of the present application, the neural network training device can also be used to perform the steps performed by the device in the second aspect and the possible implementation manners of the second aspect. The specific implementation manners of the steps in the possible implementation manners of the fifth aspect, the meanings of the terms, and the beneficial effects brought about can be referred to the second aspect, and will not be described here.

[0036] In a sixth aspect, an embodiment of the present application provides a neural network training apparatus, which can use artificial intelligence technology to denoise data. The neural network training apparatus comprises: a feature extraction module, configured to input third noisy data into a feature extraction network to obtain first feature information generated by the feature extraction network, wherein the first feature information at least comprises feature information of the third noisy data; a feature processing module, configured to process the first feature information through a third feature processing network to obtain third prediction information output by the third feature processing network, wherein the third prediction information indicates a square of predicted noise between the third noisy data and first clean data, and the third noisy data is obtained by performing multiple noise adding operations on the first clean data;

[0037] a training module, configured to train the third feature processing network according to first expected information corresponding to the third noisy data, the third prediction information and a first loss function, until a convergence condition is met, wherein the first loss function indicates a similarity between the third prediction information and the first expected information, and the first expected information indicates a square of actual noise between the third noisy data and the first clean data.

[0038] In the sixth aspect of the present application, the neural network training apparatus can also be used to perform the steps performed by the device in the third aspect and the various possible implementation manners of the third aspect. The specific implementation manners of the steps in the various possible implementation manners of the sixth aspect, the meanings of the terms and the beneficial effects brought about are all referable to the third aspect, and will not be described here.

[0039] In a seventh aspect, an embodiment of the present application provides a computer program product, which comprises a program. When the program is run on a computer, the computer executes the method of the first aspect, the second aspect or the third aspect.

[0040] In an eighth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program. When the computer program is run on a computer, the computer executes the method of the first aspect, the second aspect or the third aspect.

[0041] In a ninth aspect, an embodiment of the present application provides an execution device, which comprises a processor and a memory. The processor is coupled to the memory. The memory is configured to store a program. The processor is configured to execute the program in the memory, so that the execution device executes the denoising method of the first aspect.

[0042] In a tenth aspect, an embodiment of the present application provides a training device, which comprises a processor and a memory. The processor is coupled to the memory. The memory is configured to store a program. The processor is configured to execute the program in the memory, so that the training device executes the neural network training method of the second aspect or the third aspect.

[0043] In an eleventh aspect, the present application provides a chip system, which comprises a processor for supporting a terminal device or a communication device to implement functions involved in the above aspects, e.g., transmitting or processing data and / or information involved in the above methods. In a possible design, the chip system further comprises a memory, and the memory is configured to store necessary program instructions and data of the terminal device or the communication device. The chip system can be composed of a chip, or can comprise a chip and other discrete devices. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1a A structural schematic diagram of an artificial intelligence subject framework provided for an embodiment of the present application;

[0045] Figure 1b An application scenario diagram of a data denoising method provided for an embodiment of the present application;

[0046] Figure 2a A system architecture diagram of a data denoising system provided for an embodiment of the present application;

[0047] Figure 2b A flow schematic diagram of a data denoising method provided for an embodiment of the present application;

[0048] Figure 3 A flow schematic diagram of a neural network training method provided for an embodiment of the present application;

[0049] Figure 4 A flow schematic diagram of a clean data denoising method provided for an embodiment of the present application;

[0050] Figure 5 A structural schematic diagram of a first neural network provided for an embodiment of the present application;

[0051] Figure 6 A structural schematic diagram of a second neural network provided for an embodiment of the present application;

[0052] Figure 7 A flow schematic diagram of a neural network training method provided for an embodiment of the present application;

[0053] Figure 8 A flow schematic diagram of a neural network training method provided for an embodiment of the present application;

[0054] Figure 9 A deployment schematic diagram of a neural network in a data denoising method provided for an embodiment of the present application;

[0055] Figure 10 A flow schematic diagram of a data denoising method provided for an embodiment of the present application;

[0056] Figure 11 A schematic diagram of performing a denoising operation in the denoising method of data provided in an embodiment of the present application;

[0057] Figure 12 A flowchart of the denoising method of data provided in an embodiment of the present application;

[0058] Figure 13 A comparison diagram of the clean data generated by the method provided in an embodiment of the present application and the diffusion probability model provided in an embodiment of the present application;

[0059] Figure 14 A structural diagram of the denoising device of data provided in an embodiment of the present application;

[0060] Figure 15 A structural diagram of the training device of the neural network provided in an embodiment of the present application;

[0061] Figure 16 A structural diagram of the training device of the neural network provided in an embodiment of the present application;

[0062] Figure 17 A structural diagram of the execution device provided in an embodiment of the present application;

[0063] Figure 18 A structural diagram of the training device provided in an embodiment of the present application;

[0064] Figure 19 A structural diagram of the chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The embodiments of the present application are described below in conjunction with the accompanying drawings. It is obvious to those skilled in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0066] The terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, which is only a distinguishing way adopted in the description of the embodiments of the present application for the objects with the same attribute in the description. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or devices containing a series of units do not have to be limited to those units, but can include other units not clearly listed or inherent to these processes, methods, products or devices.

[0067] First, the overall workflow of the artificial intelligence system is described, please see Figure 1a , Figure 1a A structural diagram of an artificial intelligence subject framework is shown, and the following describes the artificial intelligence subject framework from two dimensions of "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis). The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes a condensation process of "data-information-knowledge-wisdom". The "IT value chain" reflects the value brought by artificial intelligence to the information technology industry from the underlying infrastructure of human intelligence, information (provision and processing technology implementation) to the industrial ecological process of the system.

[0068] (1) Infrastructure

[0069] The infrastructure provides computing power support for the artificial intelligence system, realizes communication with the external world, and realizes support through the underlying platform. Communication with the outside world through sensors; computing power is provided by an intelligent chip, which can specifically use a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA) hardware acceleration chip; the underlying platform includes a distributed computing framework and related platform guarantees and supports such as network, which can include cloud storage and computing, interconnection networks, etc. For example, sensors and external communication acquire data, which are provided to the intelligent chip in the distributed computing system provided by the underlying platform for calculation.

[0070] (2) Data

[0071] The data on the upper layer of the infrastructure is used to represent the data source in the field of artificial intelligence. Data involves graphics, images, voice, text, and also involves Internet of Things data of traditional devices, including business data of existing systems and sensing data such as force, displacement, liquid level, temperature, and humidity.

[0072] (3) Data processing

[0073] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0074] Among them, machine learning and deep learning can model, extract, preprocess, train, etc. of the intelligent information of the symbolization and formalization of the data.

[0075] Reasoning refers to the process of simulating human intelligent reasoning methods in a computer or intelligent system, using formalized information to solve problems according to reasoning control strategies, and the typical function is search and matching.

[0076] Decision-making refers to the process of decision-making after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0077] (4) General ability

[0078] After the data is processed as mentioned above, some general abilities can be formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0079] (5) Intelligent products and industry applications

[0080] Intelligent products and industry applications refer to the products and applications of artificial intelligence systems in various fields, which are the packaging of the overall solution of artificial intelligence, and realize the application of intelligent information decision-making products. Its application fields mainly include: intelligent terminals, intelligent manufacturing, intelligent transportation, smart home, intelligent medical treatment, intelligent security, autonomous driving, smart city, etc.

[0081] The present application can be applied in various application fields of artificial intelligence, and is specifically used for performing a de-noising operation on noisy data in each application field to obtain clean data corresponding to the noisy data. Further, the noisy data can specifically represent pure noise, that is, the noisy data can not carry information in the clean data; or the noisy data can carry information in the clean data.

[0082] As an example, for example, in the field of intelligent terminals, an intelligent terminal can receive a text description input by a user, and show an image meeting the aforementioned text description to the user; the generation process of the image can include performing multiple de-noising operations on pure noise (which is an example of noisy data) based on the constraints of the text description (which is an example of constraint information), so as to obtain an image meeting the constraints of the text description (which is an example of clean data). For a more intuitive understanding of this application scenario, please refer to Figure 1b , Figure 1b An application scenario diagram of the de-noising method of data provided by the embodiment of the present application is shown in the figure. As shown in the figure, the text description is a boy with a short head on public transportation, wearing a Bluetooth headset and playing a mobile phone; then an image can be generated Figure 1bThe right side of the picture, should be understood, Figure 1b The examples in the foregoing merely facilitate the understanding of the present solution and are not used to limit the present solution.

[0083] As another example, for example in the field of intelligent security, the monitoring device set in the city can send the collected image to the control center after collecting the image. Noise can be introduced in the transmission process of the foregoing image, and the control center can obtain an image carrying noise (i.e., an example of noisy data). The image carrying noise can be denoised multiple times to obtain a clean image (i.e., an example of clean data).

[0084] As another example, for example in the field of smart home, the smart home can provide a translation function. If a Chinese text description is converted into English voice output, the noise (i.e., an example of noisy data) can be denoised based on the constraints (i.e., an example of constraint information) of the Chinese text description to obtain English voice corresponding to the text description (i.e., an example of clean data), and the like. It should be understood that the examples of application scenarios herein merely facilitate the understanding of the present solution and are not used to limit the present solution.

[0085] Before introducing the data denoising method provided by the embodiments of the present application, please refer to Figure 2a , Figure 2a is a system architecture diagram of the data denoising system provided by the embodiments of the present application, in Figure 2a , the data denoising system 200 includes a training device 210, a database 220, an execution device 230, a data storage system 240, and a client device 250. The execution device 230 includes a computing module 231.

[0086] The database 220 stores a training data set, and the training data set includes a plurality of noisy data. The training device 210 generates a first model / rule 201 and a second model / rule 202, and iteratively trains the first model / rule 201 and the second model / rule 202 using the training data set to obtain a trained first model / rule 201. The first model / rule 201 can be specifically a neural network or a non-neural network model. In the embodiments of the present application, only the first model / rule 201 and the second model / rule 202 are taken as examples of feature processing networks.

[0087] The trained first model / rule 201 and second model / rule 202 obtained by the training device 210 can be applied to the execution device 230 in different product forms. The execution device 230 can call data, code, etc. in the data storage system 240, or store data, instructions, etc. in the data storage system 240. The data storage system 240 can be placed in the execution device 230, or the data storage system 240 can be an external storage relative to the execution device 230.

[0088] The execution device 230 can use the first model / rule 201 and the second model / rule 202 in the computing module 231 to execute the data denoising method provided in the embodiments of the present application. Specifically, please refer to Figure 2b , Figure 2b The execution device 230 can use the first model / rule 201 and the second model / rule 202 in the computing module 231 to execute the data denoising method provided in the embodiments of the present application. Specifically, please refer to Figure 2b , Figure 2b A flowchart of the data denoising method provided in the embodiments of the present application. The foregoing method comprises: the execution device 230 performing at least one denoising operation on the first noisy data to obtain clean data corresponding to the first noisy data, any target denoising operation in the at least one denoising operation comprising: A1, the execution device 230 obtaining first prediction information, wherein the first prediction information indicates the predicted noise between the second noisy data and the clean data, the first prediction information is generated by a first feature processing network, the second noisy data is the first noisy data, or the second noisy data is the noisy data after at least one denoising operation is performed on the first noisy data; A2, the execution device 230 obtains second prediction information, wherein the second prediction information indicates the square of the predicted noise between the second noisy data and the clean data, or the second prediction information indicates the square of the distance between the first prediction information and the actual noise, the actual noise comprises the actual noise between the second noisy data and the clean data, the second prediction information is generated by a second feature processing network, and the feature information processed by the first feature processing network and the second feature processing network is obtained based on the second noisy data; A3, the execution device 230 generates distribution information corresponding to the target denoising operation according to the first prediction information and the second prediction information; A4, the execution device 230 samples the denoised data corresponding to the first noisy data from the distribution space pointed by the distribution information.

[0089] Among them, since the execution device 230 needs to perform N times of denoising operation to obtain the clean data, if the target denoising operation is the Nth denoising operation, the foregoing denoised data can be expressed as clean data; if the target denoising operation is not the Nth denoising operation, the denoised data obtained after performing once denoising operation on the first noisy data can be expressed as noisy data.

[0090] In the embodiments of the present application, since the execution device 230 generates data between the noisy data and the clean data in the process of performing each denoising operation by the first feature processing network and the second feature processing network, that is, the first feature processing network and the second feature processing network do not directly generate the distribution information corresponding to the denoising operation, the value of N is not restricted by the training stage, and when it is necessary to adjust the number of denoising operations, it is not necessary to retrain the first feature processing network and the second feature processing network. The denoising scheme provided in the embodiments of the present application avoids the computer resource overhead caused by repeated training of the neural network, and improves the flexibility of the inference stage.

[0091] In some embodiments of the present application, referring to Figure 2a , the execution device 230 and the client device 250 can be independent devices, and the execution device 230 is configured with an input / output (I / O) interface to interact with the client device 250. A "user" can directly interact with the client device 250, and the client device 250 communicates data with the execution device 230 through the I / O interface. After the execution device 230 generates clean data by using the first model / rule 201 and the second model / rule 202 in the computing module 231, the execution device 230 can send the aforementioned clean data to the client device 250 through the I / O interface to provide the user.

[0092] It is worth noting that Figure 2a is only an architecture diagram of a data denoising system provided by the embodiments of the present application, and the positional relationship between the devices, components, modules and the like shown in the diagram does not constitute any limitation. For example, in another embodiment of the present application, the execution device 230 can be configured in the client device 250. For example, when the client device is a mobile phone or a tablet, the execution device 230 can be a module for performing denoising operation in the host CPU of the mobile phone or the tablet. The execution device 230 can also be a neural network processor (NPU) or a graphics processing unit (GPU) in the mobile phone or the tablet. The NPU or the GPU is mounted on the host CPU as a coprocessor and is assigned tasks by the host CPU.

[0093] In combination with the above description, the specific implementation process of the training stage and the application stage of the neural network provided in the embodiments of the present application will be described below.

[0094] I. Training stage

[0095] In an embodiment of the present application, in one case, the second prediction information indicates the square of the predicted noise between the noisy data and the clean data; in another case, the second prediction information indicates the square of the distance between the first prediction information and the actual noise, and the implementation processes corresponding to the above two cases are different. The above two cases are described below.

[0096] (I) The second prediction information indicates the square of the predicted noise between the noisy data and the clean data

[0097] Please refer to Figure 3 , Figure 3 A flowchart of a training method of a neural network provided by an embodiment of the present application can include the following steps.

[0098] 301. The training device inputs the fourth noisy data into the first feature extraction network to obtain second feature information generated by the first feature extraction network, and the second feature information at least includes feature information of the fourth noisy data.

[0099] In an embodiment of the present application, the training device is configured with a training data set, and the training data set includes a plurality of training data. Each training data can include noisy data for training and one clean data corresponding to the noisy data for training.

[0100] Specifically, for obtaining the noisy data corresponding to the clean data, the electronic device can add noise conforming to the target data distribution to the clean data after obtaining the clean data to obtain noisy data after performing one noise adding operation on the clean data, and then add noise conforming to the target data distribution to the aforementioned noisy data to obtain noisy data after performing two noise adding operations on the clean data. After the electronic device repeatedly performs the aforementioned noise adding operation M times, the noisy data generated after performing the Mth noise adding operation is obtained.

[0101] For a more intuitive understanding of the present scheme, please refer to Figure 4 , Figure 4 A flowchart of a noise adding operation on clean data provided by an embodiment of the present application is shown in the figure. As shown in the figure, after performing M-step noise adding operation on the clean data, the noisy data used for performing the training operation can be obtained. In the process of each noise adding operation, noise conforming to the target data distribution can be added to the noisy data obtained after performing the previous noise adding operation. It should be understood that Figure 4 The examples in the above description are only for facilitating understanding of the present scheme, and are not used to limit the present scheme.

[0102] The electronic device can also obtain M-1 pieces of noisy data generated when performing the M-1 steps of the noise adding operation. It should be noted that the electronic device performing the noise adding operation on the clean data and the training device can be the same device. Alternatively, the electronic device and the training device can be two independent devices, that is, the training device receives the training data sent by the electronic device.

[0103] During the process of performing multiple noise adding operations on the clean data, the generated noisy data will continuously lose information in the clean data. The noisy data obtained after performing the M steps of the noise adding operation can specifically be pure noise, that is, the noisy data input into the first neural network does not carry information in the clean data.

[0104] One training data can include noisy data obtained after performing n steps of noise adding operation on clean data, the value of n, and the clean data. The value of n is greater than 0 and less than or equal to M. Alternatively, the value of n can be determined in relation to the divisors between M and N. For example, the value of M is 1000 and the value of N is 10, and the value of n can be 1000, 900, 800, 700, 600, 500, 400, 300, 200, and 100. It should be understood that the above examples are only for the convenience of understanding the value of n and are not used to limit the present solution.

[0105] The target data distribution can be a Gaussian distribution, a gamma distribution, or other types of distributions, and the like. The noise conforming to the Gaussian distribution can also be referred to as “Gaussian noise”, the noise conforming to the gamma distribution can also be referred to as “gamma noise”, and the like. In the present embodiment, the de-noised data can be sampled in a distribution space conforming to the Gaussian distribution or the gamma distribution, that is, whether the noise carried in the noisy data is Gaussian noise or gamma noise, the de-noising method provided in the present embodiment can be used for de-noising, which expands the application scenarios of the present solution and improves the implementation flexibility of the present solution.

[0106] The clean data can specifically be used to express any one of the following types of data: image data, sound data, text data, or other types of data, and the like.

[0107] Further, the clean data, the noisy data, and the noise used in the above denoising operation have the same data size, that is, the clean data, the noisy data, and the noise used in the above denoising operation are all tensors of H dimensions, and the length of each dimension in the H dimensions is the same. As an example, if the clean data is a vector, the noisy data and the noise used in the above denoising operation are also vectors, and the number of values carried in the clean data, the noisy data, and the noise used in the above denoising operation is the same; as another example, if the clean data is a matrix, the noisy data and the noise used in the above denoising operation are also matrices, and the length and width of the clean data, the noisy data, and the noise used in the above denoising operation are the same in the two dimensions of length and width. It should be understood that the examples herein are only for the convenience of understanding the concept of "the same data size" and do not limit the present solution.

[0108] In some embodiments of the present application, the training device can train the first neural network and the second neural network separately, that is, after generating the trained first neural network, the second neural network is trained; or, after generating the trained second neural network, the first neural network is trained. In a training process of the first neural network, the training device can obtain fourth noisy data from the training data set, input the fourth noisy data and the value of n corresponding to the fourth noisy data into the first feature extraction network, and obtain the second feature information generated by the first feature extraction network, wherein the second feature information at least includes the feature information of the fourth noisy data.

[0109] The first neural network can include a first feature extraction network and a first feature processing network, that is, the formula of the first neural network can be expressed as x n represents the noisy data input into the first neural network, n represents the number of noise addition operations performed to obtain the noisy data, head(.) is the first feature processing network, and emb(.) is the first feature extraction network.

[0110] Further, the first feature extraction network can use a convolutional neural network, a recurrent neural network, a residual neural network, or other types of neural networks, and correspondingly, the first feature processing network can also use any type of neural network. The specific structure of the first neural network can be determined in combination with the type of data to be processed.

[0111] As an example, the first feature extraction network can specifically use a U-shaped neural network (Unet), and the first feature processing network can specifically use a diffusion probabilistic model (DPM). For a more intuitive understanding of the present solution, please refer to Figure 5 , Figure 5A structural schematic diagram of the first neural network provided in the embodiments of the present application is shown in FIG. 1. As shown in FIG. 1, the first neural network includes a first feature extraction network adopting a Unet, and the first neural network includes a first feature processing network which can adopt a DPM model. It should be understood that Figure 5 the examples in the above are only for facilitating the understanding of the present application, and are not used to limit the present application. Figure 5

[0112] Optionally, the training device can input the fourth noisy data, the value of n corresponding to the fourth noisy data, and the second constraint information into the first feature extraction network to obtain second feature information generated by the first feature extraction network. The second feature information includes feature information of the fourth noisy data and the second constraint information. The fourth noisy data can be pure noise or can carry information in the second clean data.

[0113] The second clean data satisfies the constraint of the second constraint information. For example, the second constraint information is a text description, the second clean data is a voice reading out the text description in the voice of a specific person, and the fourth noisy data is pure noise used to express the voice data. For another example, the second constraint information is a text description, the second clean data is an image consistent with the text description, and the fourth noisy data is pure noise used to express the image data, and the like. It should be noted that the above examples are only for facilitating the understanding of the concept of the constraint information, and are not used to limit the present application.

[0114] 302. The training device processes the second feature information through the first feature processing network to obtain first prediction information output by the first feature processing network. The first feature extraction network and the first feature processing network are included in the first neural network.

[0115] In the embodiments of the present application, after the training device generates the second feature information through the first feature extraction network, the training device can process the second feature information through the first feature processing network to obtain first prediction information output by the first feature processing network, that is, first prediction information output by the entire first neural network. The first prediction information indicates predicted noise between the fourth noisy data and the second clean data, and the data size of the first prediction information is the same as that of the noisy data.

[0116] 303. The training device trains the first neural network according to the second expected information corresponding to the fourth noisy data, the first prediction information, and a target loss function. The target loss function indicates the similarity between the first prediction information and the second expected information, and the second expected information indicates actual noise between the fourth noisy data and the second clean data.

[0117] ​In the embodiment of the present application, the training device can obtain second expected information corresponding to the fourth noisy data based on the training data to which the fourth noisy data belongs, the second expected information indicating the actual noise between the fourth noisy data and the second clean data. The training device calculates a function value of a target loss function according to the second expected information and the first predicted information, the target loss function indicating the similarity between the first predicted information and the second expected information.

[0118] The training device performs gradient derivation on the function value of the target loss function and reversely updates the parameters of the first neural network to realize one training of the first neural network. The training device can repeatedly perform steps 301 to 303 to realize iterative training of the first neural network until a first convergence condition is met, and obtain the trained first neural network, that is, the trained first feature extraction network and the trained first feature processing network. The first convergence condition can be a convergence condition of the target loss function, a number of times of iterative training of the first neural network reaching a preset number, or other types of convergence conditions, which are not limited here.

[0119] 304. The training device inputs the third noisy data into the second feature extraction network to obtain first feature information generated by the second feature extraction network, the first feature information at least including feature information of the third noisy data.

[0120] In the embodiment of the present application, the training device can initialize the second neural network, the second neural network can include the second feature extraction network and the second feature processing network. Since the second neural network and the first neural network output different predicted information, the second neural network and the first neural network include different feature processing networks.

[0121] For the second feature extraction network in the second neural network, in one implementation manner, if the training device first trains the first neural network to obtain the trained first neural network, the trained first feature extraction network can be used as the second feature extraction network of the second neural network, and the parameters of the second feature extraction network remain unchanged in the training process of the second neural network, that is, the parameters of the second feature extraction network are not updated in the training process of the second neural network, so that the trained first neural network and the second neural network include the same feature extraction network.

[0122] The formula of the second neural network can be expressed as h n (x n )=head φ (emb(x n ,n)),x nThis represents the noisy data input to the second neural network, where n represents the number of noise-adding operations performed to obtain the noisy data, and emb(.) represents the second feature extraction network (i.e., the trained first feature extraction network). During the training of the second neural network, the parameters of the second feature extraction network can be kept constant. head φ (.) represents the second feature processing network.

[0123] For a more intuitive understanding of this solution, please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram of a second neural network provided in an embodiment of this application. Figure 6 As shown, the second feature extraction network in the second neural network adopts Unet, and the second feature extraction network can be derived from the trained first neural network. During the training of the second neural network, the parameters of the second feature extraction network can be kept unchanged, while the parameters of the second feature processing network are updated. It should be understood that... Figure 6 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0124] In another implementation, the training device first trains the first neural network to obtain the trained first neural network. Then, the trained first feature extraction network can be used as the second feature extraction network of the second neural network, and the parameters of the second feature extraction network will also be updated during the training of the second neural network.

[0125] In another implementation, the training device can reinitialize the second feature extraction network in the second neural network, where the second feature extraction network and the first feature extraction network are different neural networks. It should be noted that the specific type of neural network represented by the second feature extraction network and the second feature processing network can be found in the description in step 301, and will not be repeated here.

[0126] In this embodiment of the application, the training device obtains the third noisy data from the training dataset, inputs the third noisy data and the value of n corresponding to the third noisy data into the second feature extraction network, and obtains the first feature information generated by the second feature extraction network. The first feature information includes at least the feature information of the third noisy data.

[0127] Optionally, the training device can input the third noisy data, the value of n corresponding to the third noisy data, and the first constraint information into the second feature extraction network to obtain the first feature information generated by the second feature extraction network. The first feature information includes feature information from the third noisy data and the first constraint information; the third noisy data can be pure noise or it can carry information from the first clean data.

[0128] It should be noted that if the first neural network and the second neural network adopt the same feature extraction network, the data input into the first neural network and the second neural network is the same; if the first neural network and the second neural network adopt different feature extraction networks, the constraint information can be input into the first neural network, and the constraint information is not input into the second neural network; or the constraint information can be input into both the first neural network and the second neural network.

[0129] 305、The training device processes the first feature information through the second feature processing network to obtain second prediction information output by the second feature processing network, the second prediction information indicating a square of a predicted noise between the third noisy data and the first clean data.

[0130] In the embodiment of the application, after the training device generates the first feature information through the second feature processing network, the training device can process the first feature information through the second feature processing network to obtain the second prediction information output by the second feature processing network, that is, the second prediction information output by the entire second neural network.

[0131] The second prediction information has the same data size as the noisy data, and the second prediction information indicates a square of a predicted noise between the third noisy data and the first clean data. Further, the square of the predicted noise between the third noisy data and the first clean data can be a square of each value in the predicted noise between the third noisy data and the first clean data.

[0132] 306、The training device trains the second feature processing network according to the first expected information corresponding to the third noisy data, the second prediction information, and the first loss function, wherein the first loss function indicates a similarity between the second prediction information and the first expected information, and the first expected information indicates a square of an actual noise between the third noisy data and the first clean data.

[0133] In the embodiment of the application, the training device can obtain the second expected information corresponding to the third noisy data based on the training data to which the third noisy data belongs, and the first expected information indicates a square of an actual noise between the third noisy data and the first clean data. The training device calculates a function value of the first loss function according to the first expected information and the first prediction information, and the first loss function indicates a similarity between the second prediction information and the first expected information.

[0134] The first loss function can adopt a cosine similarity between the second prediction information and the first expected information, or the first loss function can adopt an L2 distance, an Euclidean distance, or other types of formulas between the second prediction information and the first expected information, which is not exhaustively listed here.

[0135] For example, the formula of the first loss function can be h n (x n ) represents second prediction information output by the second neural network, represents the square of the actual noise between the noisy data and the clean data (i.e., the second expected information), represents the mean square error between the second prediction information and the second expected information (i.e., the L2 distance between the second prediction information and the first expected information), it should be understood that the examples herein are only for proving the feasibility of the scheme and are not used to limit the scheme.

[0136] The process of updating the parameters in the second neural network of the training device. In one implementation, the second feature extraction network is derived from the trained first neural network, i.e., the second feature extraction network adopts the trained first feature extraction network, then the training device can perform gradient derivation on the function value of the first loss function, and keep the parameters of the second feature extraction network unchanged, update the parameters of the second feature processing network in the second neural network, to realize one training of the second neural network.

[0137] In another implementation, the training device can perform gradient derivation on the function value of the first loss function, and update the parameters of the entire second neural network in reverse, to realize one training of the second neural network.

[0138] The training device can repeatedly perform steps 304 to 306 to realize iterative training of the second neural network, until a second convergence condition is met, to obtain a trained second neural network, i.e., a trained second feature processing network; wherein the second convergence condition can be that the convergence condition of the first loss function is met, the number of iterations of the second neural network reaches a preset number, or other types of convergence conditions, etc., which are not limited herein.

[0139] It should be noted that the embodiments of the present application do not limit the execution order of steps 301 to 306, if the first neural network and the second neural network are trained separately, steps 304 to 306 can be executed first, and then steps 301 to 303 can be executed, then the first feature extraction network in the first neural network can adopt the trained second feature extraction network; if the first neural network and the second neural network are trained simultaneously, the same noisy data can be input into the first neural network and the second neural network respectively, and then the first neural network and the second neural network can be trained.

[0140] Further, in the inference stage, the first prediction information (i.e., the predicted noise between the noisy data and the clean data) output by the first feature processing network and the second prediction information output by the second feature processing network are used to jointly generate distribution information corresponding to the denoising operation, and then the denoised data (i.e., the denoised data obtained after performing the denoising operation once) corresponding to the noisy data can be sampled from the distribution space pointed by the distribution information. The specific implementation principle will be described in detail in the method of the inference stage.

[0141] (ii) The second prediction information indicates the square of the distance between the first prediction information and the actual noise

[0142] Please refer to Figure 7 , Figure 7 A flowchart of the training method of the neural network provided by the embodiments of the present application. The training method of the neural network provided by the embodiments of the present application can include:

[0143] 701. The training device inputs the fourth noisy data into the first feature extraction network to obtain second feature information generated by the first feature extraction network, and the second feature information at least includes the feature information of the fourth noisy data.

[0144] 702. The training device processes the second feature information through the first feature processing network to obtain the first prediction information corresponding to the fourth noisy data output by the first feature processing network, and the first prediction information corresponding to the fourth noisy data indicates the predicted noise between the fourth noisy data and the second clean data. The first feature extraction network and the first feature processing network are included in the first neural network.

[0145] 703. The training device trains the first neural network according to the second expected information corresponding to the fourth noisy data, the first prediction information, and the target loss function. The target loss function indicates the similarity between the first prediction information corresponding to the fourth noisy data and the second expected information, and the second expected information indicates the actual noise between the fourth noisy data and the second clean data.

[0146] In the embodiments of the present application, the specific implementation modes of steps 701 to 703 can be referred to Figure 3 The specific meanings of the nouns in steps 701 to 703 can be referred to Figure 3 the description of the corresponding embodiments, which will not be described here in detail.

[0147] 704. The training device inputs the third noisy data into the first neural network to obtain the first prediction information corresponding to the third noisy data output by the first neural network, and the first prediction information corresponding to the third noisy data indicates the predicted noise between the third noisy data and the first clean data.

[0148] In some embodiments of this application, if the training device trains the first neural network and the second neural network simultaneously, that is, inputs the same noisy data into the first neural network and the second neural network respectively, then steps 701 to 703 can be skipped and step 704 can be executed directly; if the training device trains the first neural network first and then trains the second neural network, then in order to generate the second expected information, the third noisy data needs to be input into the trained first neural network to obtain the first prediction information corresponding to the third noisy data output by the trained first neural network; the first prediction information corresponding to the third noisy data indicates the prediction noise between the third noisy data and the first clean data.

[0149] For details on how to implement step 704, please refer to [link / reference]. Figure 3 For the specific meanings of the terms in step 704, which correspond to the descriptions in steps 301 and 302 in the corresponding embodiments, please refer to [link / reference]. Figure 3 The descriptions in the corresponding embodiments will not be repeated here.

[0150] 705. The training device inputs the third noisy data into the second feature extraction network to obtain the first feature information generated by the second feature extraction network. The first feature information includes at least the feature information of the third noisy data.

[0151] In this embodiment of the application, the specific implementation of step 705 can be found in [reference needed]. Figure 3 For the specific meanings of the terms in step 705, which correspond to the description in step 304 of the embodiment, please refer to [link / reference]. Figure 3 The descriptions in the corresponding embodiments will not be repeated here.

[0152] 706. The training device processes the first feature information through the second feature processing network to obtain the second prediction information output by the second feature processing network. The second prediction information indicates the square of the prediction distance between the first prediction information and the first actual noise. The first actual noise includes the actual noise between the third noisy data and the first clean data.

[0153] In this embodiment, the training device can process the first feature information through the second feature processing network to obtain the second prediction information output by the second feature processing network, that is, to obtain the second prediction information output by the entire second neural network; wherein, in this embodiment, the second prediction information indicates the square of the prediction distance between the first prediction information and the first actual noise, and the first actual noise includes the actual noise between the third noisy data and the first clean data.

[0154] 707、The training device trains the second feature processing network according to the second expected information corresponding to the third noisy data, the second predicted information, and the second loss function, where the second loss function indicates the similarity between the second predicted information and the second expected information, and the second expected information indicates the square of the actual distance between the first predicted information corresponding to the third noisy data and the first actual noise.

[0155] In the embodiments of the present application, the training device can generate the second expected information according to the first predicted information corresponding to the third noisy data and the first actual noise, where the second expected information indicates the square of the actual distance between the first predicted information corresponding to the third noisy data and the first actual noise. The training device generates the function value of the second loss function according to the second expected information corresponding to the third noisy data and the second predicted information, where the second loss function indicates the similarity between the second predicted information corresponding to the third noisy data and the second expected information.

[0156] The actual distance between the first predicted information corresponding to the third noisy data and the first actual noise can be specifically the residual error, the cosine distance, the L2 distance, or other distances, and the like, which are not exhaustively listed herein.

[0157] The second loss function can specifically adopt the cosine similarity between the second predicted information and the second expected information, or the first loss function can adopt the L2 distance, the Euclidean distance, or other types of formulas, and the like, which are not exhaustively listed herein.

[0158] As an example, the formula of "the square of the actual distance between the first predicted information corresponding to the third noisy data and the first actual noise" can be x n represents the input noisy data, represents the predicted information generated by the first neural network (i.e., the predicted noise between the noisy data and the clean data), ∈ n represents the actual noise between the input noisy data and the clean data, represents the square of the residual error between the predicted information generated by the first neural network and the actual noise.

[0159] For example, the formula of the second loss function can be x n represents the input noisy data in the second neural network, g n (x n ) represents the second predicted information output by the second neural network, represents the square of the residual error between the predicted information generated by the first neural network and the actual noise (i.e., the second expected information), The L2 distance between the second prediction information and the second expected information, and it should be understood that the examples of the actual distance between the first prediction information corresponding to the third noisy data and the first actual noise and the second loss function are only for the convenience of understanding the scheme, and are not used to limit the scheme.

[0160] The process of updating the parameters in the second neural network. In one implementation, the second feature extraction network is derived from the trained first neural network, that is, the second feature extraction network adopts the trained first feature extraction network, and the training device can perform gradient derivation on the function value of the second loss function, keep the parameters of the second feature extraction network unchanged, update the parameters of the second feature processing network in the second neural network, and implement one training of the second neural network.

[0161] In another implementation, the training device can perform gradient derivation on the function value of the second loss function, and update the parameters of the entire second neural network in reverse to implement one training of the second neural network.

[0162] The training device can repeatedly perform steps 704 to 707 to implement iterative training of the second neural network until a third convergence condition is met, and obtain a trained second neural network, that is, a trained second feature processing network; wherein the third convergence condition can be a convergence condition of the second loss function, a number of iterations of the second neural network reaching a preset number, or other types of convergence conditions, etc., which are not limited here.

[0163] It should be noted that if the training device synchronously trains the first neural network and the second neural network, steps 701 to 704 do not need to be performed, and the training device also needs to train the first neural network according to the second expected information corresponding to the third noisy data, the first prediction information corresponding to the third noisy data, and the target loss function. The target loss function indicates the similarity between the first prediction information and the second expected information, and the second expected information corresponding to the third noisy data indicates the actual noise between the third noisy data and the first clean data.

[0164] For a more intuitive understanding of the scheme, please refer to Figure 8 , Figure 8 A flowchart of a neural network training method provided by an embodiment of the present application. Figure 8 The training process of the second neural network is shown in FIG. 8. In the training phase, after performing M times of noise adding operations on each clean data x0, the pure noise corresponding to the clean data x0 can be obtained, and M noisy data can be obtained in the M times of noise adding operations. One or more noisy data x n The training device can first use the plurality of noisy data x nThe first neural network is trained to obtain a trained first neural network, and the trained first neural network includes a trained first feature extraction network and a trained first feature processing network.

[0165] The training device determines the trained first feature extraction network as a second feature extraction network included in the second neural network, keeps parameters of the second feature extraction network unchanged, and determines the trained first feature processing network as the second feature processing network included in the second neural network. n The parameters of the second feature extraction network are updated. For details of training of the first neural network and the second neural network, refer to the description in the foregoing embodiments, which will not be described here. It should be understood that, Figure 8 The examples in the foregoing merely serve to facilitate understanding of the solutions, and should not be used to define the solutions.

[0166] II. Inference phase

[0167] In the embodiments of the present application, the trained first feature processing network and the trained second feature processing network are deployed on the execution device. In one implementation manner, as shown in Figure 9 , the trained first feature processing network and the trained second feature processing network are deployed on the execution device. Figure 9 is a deployment diagram of a neural network in a data denoising method provided in the embodiments of the present application. As shown in Figure 9 , the trained first feature processing network and the trained second feature processing network are deployed on the execution device. That is, the first feature processing network and the second feature processing network adopt the same feature extraction network, and the first feature processing network and the second feature processing network are used to process the same feature information. It should be understood that, Figure 9 The examples in the foregoing merely serve to facilitate understanding of the solutions, and the first feature processing network and the second feature processing network can also be deployed on the execution device in other manners, which will not be limited here.

[0168] In another implementation manner, the first neural network and the second neural network are respectively deployed on the execution device. The first feature extraction network included in the first neural network and the second feature extraction network included in the second neural network can be the same neural network or different neural networks. If the first feature extraction network and the second feature extraction network are the same neural network, the first feature processing network and the second feature processing network are used to process the same feature information.

[0169] In the embodiments of the present application, the first feature processing network and the second feature processing network process the same feature information, that is, the first feature processing network and the second feature processing network correspond to the same feature extraction network, or the first feature processing network and the second feature processing network correspond to the same feature extraction network. That is, only one feature extraction network needs to be trained in the training phase, which is beneficial to improving the efficiency of the training phase.

[0170] Specifically, refer to Figure 10 , Figure 10 A flowchart of a data denoising method provided by an embodiment of the present application, the data denoising method provided by the embodiment of the present application can include:

[0171] 1001, the execution equipment generates first prediction information corresponding to the first noisy data through the first feature processing network, and the first prediction information corresponding to the first noisy data indicates the predicted noise between the first noisy data and the target clean data.

[0172] In the embodiment of the present application, if the execution equipment performs at least one denoising operation on the first noisy data to obtain the target clean data corresponding to the first noisy data, the execution equipment needs to generate the first prediction information corresponding to the first noisy data through the first feature processing network, and the first prediction information corresponding to the first noisy data indicates the predicted noise between the first noisy data and the target clean data.

[0173] Among them, the first noisy data can be pure noise, that is, the first noisy data does not carry information in the target clean data; or the first noisy data can also carry information in the target clean data.

[0174] Specifically, in one implementation, if the first feature processing network and the second feature processing network adopt the same feature extraction network, the execution equipment can determine the value of n corresponding to the first noisy data, input the first noisy data and the value of n corresponding to the first noisy data into the aforementioned feature extraction network, and obtain the target feature information generated by the aforementioned feature extraction network corresponding to the first noisy data, the target feature information at least includes the feature information of the first noisy data.

[0175] Among them, if the first noisy data is pure noise, the value of n can be M, and the value of N is a preconfigured hyperparameter; if the first noisy data is not pure noise, the execution equipment can estimate the values of n and N based on the first noisy data; the more noise carried in the first noisy data, the closer the value of n to M, and the larger the value of N; the more noise carried in the first noisy data, the smaller the values of n and N.

[0176] Optionally, the execution equipment inputs the first noisy data, the value of n corresponding to the first noisy data and the target constraint information into the aforementioned feature extraction network, and obtains the target feature information generated by the aforementioned feature extraction network, the target feature information includes the feature information of the first noisy data and the feature information of the target constraint information.

[0177] The execution equipment processes the target feature information through the first feature processing network to obtain the first prediction information corresponding to the first noisy data generated by the first feature processing network.

[0178] In another implementation, if the first neural network and the second neural network are respectively deployed on the execution device, the execution device can input the first noisy data into the first neural network to obtain target feature information corresponding to the first noisy data generated by a first feature extraction network in the first neural network, the target feature information at least including feature information of the first noisy data.

[0179] Optionally, the execution device inputs the first noisy data and the target constraint information into the first neural network to obtain target feature information generated by the first feature extraction network in the first neural network, the target feature information including feature information of the first noisy data and feature information of the target constraint information.

[0180] The execution device processes the target feature information through a first feature processing network in the first neural network to obtain first prediction information corresponding to the first noisy data generated by the first neural network.

[0181] 1002、The execution device generates second prediction information corresponding to the first noisy data through the second feature processing network, the second prediction information corresponding to the first noisy data indicating a square of a predicted noise between the first noisy data and the target clean data, or the second prediction information indicating a square of a predicted distance between the first prediction information and a target actual noise, the target actual noise including an actual noise between the first noisy data and the target clean data.

[0182] In the embodiments of the present application, the execution device needs to generate second prediction information corresponding to the first noisy data through the second feature processing network, the second prediction information corresponding to the first noisy data indicating a square of a predicted noise between the first noisy data and the target clean data, or the second prediction information indicating a square of a predicted distance between the first prediction information and a target actual noise, the target actual noise including an actual noise between the first noisy data and the target clean data.

[0183] Specifically, in one implementation, if the first feature processing network and the second feature processing network adopt the same feature extraction network, the execution device can obtain target feature information corresponding to the first noisy data generated by the aforementioned feature extraction network; and process the aforementioned target feature information through the second feature processing network to obtain second prediction information corresponding to the first noisy data generated by the second feature processing network.

[0184] The target feature information corresponding to the first noisy data at least includes feature information of the first noisy data. Optionally, the target feature information includes feature information of the first noisy data and feature information of the target constraint information.

[0185] In another implementation, if the second neural network and the second neural network are respectively deployed on the execution device, the execution device can input the first noisy data into the second neural network to obtain target feature information corresponding to the first noisy data generated by the second feature extraction network in the second neural network, and the target feature information at least includes feature information of the first noisy data.

[0186] Optionally, the execution device inputs the first noisy data and the target constraint information into the second neural network to obtain target feature information generated by the second feature extraction network in the second neural network, and the target feature information includes feature information of the first noisy data and feature information of the target constraint information.

[0187] The execution device processes the target feature information through the second feature processing network in the second neural network to obtain second prediction information corresponding to the first noisy data generated by the second neural network.

[0188] It should be noted that the specific meanings of various nouns in steps 1001 and 1002 can be referred to Figure 3 and Figure 7 the description in the corresponding various embodiments, which will not be described one by one here.

[0189] In the embodiments of the present application, based on the constraint information, the denoising operation can be continuously performed on the pure noise (i.e. not carrying the information in the clean data) to obtain the clean data conforming to the constraint information, that is, the first neural network and the second neural network can generate the clean data conforming to the constraint information from nothing, which expands the acquisition channel of the specific clean data.

[0190] 1003、The execution device generates distribution information corresponding to the target denoising operation according to the first prediction information and the second prediction information corresponding to the first noisy data.

[0191] In the embodiments of the present application, for any one of the N times of denoising operations performed by the execution device on the first noisy data (for convenience of description, hereinafter referred to as "target denoising operation"), the execution device can obtain the first prediction information and the second prediction information corresponding to the first noisy data, determine that the Hth time of denoising operation in the N times of denoising operation is being performed, and the distribution information corresponding to different times of denoising operation in the N times of denoising operation can be different; the training device generates distribution information corresponding to the target denoising operation according to the first prediction information and the second prediction information corresponding to the first noisy data.

[0192] In the training phase, the pure noise is obtained after the M-step noise adding process is performed on the clean data, and the corresponding clean data can be obtained by performing the M-step noise removing operation on the pure noise. The most ideal state is that the noise-removed data obtained by each noise removing operation is the complete inverse process of the corresponding noise adding operation. For example, if the value of M is 1000, the noise-removed data obtained by the first noise removing operation is consistent with the noise data obtained by the 999th noise adding operation in the ideal state. For another example, if the value of M is 1000, the noise-removed data obtained by the 100th noise removing operation is consistent with the noise data obtained by the 900th noise adding operation in the ideal state. It should be understood that the examples are only for the convenience of understanding the scheme and do not limit the scheme.

[0193] In order to improve the efficiency of the inference phase, only N times of noise removing operations are performed in the inference phase in the scheme, and the value of N is much smaller than s. Therefore, the purpose of each noise removing operation in the N times of noise removing operations is to obtain the noise data corresponding to the s-th noise adding operation in the M-step noise removing operation based on the noise data corresponding to the n-th noise adding operation in the M-step noise adding operation. The values of n and s are positive integers, the value of n is greater than the value of s, and the difference between n and s is greater than or equal to 2. That is, each noise removing operation in the N times of noise removing operations includes the noise removing effect of multiple noise removing operations in the M-step noise removing operation in the ideal state.

[0194] For example, if the value of M is 1000 and the value of N is 10, the purpose of the first noise removing operation in the 10 times of noise removing operations is to generate the noise data obtained after 900-step noise adding operation based on the noise data obtained after 1000-step noise adding operation. That is, if the value of n is 1000 and the value of s is 900 in the first noise removing operation.

[0195] The purpose of the second noise removing operation in the 10 times of noise removing operations is to generate the noise data obtained after 800-step noise adding operation based on the noise data obtained after 900-step noise adding operation. That is, if the value of n is 900 and the value of s is 800 in the second noise removing operation.

[0196] The purpose of the third noise removing operation in the 10 times of noise removing operations is to generate the noise data obtained after 700-step noise adding operation based on the noise data obtained after 800-step noise adding operation. That is, if the value of n is 800 and the value of s is 700 in the second noise removing operation. The remaining seven noise removing operations in the 10 times of noise removing operations can be deduced in the same way, and the effects of the remaining seven noise removing operations are not described herein.

[0197] Specifically, if the distribution space corresponding to the denoising operation satisfies a Gaussian distribution or a Gamma distribution, the execution device can generate a mean corresponding to the target denoising operation according to the first prediction information corresponding to the first noisy data, and generate a covariance corresponding to the target denoising operation according to the second prediction information corresponding to the first noisy data. The mean and the covariance corresponding to the target denoising operation can point to a specific distribution space.

[0198] More specifically, the execution device can determine the values of n and s corresponding to the Hth denoising operation (i.e., the target denoising operation) in the N denoising operations, and then generate a mean corresponding to the Hth denoising operation (i.e., the target denoising operation) in the N denoising operations according to the first prediction information corresponding to the first noisy data, the value of n, and the value of s. The value of H is greater than or equal to 1, and the value of H in step 1002 is 1.

[0199] Correspondingly, the execution device can generate a covariance corresponding to the Hth denoising operation (i.e., the target denoising operation) in the N denoising operations according to the second prediction information corresponding to the first noisy data, the value of n, and the value of s.

[0200] The second prediction information corresponding to the first noisy data indicates the square of the predicted noise between the first noisy data and the target clean data, or the second prediction information indicates the square of the predicted distance between the first prediction information and the target actual noise. To further understand the scheme, two examples of the formula for generating the covariance corresponding to the target denoising operation based on the second prediction information corresponding to the first noisy data are disclosed below.

[0201] If the second prediction information indicates the square of the predicted noise between the first noisy data and the target clean data, one example of the calculation formula of the covariance corresponding to the target denoising operation is disclosed as follows:

[0202]

[0203] wherein, represents the covariance corresponding to the target denoising operation, x n represents the noisy data input into the first neural network, which specifically can represent the first noisy data or the second noisy data, represents the variance of the total amount of noise added from the 0th denoising operation to the nth denoising operation in the noise adding process, η is a hyperparameter, the value of η is between 0 and 1, generally η is taken as 0 or 1, represents the variance of the total amount of noise added from the 0th denoising operation to the nth denoising operation in the noise adding process, n|s represents the variance of the total amount of noise added from the 0th denoising operation to the nth denoising operation in the noise adding process; hn (x n ) represents the square of the predicted noise between the noisy data and the clean data (i.e. one meaning of the second predicted information generated by the second feature processing network), represents the square of the first predicted information generated by the first feature processing network (i.e. the predicted noise between the noisy data and the clean data), it should be understood that the example in formula (1) is only used to prove the feasibility of the scheme, and is not used to limit the scheme.

[0204] If the second predicted information indicates the square of the predicted distance between the first predicted information and the target actual noise, another example of the calculation formula of the covariance corresponding to the target denoising operation is disclosed as follows:

[0205]

[0206] wherein, represents the covariance corresponding to the target denoising operation, x n 、 and The meanings of g, x and x can be referred to the above explanation of the parameters in formula (1), which will not be repeated here; g n (x n ) represents the square of the predicted distance between the first predicted information and the target actual noise (i.e. another meaning of the second predicted information generated by the second feature processing network), it should be understood that the example in formula (2) is only used to prove the feasibility of the scheme, and is not used to limit the scheme.

[0207] Further, according to the generation principle of formula (1), in an ideal state, any one-step denoising operation in the M-step denoising operation can realize the inverse process of the corresponding noise adding operation, i.e. the denoised data obtained by each denoising operation is completely consistent with the noise-added data at the corresponding position; therefore, the effect of each denoising operation in the N-step denoising operation can be optimized by the similarity between the first probability distribution and the second probability distribution. Wherein, any one denoising operation (i.e. the Hth denoising operation) in the N-step denoising operation is used to obtain the noisy data corresponding to the s-step noise adding operation according to the noisy data corresponding to the n-step noise adding operation, the first probability distribution refers to the probability distribution of the noisy data obtained after the n-step noise adding operation (i.e. the ideal state of the denoised data obtained after the M-n-step denoising operation) and the noisy data obtained after the s-step noise adding operation (i.e. the ideal state of the denoised data obtained after the M-s-step denoising operation), and the second probability distribution refers to the probability distribution of the denoised data obtained by executing the H-1th denoising operation through the first neural network and the second neural network and the denoised data obtained after executing the Hth denoising operation. The higher the similarity between the first probability distribution and the second probability distribution, the better the effect of the Hth denoising operation executed through the first neural network and the second neural network.

[0208] Optimizing the similarity between the first and second probability distributions is equivalent to optimizing the effect of the Hth denoising operation. A first formula is then obtained to express the objective of minimizing the similarity between the first and second probability distributions. Since each denoising operation performed by the first and second neural networks includes: generating the mean corresponding to the Hth denoising operation based on the first prediction information generated by the first neural network, and generating the covariance corresponding to the Hth denoising operation based on the second prediction information generated by the second neural network, the first formula contains a function term representing the covariance corresponding to the Hth denoising operation. After obtaining the first formula, it is transformed based on the moment matching principle to obtain the second formula; further, the second formula is transformed based on the covariance decomposition formula to obtain the final optimal covariance expression.

[0209] To further understand this scheme, we will take the similarity between the first and second probability distributions as specifically using the KL divergence between the first and second probability distributions, and express the optimal covariance by using the square of the prediction noise between the noisy data and the clean target data. An example of the optimal covariance expression can be as follows:

[0210]

[0211] in, This represents the covariance corresponding to the target denoising operation. and The meaning of can be found in the explanation of the parameters in equation (1) above, and will not be repeated here. Representative expects to use x n Able to predict the most accurate Represents x n The square of the actual noise between the noisy data (that is, the noisy data processed in the Hth denoising operation) and the clean data, when h n (x n When h is the optimal result (i.e., the output of the second feature processing network), n (x n )and Equivalent; represent The square of, Representative expects to use x n The most accurate ∈ n ,∈ n Represents x n The actual noise between clean data and clean data, when When (i.e., the output of the first feature processing network) is the optimal result and Equivalently, therefore, based on the continued derivation of equation (3), we can obtain equation (1).

[0212] To further understand this scheme, we will take the similarity between the first and second probability distributions, specifically using the KL divergence between them, and express the optimal covariance using the square of the predicted distance between the first predicted information and the actual target noise. An example of the optimal covariance expression would be as follows:

[0213]

[0214] in, This represents the covariance corresponding to the target denoising operation. and The meaning of can be found in the explanation of the parameters in equation (1) above, and will not be repeated here. Representative expects to utilize x n Able to predict the most accurate Represents x n The actual noise between the clean data and the output of the first feature extraction network and x n The square of the residual between the corresponding first predicted information, when g n (x n When (i.e., the result output by the second feature processing network) is optimal, g n (x n )and Equivalently, by continuing to derive equation (4), we can obtain equation (2). It should be understood that the examples of equations (3) and (4) here are only to prove the feasibility of equations (1) and (2) and are not intended to limit this scheme.

[0215] In this embodiment of the application, the implementation details of generating distribution information for each denoising operation based on the first prediction information and the second prediction information are further disclosed, which improves the feasibility of the solution.

[0216] 1004. The execution device samples the denoised data corresponding to the first noisy data from the distribution space pointed to by the distribution information corresponding to the target denoising operation.

[0217] In this embodiment of the application, after obtaining the distribution information corresponding to the target denoising operation, the execution device can randomly sample a data from the distribution space pointed to by the distribution information to obtain a denoised data corresponding to the first noisy data, that is, complete one denoising operation in N denoising operations and obtain the denoised data after performing one denoising operation in N denoising operations.

[0218] For a more intuitive understanding of this solution, please refer to [link / reference]. Figure 11 , Figure 11 This is a schematic diagram illustrating the denoising operation performed in the data denoising method provided in the embodiments of this application, such as... Figure 11 As shown, after performing N denoising operations on the first noisy data, the training device can obtain the target clean data corresponding to the first noisy data. Ideally, the H-th denoising operation in the N denoising operations is derived from the noisy data corresponding to the n-th denoising operation (i.e., ... Figure 11 x in n This yields the noisy data corresponding to the s-th step noise addition operation (i.e., ... Figure 11 x in s It should be understood that... Figure 11 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.

[0219] 1005. The execution device determines whether the number of times the denoising operation is performed on the first noisy data has reached N times. If it has not reached N times, proceed to step 1006. If it has reached N times, output the target clean data corresponding to the first noisy data obtained after performing the Nth denoising operation.

[0220] In this embodiment, after performing a denoising operation on the first noisy data, the execution device can determine whether the total number of denoising operations on the first noisy data has reached N times. If it has not reached N times, it proceeds to step 1006; if it has reached N times, it outputs the target clean data corresponding to the first noisy data obtained after performing the Nth denoising operation. Here, N is an integer greater than or equal to 1, and N is a hyperparameter; correspondingly, given a fixed number of M denoising operations, n and s corresponding to each denoising operation in the N denoising operations are also hyperparameters.

[0221] 1006. The execution device generates first prediction information corresponding to the second noisy data through a first feature processing network. The first prediction information corresponding to the second noisy data indicates the prediction noise between the second noisy data and the target clean data.

[0222] 1007. The execution device generates second prediction information corresponding to the second noisy data through the second feature processing network. The second prediction information corresponding to the second noisy data indicates the square of the predicted noise between the second noisy data and the target clean data, or the second prediction information indicates the square of the predicted distance between the first prediction information and the target actual noise.

[0223] 1008. The execution device generates distribution information corresponding to the target denoising operation based on the first prediction information and the second prediction information corresponding to the second noisy data.

[0224] 1009. The execution device samples the denoised data corresponding to the second noisy data from the distribution space pointed to by the distribution information corresponding to the target denoising operation.

[0225] In this embodiment, steps 1005 to 1009 are optional. If N is 1, after step 1004 is executed, the denoised data generated in step 1004 can be identified as the clean data corresponding to the first noisy data. If N is an integer greater than or equal to 2, steps 1005 to 1009 are executed, and after step 1009 is executed, step 1005 is re-entered. The specific implementation of steps 1005 to 1009 can be found in the above description of the specific implementation of steps 1001 to 1004. The difference is that the initial noisy data (i.e., the first noisy data) in steps 1001 to 1004 is replaced with the noisy data after at least one denoising operation in steps 1005 to 1009.

[0226] For a more intuitive understanding of this solution, please refer to [link / reference]. Figure 12 , Figure 12 This is a flowchart illustrating a data denoising method provided in the embodiments of this application, such as... Figure 12 As shown, after obtaining the first noisy data, the execution device needs to perform N denoising operations on the first noisy data; in the first denoising operation, the execution device inputs the first noisy data into the feature extraction network to obtain the feature information generated by the feature extraction network.

[0227] The execution device processes the aforementioned feature information through a first feature processing network to obtain first prediction information generated by the first feature processing network, and generates the mean value corresponding to the current denoising operation based on the first prediction information; the execution device processes the aforementioned feature information through a second feature processing network to obtain second prediction information generated by the second feature processing network, and generates the covariance corresponding to the current denoising operation based on the second prediction information.

[0228] The execution device determines the distribution space corresponding to the current denoising operation based on the mean and covariance, and performs a sampling operation on the aforementioned distribution space to obtain the denoised data corresponding to the first denoising operation, which is also the noisy data x used in the second denoising operation. n .

[0229] In the second denoising operation, the executing device will remove the noisy data x n The input is a feature extraction network, which generates feature information. Then, a second denoising operation is performed based on the aforementioned feature information to obtain the denoised data corresponding to the second denoising operation, which is also the noisy data used in the third denoising operation.

[0230] The execution device repeatedly performs the denoising operation N times to obtain clean data corresponding to the first noisy data. It should be understood that, Figure 12 The examples in the foregoing merely serve to facilitate understanding of the present scheme and do not serve to limit the present scheme.

[0231] In the embodiments of the present application, in the process of performing the denoising operation, the first feature processing network is used to generate the first prediction information, i.e., the predicted noise between the noisy data and the clean data; the second feature processing network is used to generate the second prediction information, which indicates the square of the predicted noise between the noisy data and the clean data, or indicates the square of the distance between the first prediction information and the actual noise; and then, based on the first prediction information and the second prediction information, the distribution information corresponding to the denoising operation is generated, and the denoised data corresponding to the noisy data is sampled from the distribution space pointed by the distribution information, i.e., one denoising operation in the M-step denoising operation is completed. Since the first feature processing network and the second feature processing network learn the data between the noisy data and the clean data, i.e., the first feature processing network and the second feature processing network do not directly learn the distribution information corresponding to the denoising operation, the number of times of performing the denoising operation on the first noisy data is not constrained by the training phase. When the number of times of the denoising operation needs to be adjusted, the first feature processing network and the second feature processing network do not need to be retrained. The denoising scheme provided in the embodiments of the present application avoids the computer resource consumption caused by repeated training of the neural network, and improves the flexibility of the inference phase.

[0232] Next, the beneficial effects brought by the embodiments of the present application are demonstrated in combination with experimental data. Compared with the original diffusion probability model for performing the denoising operation on the data, the method provided in the embodiments of the present application can greatly improve the speed of the inference process and reduce the consumption of the computer resources in the inference phase while ensuring the quality of the clean data. As shown in Figure 13 Figure 13 is a comparison diagram of the clean data generated by the diffusion probability model provided in the embodiments of the present application and the method provided in the embodiments of the present application. Figure 13 The upper sub-diagram of FIG. 8 is an experiment performed on the data set CIFAR10, Figure 13 The lower sub-diagram of FIG. 8 is an experiment performed on the data set ImageNet. As can be seen from the comparison, the quality of the clean data obtained based on the method provided in the embodiments of the present application is also not bad.

[0233] When the denoising operation is performed on the data in CIFAR10 to obtain clean data, the inference speed of the neural network provided in the embodiments of the present application is improved by 59 times compared with the original diffusion probability model, and the computer resource consumption is only 1.7% of the original.

[0234] ​When the data in the ImageNet is denoised to obtain clean data, the inference speed of the neural network provided in the embodiments of the present application is improved by 80 times relative to the original diffusion probability model, and the computing resource consumption is only 1.3% of the original.

[0235] In Figures 1a to 13 Based on the corresponding embodiments, in order to better implement the above-mentioned scheme of the embodiments of the present application, the related equipment for implementing the above-mentioned scheme is also provided below. For details, see Figure 14 , Figure 14 A structural schematic diagram of the data denoising device provided in the embodiments of the present application is shown in FIG. 14. The data denoising device 1400 is configured to perform at least one denoising operation on the first noisy data to obtain clean data corresponding to the first noisy data. The data denoising device 1400 includes: an acquisition module 1401 configured to acquire first prediction information, wherein the first prediction information indicates the predicted noise between the second noisy data and the clean data, the first prediction information is generated by a first feature processing network, and the second noisy data is the first noisy data or the second noisy data is the noisy data after at least one denoising operation is performed on the first noisy data; the acquisition module 1401 is further configured to acquire second prediction information, wherein the second prediction information indicates the square of the predicted noise between the second noisy data and the clean data, or the second prediction information indicates the square of the predicted distance between the first prediction information and the actual noise, the actual noise includes the actual noise between the second noisy data and the clean data, and the second prediction information is generated by a second feature processing network, the feature information processed by the first feature processing network and the second feature processing network is obtained based on the second noisy data; a generation module 1402 configured to generate distribution information corresponding to a target denoising operation according to the first prediction information and the second prediction information, the target denoising operation is any one of the at least one denoising operation; and a sampling module 1403 configured to sample the denoised data corresponding to the second noisy data from the distribution space pointed by the distribution information.

[0236] In a possible design, the first feature processing network and the second feature processing network are configured to process the same feature information.

[0237] In a possible design, if the first noisy data is pure noise, the at least one denoising operation is N times of denoising operation, N is an integer greater than or equal to 11, and N is a hyperparameter, the feature information processed by the first feature processing network includes the feature information of the second noisy data and the feature information of the constraint information, and the clean data satisfies the constraint of the constraint information.

[0238] In a possible design, the generating module 1402 includes: a first generating submodule, configured to generate a mean corresponding to the target denoising operation according to the first prediction information; and a second generating submodule, configured to generate a covariance corresponding to the target denoising operation according to the second prediction information.

[0239] In a possible design, the distribution space to which the distribution information points complies with a Gaussian distribution or a Gamma distribution.

[0240] It should be noted that the information interaction and execution process between the modules / units in the data denoising apparatus 1400 are similar to those of the data denoising apparatus 1300, which will not be described herein again. Figures 10 to 12 The corresponding method embodiments are based on the same concept, and specific content can be referred to the description of the method embodiments described above, which will not be described herein again.

[0241] Please refer to Figure 15 , Figure 15 FIG. 1 shows a structural schematic diagram of a neural network training apparatus provided by an embodiment of the present application. The neural network training apparatus 1500 includes: a feature extraction module 1501, configured to input third noisy data into a feature extraction network to obtain first feature information generated by the feature extraction network, the first feature information at least including feature information of the third noisy data; a feature processing module 1502, configured to process the first feature information through a second feature processing network to obtain second prediction information output by the second feature processing network, the second prediction information indicating a square of a predicted noise between the third noisy data and first clean data, the third noisy data being obtained by performing multiple noise adding operations on the first clean data; and a training module 1503, configured to train the second feature processing network according to first expected information corresponding to the third noisy data, the second prediction information, and a first loss function, until a convergence condition is met, where the first loss function indicates a similarity between the second prediction information and the first expected information, and the first expected information indicates a square of an actual noise between the third noisy data and the first clean data.

[0242] In a possible design, the training module 1503 is specifically configured to keep parameters of the feature extraction network unchanged, and update parameters of the second feature processing network, where the feature extraction network is derived from a trained first neural network, and the first neural network is used to generate a predicted noise between fourth noisy data and second clean data based on input of the fourth noisy data, the second clean data being obtained by performing multiple noise adding operations on the fourth noisy data.

[0243] In a possible design, the neural network training apparatus 1500 is further configured to obtain the trained first neural network, and the first neural network is used to generate first prediction information based on input of pure noise and constraint information, the first prediction information indicating a predicted noise between the pure noise and clean data satisfying the constraint information.

[0244] It should be noted that the information interaction, execution process and the like between the modules / units in the neural network training apparatus 1500 are consistent with the descriptions of the neural network training apparatus in the present application. Figures 3 to 6 The corresponding method embodiments are based on the same concept, and specific content can be referred to the descriptions of the method embodiments in the foregoing of the present application, which will not be described here.

[0245] Please refer to Figure 16 , Figure 16 A structural schematic diagram of the neural network training apparatus provided by the embodiments of the present application is provided, and the neural network training apparatus 1600 includes: a feature extraction module 1601 configured to input third noisy data into a feature extraction network to obtain first feature information generated by the feature extraction network, the first feature information at least including feature information of the third noisy data; a feature processing module 1602 configured to process the first feature information through a second feature processing network to obtain second prediction information output by the second feature processing network, the third noisy data being obtained after performing multiple noise adding operations on the first clean data, the second prediction information indicating a square of a predicted distance between the first prediction information and actual noise, the first prediction information including predicted noise between the third noisy data and the first clean data generated by a first neural network, the first neural network being configured to generate the first prediction information based on the input third noisy data, and the actual noise including actual noise between the third noisy data and the first clean data;

[0246] a training module 1603 configured to train the second feature processing network according to second expected information corresponding to the third noisy data, the second prediction information, and a second loss function until a convergence condition is met, wherein the second loss function indicates a similarity between the second prediction information and the second expected information, and the second expected information indicates a square of an actual distance between the first prediction information and the actual noise.

[0247] In a possible design, the training module 1603 is specifically configured to keep parameters of the feature extraction network unchanged and update parameters of the second feature processing network, wherein the feature extraction network is derived from the trained first neural network.

[0248] It should be noted that the information interaction, execution process and the like between the modules / units in the neural network training apparatus 1600 are consistent with the descriptions of the neural network training apparatus in the present application. Figures 7 to 9 The corresponding method embodiments are based on the same concept, and specific content can be referred to the descriptions of the method embodiments in the foregoing of the present application, which will not be described here.

[0249] Next, a kind of execution equipment provided by the embodiments of the present application is introduced, please refer to Figure 17 , Figure 17A structural schematic diagram of an execution device provided by an embodiment of the present application is shown in FIG. 17. Specifically, the execution device 1700 includes a receiver 1701, a transmitter 1702, a processor 1703, and a memory 1704 (wherein the number of processors 1703 in the execution device 1700 can be one or more, and the processor 1703 in the embodiment is exemplified by one processor), Figure 17 The processor 1703 can include an application processor 17031 and a communication processor 17032. In some embodiments of the present application, the receiver 1701, the transmitter 1702, the processor 1703, and the memory 1704 can be connected by a bus or other means.

[0250] The memory 1704 can include a read-only memory and a random access memory, and provide the processor 1703 with instructions and data. A part of the memory 1704 can also include a non-volatile random access memory (NVRAM). The memory 1704 stores processor and operation instructions, executable modules or data structures, or a subset thereof, or an expanded set thereof, wherein the operation instructions can include various operation instructions for implementing various operations.

[0251] The processor 1703 controls the operation of the execution device. In a specific application, various components of the execution device are coupled together through a bus system, which can include a data bus, a power supply bus, a control bus, and a state signal bus, etc. in addition to a data bus. However, for the sake of clarity, all kinds of buses are referred to as a bus system in the figure.

[0252] The method disclosed in the embodiments of the present application can be applied to the processor 1703 or implemented by the processor 1703. The processor 1703 can be an integrated circuit chip with a signal processing capability. In the implementation process, the steps of the method disclosed above can be completed by an integrated logic circuit or a software form of an instruction in the processor 1703. The processor 1703 disclosed above can be a general processor, a digital signal processor (DSP), a microprocessor or a microcontroller. The processor 1703 can further include an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 1703 can implement or execute the methods, steps and logical block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, or other mature storage medium in the art. The storage medium is located in the storage 1704, and the processor 1703 reads the information in the storage 1704 and combines the hardware to complete the steps of the method.

[0253] The receiver 1701 can be used to receive input digital or character information, and generate signal input related to the relevant settings and function control of the execution device. The transmitter 1702 can be used to output digital or character information through the first interface; the transmitter 1702 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1702 can also include a display device such as a display screen.

[0254] In the embodiments of the present application, the application processor 17031 in the processor 1703 is used to execute Figures 10 to 12 the data denoising method executed by the execution device in the corresponding embodiment. It should be noted that the specific manner in which the application processor 17031 executes each of the steps above is the same as that in the corresponding method embodiments of the present application Figures 10 to 12 The corresponding method embodiments are based on the same concept, and the technical effects brought by them are the same as those in the corresponding method embodiments of the present application Figures 10 to 12 The corresponding method embodiments are based on the same concept, and the technical effects brought by them are the same as those in the corresponding method embodiments of the present application

[0255] The embodiments of the present application also provide a training device, please refer toFigure 18 , Figure 18 is a structural schematic diagram of a training device provided by an embodiment of the present application. Specifically, the training device 1800 is implemented by one or more servers. The training device 1800 can have a large difference due to different configurations or performances, and can include one or more central processing units (CPUs) 1822 (for example, one or more processors) and a memory 1832, one or more storage media 1830 (for example, one or more mass storage devices) storing application programs 1842 or data 1844. The memory 1832 and the storage media 1830 can be temporary storage or persistent storage. The programs stored in the storage media 1830 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the training device. Further, the central processing unit 1822 can be configured to communicate with the storage media 1830 and execute the series of instruction operations in the storage media 1830 on the training device 1800.

[0256] The training device 1800 can also include one or more power supplies 1826, one or more wired or wireless network interfaces 1850, one or more input and output interfaces 1858, and / or one or more operating systems 1841, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and the like.

[0257] In an embodiment of the present application, the central processing unit 1822 is configured to execute Figures 3 to 6 the data denoising method performed by the training device in the corresponding embodiment. It should be noted that the specific manner in which the central processing unit 1822 executes the above steps is the same as that of the corresponding method embodiments in the present application. Figures 3 to 6 The corresponding method embodiments are based on the same concept, and the technical effects brought by them are the same as those of the corresponding method embodiments in the present application. Figures 3 to 6 The specific content can be referred to the description of the method embodiments in the foregoing embodiments of the present application, and will not be described here.

[0258] In an embodiment of the present application, the central processing unit 1822 is configured to execute Figures 7 to 9 the data denoising method performed by the training device in the corresponding embodiment. It should be noted that the specific manner in which the central processing unit 1822 executes the above steps is the same as that of the corresponding method embodiments in the present application. Figures 7 to 9 The corresponding method embodiments are based on the same concept, and the technical effects brought by them are the same as those of the corresponding method embodiments in the present application. Figures 7 to 9 The specific content can be referred to the description of the method embodiments in the foregoing embodiments of the present application, and will not be described here.

[0259] The embodiment of the present application also provides a computer program product comprising a program, which, when executed on a computer, causes the computer to perform the steps executed by the execution device in the method described in the foregoing Figures 10 to 12 The embodiment of the present application also provides a computer program product comprising a program, which, when executed on a computer, causes the computer to perform the steps executed by the training device in the method described in the foregoing Figures 3 to 6 The embodiment of the present application also provides a computer program product comprising a program, which, when executed on a computer, causes the computer to perform the steps executed by the training device in the method described in the foregoing Figures 7 to 9 The embodiment of the present application also provides a computer program product comprising a program, which, when executed on a computer, causes the computer to perform the steps executed by the training device in the method described in the foregoing

[0260] The embodiment of the present application also provides a computer program product comprising a program, which, when executed on a computer, causes the computer to perform the steps executed by the execution device in the method described in the foregoing Figures 10 to 12 The embodiment of the present application also provides a computer program product comprising a program, which, when executed on a computer, causes the computer to perform the steps executed by the training device in the method described in the foregoing Figures 3 to 6 The embodiment of the present application also provides a computer program product comprising a program, which, when executed on a computer, causes the computer to perform the steps executed by the training device in the method described in the foregoing Figures 7 to 9 The embodiment of the present application also provides a computer program product comprising a program, which, when executed on a computer, causes the computer to perform the steps executed by the training device in the method described in the foregoing

[0261] The execution device, the training device, the data denoising apparatus or the neural network training apparatus provided by the embodiment of the present application can be a chip, which comprises a processing unit and a communication unit. The processing unit can be a processor, and the communication unit can be an input / output interface, a pin or a circuit, etc. The processing unit can execute computer execution instructions stored in a storage unit, so as to make the chip execute the data denoising method described in the foregoing Figures 10 to 12 The execution device, the training device, the data denoising apparatus or the neural network training apparatus provided by the embodiment of the present application can be a chip, which comprises a processing unit and a communication unit. The processing unit can be a processor, and the communication unit can be an input / output interface, a pin or a circuit, etc. The processing unit can execute computer execution instructions stored in a storage unit, so as to make the chip execute the data denoising method described in the foregoing Figures 3 to 6 The execution device, the training device, the data denoising apparatus or the neural network training apparatus provided by the embodiment of the present application can be a chip, which comprises a processing unit and a communication unit. The processing unit can be a processor, and the communication unit can be an input / output interface, a pin or a circuit, etc. The processing unit can execute computer execution instructions stored in a storage unit, so as to make the chip execute the data denoising method described in the foregoing Figures 7 to 9 The execution device, the training device, the data denoising apparatus or the neural network training apparatus provided by the embodiment of the present application can be a chip, which comprises a processing unit and a communication unit. The processing unit can be a processor, and the communication unit can be an input / output interface, a pin or a circuit, etc. The processing unit can execute computer execution instructions stored in a storage unit, so as to make the chip execute the data denoising method described in the foregoing

[0262] Specifically, refer to Figure 19 , Figure 19A structure diagram of a chip provided by an embodiment of the present application can be a neural network processor NPU 190, which is mounted on a host CPU as a coprocessor and is assigned tasks by the host CPU. The core part of the NPU 190 is an operation circuit 1903, which extracts matrix data in a memory and performs multiplication operation under the control of a controller 1904.

[0263] In some implementations, the operation circuit 1903 internally includes a plurality of processing units (PEs). In some implementations, the operation circuit 1903 is a two-dimensional systolic array. The operation circuit 1903 can also be a one-dimensional systolic array or other electronic circuit capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 1903 is a general-purpose matrix processor.

[0264] For example, it is assumed that there are an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes corresponding data of the matrix B from the weight memory 1902 and buffers the data on each PE of the operation circuit. The operation circuit takes the matrix A data from the input memory 1901 and performs matrix operation with the matrix B, and the partial result or final result of the obtained matrix is saved in an accumulator 1908.

[0265] The unified memory 1906 is used to store input data and output data. The weight data is transferred to the weight memory 1902 through a direct memory access controller (DMAC) 1905. The input data is also transferred to the unified memory 1906 through the DMAC.

[0266] The BIU is a bus interface unit 1910 for interaction between the AXI bus and the DMAC and an instruction fetch buffer (IFB) 1909.

[0267] The bus interface unit 1910 (BIU) is used for the instruction fetch buffer 1909 to obtain instructions from an external memory, and is also used for the direct memory access controller 1905 to obtain original data of the input matrix A or the weight matrix B from the external memory.

[0268] The DMAC is mainly used to transfer input data in an external memory DDR to the unified memory 1906, or to transfer weight data to the weight memory 1902, or to transfer input data to the input memory 1901.

[0269] The vector computation unit 1907 includes a plurality of operation processing units that further process the output of the operation circuit, if necessary, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, and the like. It is mainly used for non-convolution / full connection layer network computation in a neural network, such as Batch Normalization, pixel-level summation, up-sampling of a feature plane, and the like.

[0270] In some implementations, the vector computation unit 1907 can store the processed output vector to the unified memory 1906. For example, the vector computation unit 1907 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1903, such as linear interpolation on a feature plane extracted by a convolution layer, and further such as vector accumulation of values to generate activation values. In some implementations, the vector computation unit 1907 generates normalized values, pixel-level summation values, or both. In some implementations, the processed output vector can be used as activation input to the operation circuit 1903, such as for use in a subsequent layer in a neural network.

[0271] The controller 1904 is connected to an instruction fetch buffer 1909 for storing instructions used by the controller 1904;

[0272] The unified memory 1906, the input memory 1901, the weight memory 1902, and the instruction fetch buffer 1909 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0273] In the above method embodiments, the operations of each layer in the neural network can be performed by the operation circuit 1903 or the vector computation unit 1907.

[0274] In the above method embodiments, the processor can be a general central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the first aspect.

[0275] It should be noted that the apparatus embodiments described above are merely exemplary, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. In addition, the connection relationship between the modules in the apparatus embodiment provided in the present application indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0276] Those skilled in the art can clearly understand that the application can be implemented by means of software plus necessary universal hardware, and of course can also be implemented by means of dedicated hardware including special integrated circuit, special CPU, special memory, special component, etc. Generally, any function completed by computer program can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuit, digital circuit or special circuit, etc. However, for the application, software program implementation is a better embodiment. Based on such understanding, the technical solution of the application or the part of the application which makes contribution to the prior art can be embodied in the form of software product, which is stored in readable storage medium, such as computer floppy disk, U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a plurality of instructions for making a computer device (which can be personal computer, training device or network device, etc.) execute the method described in various embodiments of the application.

[0277] In the above embodiments, the implementation can be achieved by software, hardware, firmware or any combination thereof, entirely or partially. When implemented by software, the implementation can be achieved in the form of a computer program product, entirely or partially.

[0278] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function described in the embodiments of the application is generated entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be stored by a computer or a data storage device such as a training device, a data center, etc. integrated with one or more available media sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

Claims

1. A method for denoising data, characterized in that, The method is used to perform at least one denoising operation on the first noisy data to obtain clean data corresponding to the first noisy data, wherein any one of the at least one denoising operation includes: Obtain first prediction information, wherein the first prediction information indicates the prediction noise between the second noisy data and the clean data, the first prediction information is generated by a first feature processing network, the second noisy data is the first noisy data, or the second noisy data is noisy data after performing at least one denoising operation on the first noisy data; A second prediction information is obtained, wherein the second prediction information indicates the square of the predicted noise between the second noisy data and the clean data, or the second prediction information indicates the square of the predicted distance between the first prediction information and the actual noise, wherein the actual noise includes the actual noise between the second noisy data and the clean data, and the second prediction information is generated by a second feature processing network, wherein the feature information processed by the first feature processing network and the second feature processing network is obtained based on the second noisy data; Based on the first prediction information and the second prediction information, generate distribution information corresponding to the target denoising operation; The denoised data corresponding to the second noisy data is obtained by sampling from the distribution space pointed to by the distribution information.

2. The method according to claim 1, characterized in that, The first feature processing network and the second feature processing network are used to process the same feature information.

3. The method according to claim 1 or 2, characterized in that, The at least one denoising operation is N denoising operations, where N is an integer greater than or equal to 1. If the first noisy data is pure noise, then N is a hyperparameter. The feature information processed by the first feature processing network includes the feature information of the second noisy data and the feature information of the constraint information. The clean data satisfies the constraints of the constraint information.

4. The method according to claim 1 or 2, characterized in that, The step of generating distribution information corresponding to the target denoising operation based on the first prediction information and the second prediction information includes: Based on the first prediction information, generate the mean value corresponding to the target denoising operation; Based on the second prediction information, a covariance corresponding to the target denoising operation is generated.

5. The method according to claim 1 or 2, characterized in that, The distribution space indicated by the distribution information follows a Gaussian or gamma distribution.

6. A method for training a neural network, characterized in that, The method includes: The third noisy data is input into the feature extraction network to obtain the first feature information generated by the feature extraction network. The first feature information includes at least the feature information of the third noisy data. The first feature information is processed by the second feature processing network to obtain the second prediction information output by the second feature processing network. The second prediction information indicates the square of the prediction noise between the third noisy data and the first clean data. The third noisy data is obtained by performing multiple noise addition operations on the first clean data. The second feature processing network is trained based on the first expected information, the second predicted information and the first loss function corresponding to the third noisy data until the convergence condition is met. The first loss function indicates the similarity between the second predicted information and the first expected information, and the first expected information indicates the square of the actual noise between the third noisy data and the first clean data. The second prediction information generated by the second feature processing network during the inference phase is used to obtain the distribution information corresponding to the denoising operation, and the distribution information corresponding to the denoising operation is used to obtain the denoised data corresponding to the noisy data.

7. The method according to claim 6, characterized in that, Training the second feature processing network includes: Keeping the parameters of the feature extraction network unchanged, the parameters of the second feature processing network are updated; The feature extraction network is derived from a trained first neural network. The first neural network is used to generate predictive noise between the fourth noisy data and the second clean data based on the input fourth noisy data. The second clean data is obtained by performing multiple noise-adding operations on the fourth noisy data.

8. The method according to claim 6 or 7, characterized in that, The method is further used to obtain a trained first neural network, which generates first prediction information based on input pure noise and constraint information, the first prediction information indicating the predicted noise between the pure noise and clean data that satisfies the constraint information.

9. A method for training a neural network, characterized in that, The method includes: The third noisy data is input into the feature extraction network to obtain the first feature information generated by the feature extraction network. The first feature information includes at least the feature information of the third noisy data. The first feature information is processed by the second feature processing network to obtain the second prediction information output by the second feature processing network. The third noisy data is obtained by performing multiple noise addition operations on the first clean data. The second prediction information indicates the square of the prediction distance between the first prediction information and the actual noise. The first prediction information includes the prediction noise between the third noisy data and the first clean data generated by the first neural network. The first neural network is used to generate the first prediction information based on the input third noisy data. The actual noise includes the actual noise between the third noisy data and the first clean data. The second feature processing network is trained based on the second expected information, the second predicted information, and the second loss function corresponding to the third noisy data until the convergence condition is met. The second loss function indicates the similarity between the second predicted information and the second expected information, and the second expected information indicates the square of the actual distance between the first predicted information and the actual noise. The second prediction information generated by the second feature processing network during the inference phase is used to obtain the distribution information corresponding to the denoising operation, and the distribution information corresponding to the denoising operation is used to obtain the denoised data corresponding to the noisy data.

10. The method according to claim 9, characterized in that, Training the second feature processing network includes: Keeping the parameters of the feature extraction network unchanged, the parameters of the second feature processing network are updated, wherein the feature extraction network is derived from the first neural network after training.

11. A data denoising device, characterized in that, The data denoising device is used to perform at least one denoising operation on the first noisy data to obtain clean data corresponding to the first noisy data. The denoising device includes: An acquisition module is used to acquire first prediction information, wherein the first prediction information indicates the prediction noise between the second noisy data and the clean data, the first prediction information is generated by a first feature processing network, the second noisy data is the first noisy data, or the second noisy data is noisy data after at least one denoising operation has been performed on the first noisy data; The acquisition module is further configured to acquire second prediction information, wherein the second prediction information indicates the square of the predicted noise between the second noisy data and the clean data, or the second prediction information indicates the square of the predicted distance between the first prediction information and the actual noise, wherein the actual noise includes the actual noise between the second noisy data and the clean data, and the second prediction information is generated by a second feature processing network, wherein the feature information processed by the first feature processing network and the second feature processing network is obtained based on the second noisy data; The generation module is used to generate distribution information corresponding to the target denoising operation based on the first prediction information and the second prediction information, wherein the target denoising operation is any one of the at least one denoising operation; The sampling module is used to sample the denoised data corresponding to the second noisy data from the distribution space pointed to by the distribution information.

12. The apparatus according to claim 11, characterized in that, The first feature processing network and the second feature processing network are used to process the same feature information.

13. The apparatus according to claim 11 or 12, characterized in that, If the first noisy data is pure noise, then the at least one denoising operation is N denoising operations, where N is an integer greater than or equal to 11, and N is a hyperparameter. The feature information processed by the first feature processing network includes the feature information of the second noisy data and the feature information of the constraint information. The clean data satisfies the constraints of the constraint information.

14. The apparatus according to claim 11 or 12, characterized in that, The generation module includes: The first generation submodule is used to generate the mean value corresponding to the target denoising operation based on the first prediction information; The second generation submodule is used to generate the covariance corresponding to the target denoising operation based on the second prediction information.

15. The apparatus according to claim 11 or 12, characterized in that, The distribution space indicated by the distribution information follows a Gaussian or gamma distribution.

16. A training device for a neural network, characterized in that, The device includes: The feature extraction module is used to input the third noisy data into the feature extraction network to obtain the first feature information generated by the feature extraction network, wherein the first feature information includes at least the feature information of the third noisy data. The feature processing module is used to process the first feature information through a second feature processing network to obtain second prediction information output by the second feature processing network. The second prediction information indicates the square of the prediction noise between the third noisy data and the first clean data. The third noisy data is obtained by performing multiple noise addition operations on the first clean data. The training module is used to train the second feature processing network based on the first expected information, the second predicted information and the first loss function corresponding to the third noisy data until the convergence condition is met. The first loss function indicates the similarity between the second predicted information and the first expected information, and the first expected information indicates the square of the actual noise between the third noisy data and the first clean data. The second prediction information generated by the second feature processing network during the inference phase is used to obtain the distribution information corresponding to the denoising operation, and the distribution information corresponding to the denoising operation is used to obtain the denoised data corresponding to the noisy data.

17. The apparatus according to claim 16, characterized in that, The training module is specifically used to keep the parameters of the feature extraction network unchanged and update the parameters of the second feature processing network. The feature extraction network is derived from a trained first neural network. The first neural network is used to generate predictive noise between the fourth noisy data and the second clean data based on the input fourth noisy data. The second clean data is obtained by performing multiple noise-adding operations on the fourth noisy data.

18. The apparatus according to claim 16 or 17, characterized in that, The apparatus is further configured to acquire a trained first neural network, which generates first prediction information based on input pure noise and constraint information, the first prediction information indicating the predicted noise between the pure noise and clean data that satisfies the constraint information.

19. A training device for a neural network, characterized in that, The device includes: The feature extraction module is used to input the third noisy data into the feature extraction network to obtain the first feature information generated by the feature extraction network, wherein the first feature information includes at least the feature information of the third noisy data. The feature processing module is used to process the first feature information through a second feature processing network to obtain second prediction information output by the second feature processing network. The third noisy data is obtained by performing multiple noise addition operations on the first clean data. The second prediction information indicates the square of the prediction distance between the first prediction information and the actual noise. The first prediction information includes the prediction noise between the third noisy data and the first clean data generated by the first neural network. The first neural network is used to generate the first prediction information based on the input third noisy data. The actual noise includes the actual noise between the third noisy data and the first clean data. The training module is used to train the second feature processing network based on the second expected information, the second predicted information and the second loss function corresponding to the third noisy data until the convergence condition is met. The second loss function indicates the similarity between the second predicted information and the second expected information, and the second expected information indicates the square of the actual distance between the first predicted information and the actual noise. The second prediction information generated by the second feature processing network during the inference phase is used to obtain the distribution information corresponding to the denoising operation, and the distribution information corresponding to the denoising operation is used to obtain the denoised data corresponding to the noisy data.

20. The apparatus according to claim 19, characterized in that, The training module is specifically used to keep the parameters of the feature extraction network unchanged and update the parameters of the second feature processing network, wherein the feature extraction network is derived from the first neural network after training.

21. A computer program product, characterized in that, The computer program product includes a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 10.

23. An execution device, characterized in that, It includes a processor and a memory, wherein the processor is coupled to the memory. The memory is used to store programs; The processor is configured to execute a program in the memory, causing the execution device to perform the method as described in any one of claims 1 to 5.

24. A training device, characterized in that, It includes a processor and a memory, wherein the processor is coupled to the memory. The memory is used to store programs; The processor is configured to execute a program in the memory, causing the training device to perform the method as described in any one of claims 6 to 8, or to cause the training device to perform the method as described in claim 9 or 10.

Images