Training methods, devices, storage media, and electronic devices for image reconstruction models

Abstract

This invention provides a training method, apparatus, storage medium, and electronic device for an image reconstruction model. The method includes: acquiring a target teacher image reconstruction model, which is a model trained using a target sample image set; training a student image reconstruction model to be trained using the target sample image set until a target loss value meets a preset loss condition, then ending the training to obtain a target student image reconstruction model. The target loss value is determined based on a first loss value and a second loss value, whereby the second loss value is determined based on the difference between the predicted values ​​and the true values ​​determined by the student image reconstruction model to be trained and the target teacher image reconstruction model, respectively. This invention solves the problem of high computational load in image reconstruction models in related technologies, achieving the effect of reducing the computational load of network models.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a training method, apparatus, storage medium, and electronic device for an image reconstruction model. Background Technology

[0002] In recent years, super-resolution reconstruction technology has become an important research topic in the fields of image processing and computer vision. It has wide applications in satellite imagery, military reconnaissance, medical diagnosis, and remote sensing. Super-resolution reconstruction converts low-resolution images into high-resolution images with better visual effects. One related technology applies deep learning methods to image reconstruction, using neural networks to learn the mapping relationship between low-resolution and high-resolution images. This method aims to establish the mapping relationship between low-resolution and high-resolution images by learning from a large number of data samples. However, as the network model becomes more complex, more network parameters need to be trained, increasing the training difficulty. In other words, the methods in this technology suffer from problems such as high computational cost and high training difficulty.

[0003] There is currently no effective solution to the problem of high computational cost in image reconstruction models in related technologies. Summary of the Invention

[0004] This invention provides a training method, apparatus, storage medium, and electronic device for an image reconstruction model, to at least solve the problem of high computational load in image reconstruction models in related technologies.

[0005] According to one aspect of the present invention, a training method for an image reconstruction model is provided, comprising: acquiring a target teacher image reconstruction model, a target sample image set, and a set of real reconstructed images corresponding to the target sample image set, wherein the target teacher image reconstruction model is a model obtained by training a teacher image reconstruction model to be trained using the target sample image set, and the loss value between the predicted reconstructed image set determined by the target teacher image reconstruction model based on the target sample image set and the real reconstructed image set satisfies a first preset loss condition; training a student image reconstruction model to be trained using the target sample image set until the target loss value corresponding to the student image reconstruction model to be trained satisfies a second preset loss condition, thereby ending the training and obtaining a target student image reconstruction model, wherein the target loss value is a loss value determined based on a first loss value and a second loss value, wherein the first... The loss value is the loss between the predicted reconstructed image determined by the student image reconstruction model to be trained based on the sample images in the target sample image set and the corresponding real reconstructed image in the real reconstructed image set. The second loss value is the loss value determined based on the first difference and the second difference. The first difference is the difference determined based on the difference of each pixel between the first predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The first predicted reconstructed image is the predicted reconstructed image determined by the student image reconstruction model to be trained based on the sample images in the target sample image set. The second difference is the difference determined based on the difference of each pixel between the second predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The second predicted reconstructed image is the predicted reconstructed image determined by the target teacher image reconstruction model based on the sample images in the target sample image set.

[0006] In an exemplary embodiment, training a student image reconstruction model to be trained using the target sample image set until the target loss value corresponding to the student image reconstruction model to be trained satisfies a second preset loss condition includes: repeatedly performing the following steps until the target loss value corresponding to the student image reconstruction model to be trained satisfies the second preset loss condition: obtaining a current sample image in the target sample image set and a current real reconstructed image corresponding to the current sample image in the real reconstructed image set; inputting the current sample image into the student image reconstruction model to be trained to obtain a first current predicted reconstructed image; determining a first current loss value based on the first current predicted reconstructed image and the current real reconstructed image, wherein the first current loss value represents the sum of the first current predicted reconstructed image and the current real reconstructed image. The training process involves several steps: First, the loss between images is calculated. The current sample image is input into the target teacher image reconstruction model to obtain a second currently predicted reconstructed image. Then, based on the differences between pixels in the first currently predicted reconstructed image and the current real reconstructed image, and the differences between pixels in the second currently predicted reconstructed image and the current real reconstructed image, a second current loss value is determined for the current sample image. Based on the first current loss value and the second current loss value, a target loss value is determined for the current sample image. If the target loss value for the current sample image satisfies the second preset loss condition, training ends, and the target student image reconstruction model is obtained. If the target loss value for the current sample image does not satisfy the second preset loss condition, the parameters in the student image reconstruction model to be trained are adjusted.

[0007] In an exemplary embodiment, determining the second current loss value corresponding to the current sample image based on the differences between each pixel point between the first currently predicted reconstructed image and the current true reconstructed image, and the differences between each pixel point between the second currently predicted reconstructed image and the current true reconstructed image, includes: obtaining the differences between each pixel point between the first currently predicted reconstructed image and the current true reconstructed image; determining a first mean and a first variance value based on the differences between each pixel point between the first currently predicted reconstructed image and the current true reconstructed image; obtaining the differences between each pixel point between the second currently predicted reconstructed image and the current true reconstructed image; determining a second mean and a second variance value based on the differences between each pixel point between the second currently predicted reconstructed image and the current true reconstructed image; and determining the second current loss value based on the first mean, the first variance value, the second mean, and the second variance value.

[0008] In an exemplary embodiment, determining the first mean and the first variance based on the differences between corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image includes: determining the first mean as equal to the mean of the differences between corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image, and determining the first variance as equal to the variance of the differences between corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image; or adjusting the differences between corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image according to preset adjustment parameters to obtain a first set of adjusted differences, wherein the first set of adjusted differences corresponds to each pixel; determining the first mean as equal to the mean of the first set of adjusted differences, and determining the first variance as equal to the variance of the first set of adjusted differences.

[0009] In an exemplary embodiment, adjusting the differences in pixel values ​​of corresponding pixels between the first currently predicted reconstructed image and the current real reconstructed image according to preset adjustment parameters to obtain a first set of adjusted differences includes: obtaining the first set of adjusted differences according to the following formula: x′ i =Tx i +(1-T)μ, 1≤i≤C*H*W, where C, H, and W are the number of channels, height, and width of the first currently predicted reconstructed image, respectively, and T is the adjustment parameter, x i x is the difference between the pixel value of the i-th pixel point corresponding to the first currently predicted reconstructed image and the current true reconstructed image. i ' represents the i-th difference in the first set of adjusted differences, and μ represents the mean of the differences between the pixel values ​​of each corresponding pixel point between the first current predicted reconstructed image and the current true reconstructed image.

[0010] In an exemplary embodiment, determining the second mean and the second variance based on the differences between corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image includes: determining the second mean as equal to the mean of the differences between corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image, and determining the second variance as equal to the variance of the differences between corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image; or adjusting the differences between corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image according to preset adjustment parameters to obtain a second set of adjusted differences, wherein the second set of adjusted differences corresponds to each pixel; determining the second mean as equal to the mean of the second set of adjusted differences, and determining the second variance as equal to the variance of the second set of adjusted differences.

[0011] In an exemplary embodiment, adjusting the differences in pixel values ​​of corresponding pixels between the second currently predicted reconstructed image and the current real reconstructed image according to preset adjustment parameters to obtain two sets of adjusted differences includes: obtaining the second set of adjusted differences according to the following formula: x′ j =Tx j +(1-T)μ, 1≤j≤C*H*W, where C, H, and W are the number of channels, height, and width of the second currently predicted reconstructed image, respectively, and T is the adjustment parameter, x j x is the difference between the pixel value of the j-th pixel point corresponding to the second currently predicted reconstructed image and the current true reconstructed image. j ' is the j-th difference in the second set of adjusted differences, and μ is the mean of the differences between the pixel values ​​of each corresponding pixel point between the second current predicted reconstructed image and the current real reconstructed image.

[0012] In an exemplary embodiment, determining the second current loss value based on the first mean, the first variance, the second mean, and the second variance includes: determining the second current loss value according to the following formula: Among them, 2b S 2 =var(x) i (1≤i≤C*H*W),2b T 2 =var(x) j (1≤j≤C*H*W), where C, H, and W are the number of channels, height, and width of the first currently predicted reconstructed image and the second currently predicted reconstructed image, respectively, and x ix is the difference between the pixel value of the i-th pixel point corresponding to the first currently predicted reconstructed image and the current true reconstructed image. j μ is the difference between the pixel value of the j-th pixel point corresponding to the second currently predicted reconstructed image and the current true reconstructed image. S For the first mean, var(x) i ) represents the first variance value, μ T The second mean, var(x) j ) is the second variance value, and Loss2 is the second current loss value.

[0013] In an exemplary embodiment, determining the target loss value corresponding to the current sample image based on the first current loss value and the second current loss value includes: determining the target loss value corresponding to the current sample image as equal to the sum of the first current loss value and the second current loss value.

[0014] According to another aspect of the present invention, a training apparatus for an image reconstruction model is also provided, comprising: an acquisition module, configured to acquire a target teacher image reconstruction model, a target sample image set, and a set of real reconstructed images corresponding to the target sample image set, wherein the target teacher image reconstruction model is a model obtained by training a teacher image reconstruction model to be trained using the target sample image set, and the loss value between the predicted reconstructed image set determined by the target teacher image reconstruction model based on the target sample image set and the real reconstructed image set satisfies a first preset loss condition; and a training module, configured to train a student image reconstruction model to be trained using the target sample image set until the target loss value corresponding to the student image reconstruction model to be trained satisfies a second preset loss condition, thereby ending the training and obtaining a target student image reconstruction model, wherein the target loss value is determined based on the first loss value and the second loss value. The loss values ​​are as follows: the first loss value is the loss between the predicted reconstructed image determined by the student image reconstruction model to be trained based on the sample images in the target sample image set and the corresponding real reconstructed image in the real reconstructed image set; the second loss value is the loss value determined based on the first difference and the second difference. The first difference is the difference determined based on the difference of each pixel between the first predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The first predicted reconstructed image is the predicted reconstructed image determined by the student image reconstruction model to be trained based on the sample images in the target sample image set. The second difference is the difference determined based on the difference of each pixel between the second predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The second predicted reconstructed image is the predicted reconstructed image determined by the target teacher image reconstruction model based on the sample images in the target sample image set.

[0015] According to another aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0016] According to another aspect of the present invention, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor performs the steps of any of the above method embodiments through the computer program.

[0017] This invention trains a student image reconstruction model by using the output of a target teacher image reconstruction model trained with a set of target sample images and corresponding sets of real reconstructed images as supervision. The difference between the predicted and real reconstructed images from the target teacher model is used to apply a loss constraint to the difference between the predicted and real reconstructed images from the student model, resulting in a target student image reconstruction model whose target loss value satisfies a second preset loss condition. This allows the student model to learn the knowledge of the target teacher model more effectively, achieving a level of accuracy close to that of the target teacher model while significantly reducing computational load. This technical solution addresses the problem of high computational cost in image reconstruction models in related technologies, effectively reducing the computational burden on the network model. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with the description thereof, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0019] Figure 1 This is a block diagram of the terminal hardware structure of the training method for the image reconstruction model according to an embodiment of the present invention.

[0020] Figure 2 This is a flowchart of a training method for an image reconstruction model according to an embodiment of the present invention;

[0021] Figure 3 This is a schematic diagram of teacher network training according to a specific embodiment of the present invention;

[0022] Figure 4 This is a schematic diagram of student network training according to a specific embodiment of the present invention;

[0023] Figure 5 This is a structural block diagram of a training device for an image reconstruction model according to an embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] The methods and embodiments provided in this application can be executed on a terminal or server, such as a mobile terminal, computer terminal, server, or similar computing device. Taking running on a terminal as an example... Figure 1 This is a block diagram of the terminal hardware structure of the image reconstruction model training method according to an embodiment of the present invention. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. In one exemplary embodiment, the terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0027] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the image reconstruction model training method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0028] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0029] In the current field of computer vision technology, image upscaling methods based on convolutional neural networks (CNNs) have demonstrated superior performance compared to traditional methods. However, in practical applications, the high number of network parameters and computational complexity often make network models difficult to deploy. Therefore, it is usually necessary to lightweight the network model. Lightweighting methods typically include pruning, quantization, and knowledge distillation. Knowledge distillation, analogous to the "distillation process" in physics, usually involves two network models: a "teacher" model with a large number of parameters and high complexity, and a "student" model with a small number of parameters and low complexity. In CNN-based knowledge distillation algorithms, most algorithms are typically used to solve classification problems, usually using the logits output of the "teacher" network, the outputs of intermediate layers, or the relationship between the outputs of intermediate layers as supervision to train the "student" network. However, it has been almost entirely neglected for regression problems (such as image reconstruction or super-resolution upscaling), another major category of tasks in computer vision. This invention provides an improved training method for image reconstruction models based on knowledge distillation algorithms.

[0030] This embodiment provides a training method for an image reconstruction model. Figure 2 This is a flowchart of a training method for an image reconstruction model according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0031] Step S202: Obtain the target teacher image reconstruction model, the target sample image set, and the real reconstructed image set corresponding to the target sample image set. The target teacher image reconstruction model is a model obtained by training the teacher image reconstruction model to be trained using the target sample image set. The loss value between the predicted reconstructed image set determined by the target teacher image reconstruction model based on the target sample image set and the real reconstructed image set satisfies the first preset loss condition.

[0032] Step S204: Train the student image reconstruction model to be trained using the target sample image set until the target loss value corresponding to the student image reconstruction model to be trained meets the second preset loss condition, then end the training and obtain the target student image reconstruction model. The target loss value is a loss value determined based on a first loss value and a second loss value. The first loss value is the loss value between the predicted reconstructed image determined by the student image reconstruction model to be trained based on the sample images in the target sample image set and the corresponding real reconstructed image in the real reconstructed image set. The second loss value is a loss value determined based on a first difference and a second difference. The first difference is the difference between each pixel point of the first predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The first predicted reconstructed image is the predicted reconstructed image determined by the student image reconstruction model to be trained based on the sample images in the target sample image set. The second difference is the difference between each pixel point of the second predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The second predicted reconstructed image is the predicted reconstructed image determined by the target teacher image reconstruction model based on the sample images in the target sample image set.

[0033] Through the above steps, the output of the target teacher image reconstruction model trained using the target sample image set and the corresponding real reconstructed image set is used as supervision to train the student image reconstruction model to be trained. The difference between the predicted reconstructed image result of the target teacher image reconstruction model and the real reconstructed image is used to apply a loss constraint to the difference between the predicted image reconstruction result of the student image reconstruction model and the corresponding real reconstructed image, resulting in a target student image reconstruction model whose target loss value satisfies the second preset loss condition. This achieves the goal of enabling the target student image reconstruction model to learn the knowledge of the target teacher image reconstruction model more effectively, thereby achieving a level of accuracy close to that of the target teacher image reconstruction model while significantly reducing the computational load. This technical solution solves the problem of high computational load in image reconstruction models in related technologies, achieving the effect of reducing the computational load of the network model.

[0034] The entity executing the above steps can be a terminal, or an image processing device, such as a network model training device, a model training platform, or a model trainer. The entity executing these steps can also be other processing devices or units with similar processing capabilities, but is not limited to these. The following explanation uses an image processing device performing the above operations as an example (this is merely an illustrative example; in actual operation, other devices or modules can also perform the above operations).

[0035] In the above embodiments, the image processing device acquires a pre-trained target teacher image reconstruction model (or target teacher network), a target sample image set, and a set of real reconstructed images corresponding to the target sample image set. The target teacher image reconstruction model is a model obtained by training the teacher image reconstruction model to be trained using the target sample image set. The loss value between the predicted reconstructed image set and the real reconstructed image set determined by the target teacher image reconstruction model based on the target sample image set satisfies a first preset loss condition. The first preset loss condition is that the loss value of the target teacher image reconstruction model reaches a convergence condition, for example, a loss value of 0, 0.1, 0.5, or other values. Then, the target sample image set is used to train the student image reconstruction model to be trained (or student network to be trained) until the target loss value corresponding to the student image reconstruction model to be trained satisfies a second preset loss condition, at which point training ends, resulting in a target student image reconstruction model (or target student network). The second preset loss condition is that the loss value of the target student image reconstruction model reaches a convergence condition, for example, a target loss value of 0, 0.2, 0.6, or other values. If the target loss value fails to meet the second preset condition, the student image reconstruction model to be trained continues... Training is performed; since the target loss value is determined based on the first loss value and the second loss value, and the second loss value is determined based on the first difference between the pixel differences between the reconstructed image predicted by the student image reconstruction model and the corresponding real reconstructed image, and the second difference between the pixel differences between the reconstructed image predicted by the target teacher image reconstruction model and the corresponding real reconstructed image, the output of the target teacher image reconstruction model is used as supervision to train the student image reconstruction model. The difference between the predicted reconstructed image result of the target teacher image reconstruction model and the corresponding real reconstructed image is used to impose loss constraints on the difference between the predicted image reconstruction result of the student image reconstruction model and the real reconstructed image, so as to obtain a target student image reconstruction model whose target loss value meets the second preset loss condition. This can improve the accuracy of the student image reconstruction model to be trained. After training, the target student image reconstruction model can be deployed to the corresponding scene, so as to enable the target student image reconstruction model to learn the knowledge of the target teacher image reconstruction model more effectively. This achieves the effect of making the accuracy of the target student image reconstruction model close to that of the target teacher image reconstruction model, while significantly reducing the amount of computation. By adopting the above technical solution, the problem of lacking a method for lightweighting network models in related technologies is solved, thereby reducing the computational load of network models.

[0036] In an optional embodiment, training the student image reconstruction model to be trained using the target sample image set until the target loss value corresponding to the student image reconstruction model to be trained satisfies a second preset loss condition includes: repeatedly performing the following steps until the target loss value corresponding to the student image reconstruction model to be trained satisfies the second preset loss condition: obtaining the current sample image in the target sample image set and the current real reconstructed image corresponding to the current sample image in the real reconstructed image set; inputting the current sample image into the student image reconstruction model to be trained to obtain a first current predicted reconstructed image; determining a first current loss value based on the first current predicted reconstructed image and the current real reconstructed image, wherein the first current loss value represents the sum of the first current predicted reconstructed image and the current real reconstructed image. The training process involves several steps: First, the loss between images is calculated. The current sample image is input into the target teacher image reconstruction model to obtain a second currently predicted reconstructed image. Then, based on the differences between pixels in the first currently predicted reconstructed image and the current real reconstructed image, and the differences between pixels in the second currently predicted reconstructed image and the current real reconstructed image, a second current loss value is determined for the current sample image. Based on the first current loss value and the second current loss value, a target loss value is determined for the current sample image. If the target loss value for the current sample image satisfies the second preset loss condition, training ends, and the target student image reconstruction model is obtained. If the target loss value for the current sample image does not satisfy the second preset loss condition, the parameters in the student image reconstruction model to be trained are adjusted. In this embodiment, the second current loss value is determined based on the differences between each pixel of the reconstructed image (i.e., the first currently predicted reconstructed image) predicted by the student image reconstruction model to be trained on the current sample image and the current real reconstructed image (corresponding to the aforementioned first difference), and the differences between each pixel of the reconstructed image (i.e., the second currently predicted reconstructed image) predicted by the target teacher image reconstruction model on the current sample image and the current real reconstructed image (corresponding to the aforementioned second difference). That is, the difference obtained by the target teacher image reconstruction model is used to constrain the difference of the student image reconstruction model to be trained, so as to enable the target student image reconstruction model to learn the knowledge of the target teacher image reconstruction model more effectively. Based on the first current loss value and the second current loss value, the target loss value of the current sample image can be determined. Training ends when the target loss value meets the second preset condition, and the target student image reconstruction model that meets the requirements can be obtained. The second preset loss condition is that the loss value of the target student image reconstruction model reaches the convergence condition, such as the target loss value reaching 0, 0.2, 0.6 or other values.This embodiment achieves the goal of using the target teacher image reconstruction model to guide the student image reconstruction model under training during training, thereby improving the accuracy of the student image reconstruction model under training and obtaining a target student image reconstruction model that meets the requirements.

[0037] In an optional embodiment, determining the second current loss value corresponding to the current sample image based on the differences between each pixel point between the first currently predicted reconstructed image and the current true reconstructed image, and the differences between each pixel point between the second currently predicted reconstructed image and the current true reconstructed image, includes: obtaining the differences between each pixel point between the first currently predicted reconstructed image and the current true reconstructed image; determining a first mean and a first variance based on the differences between each pixel point between the first currently predicted reconstructed image and the current true reconstructed image; obtaining the differences between each pixel point between the second currently predicted reconstructed image and the current true reconstructed image; determining a second mean and a second variance based on the differences between each pixel point between the second currently predicted reconstructed image and the current true reconstructed image; and determining the second current loss value based on the first mean, the first variance, the second mean, and the second variance. In this embodiment, the differences between each pixel in the first currently predicted reconstructed image and the current real reconstructed image follow a specific distribution pattern. Similarly, the differences between each pixel in the second currently predicted reconstructed image and the current real reconstructed image also follow a specific distribution pattern. In practical applications, when using L1 loss as the image reconstruction loss for training, the differences between each pixel can be assumed a priori to follow a Laplacian distribution. When using L2 loss as the image reconstruction loss for training, the differences between each pixel can be assumed a priori to follow a Gaussian distribution. The following example uses L1 loss as the image reconstruction loss for training. Let x... i Let x be the difference of the i-th pixel among the differences between the first currently predicted reconstructed image and the current real reconstructed image, where 1 ≤ i ≤ C*H*W (C, H, and W are the number of channels, height, and width of the first currently predicted reconstructed image, respectively). i It follows a Laplace distribution with a first mean of x. i The mean, for example, μ1 = mean(x i ), where μ1 is the first mean and the first variance is D1 = var(x i Then, μ1 is used as the position parameter in the Laplace distribution function, and according to 2b1 2 =var(x) i By determining b1 as the scale parameter in the Laplace distribution function, x can be determined. iThe Laplace distribution function of the distribution; similarly, assume x j Let x be the difference of the j-th pixel among the differences between the pixels in the second currently predicted reconstructed image and the current real reconstructed image, where 1 ≤ j ≤ C*H*W (C, H, and W are the number of channels, height, and width of the second currently predicted reconstructed image, respectively). j It follows a Laplace distribution, and the second mean is x. j The mean, for example, μ2 = mean(x j ), where μ2 is the second mean and the second variance is D2 = var(x j Then, μ2 is used as the position parameter in the Laplace distribution function, and according to 2b2 2 =var(x) j By determining b2 as the scale parameter in the Laplace distribution function, x can be determined. j The Laplace distribution function of the distribution is used, and then the second current loss value is determined based on the first mean, the first variance, the second mean, and the second variance. For example, relative entropy (or KL divergence) can be used to measure x. i and x j The difference between the two distributions, i.e., determining the second current loss value, can also be measured in practical applications using geometric distance (or mean squared error), Jensen-Shannon divergence, or Wasserstein distance. i and x j The difference between the two distributions (i.e., the second current loss value). Through this embodiment, the purpose of determining the second current loss value is achieved, that is, the purpose of measuring the distance between the difference distribution of each pixel between the first currently predicted reconstructed image and the current true reconstructed image and the difference distribution of each pixel between the second currently predicted reconstructed image and the current true reconstructed image is achieved.

[0038] In an optional embodiment, determining the first mean and the first variance based on the differences between corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image includes: determining the first mean as equal to the mean of the differences between corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image, and determining the first variance as equal to the variance of the differences between corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image; or adjusting the differences between corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image according to preset adjustment parameters to obtain a first set of adjusted differences, wherein the first set of adjusted differences corresponds to each pixel; determining the first mean as equal to the mean of the first set of adjusted differences, and determining the first variance as equal to the variance of the first set of adjusted differences. In this embodiment, the mean of the differences between the pixel values ​​of corresponding pixels between the first currently predicted reconstructed image and the current real reconstructed image can be determined as the first mean, and the variance of the differences between the pixel values ​​of corresponding pixels between the first currently predicted reconstructed image and the current real reconstructed image can be determined as the first variance. Alternatively, a preset adjustment parameter can be introduced to adjust the differences between the pixel values ​​of corresponding pixels between the first currently predicted reconstructed image and the current real reconstructed image to obtain a first set of adjusted differences, and then the corresponding first mean and first variance can be determined based on the first set of adjusted differences. Through this embodiment, the purpose of determining the first mean and first variance based on the differences between the pixel values ​​of corresponding pixels between the first currently predicted reconstructed image and the current real reconstructed image can be achieved, or the purpose of adjusting the differences between the pixel values ​​of corresponding pixels between the first currently predicted reconstructed image and the current real reconstructed image can be achieved based on preset adjustment parameters.

[0039] In an optional embodiment, adjusting the differences in pixel values ​​of corresponding pixels between the first currently predicted reconstructed image and the current realistic reconstructed image according to preset adjustment parameters to obtain a first set of adjusted differences includes: obtaining the first set of adjusted differences according to the following formula: x′ i =Tx i +(1-T)μ, 1≤i≤C*H*W, where C, H, and W are the number of channels, height, and width of the first currently predicted reconstructed image, respectively, and T is the adjustment parameter, x i x is the difference between the pixel value of the i-th pixel point corresponding to the first currently predicted reconstructed image and the current true reconstructed image. i' represents the i-th difference in the first set of adjusted differences, and μ is the mean of the differences in pixel values ​​between corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image. In this embodiment, analogous to the traditional knowledge distillation method, a learnable temperature parameter T is introduced, and a linear change is used as a mapping function to obtain the aforementioned first set of adjusted differences, for example, according to formula x′. i =Tx i The first set of adjusted differences is obtained by adding (1-T)μ, where T is the preset adjustment parameter mentioned above. In practical applications, a polynomial function, a learnable neural network, or metric learning can also be used as the mapping function to obtain the first set of adjusted differences. In practical applications, depending on the training needs of different network models, T can be 0 to 1, such as 0.2, 0.5, or other values, or T can be greater than or equal to 1, such as 1, 1.2, or other values. The parameter T can be adjusted during training until the requirements are met. Through this embodiment, the purpose of adjusting the difference distribution of each pixel between the first currently predicted reconstructed image and the current real reconstructed image by introducing the temperature parameter T is achieved.

[0040] In an optional embodiment, determining the second mean and the second variance based on the differences between corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image includes: determining the second mean as equal to the mean of the differences between corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image, and determining the second variance as equal to the variance of the differences between corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image; or adjusting the differences between corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image according to preset adjustment parameters to obtain a second set of adjusted differences, wherein the second set of adjusted differences corresponds to each pixel; determining the second mean as equal to the mean of the second set of adjusted differences, and determining the second variance as equal to the variance of the second set of adjusted differences. In this embodiment, the mean of the differences between the pixel values ​​of corresponding pixels between the second currently predicted reconstructed image and the current real reconstructed image can be determined as the second mean, and the variance of the differences between the pixel values ​​of corresponding pixels between the second currently predicted reconstructed image and the current real reconstructed image can be determined as the second variance. Alternatively, a preset adjustment parameter can be introduced to adjust the differences between the pixel values ​​of corresponding pixels between the second currently predicted reconstructed image and the current real reconstructed image to obtain a second set of adjusted differences, and then the corresponding second mean and second variance can be determined based on the second set of adjusted differences. Through this embodiment, the purpose of determining the second mean and second variance based on the differences between the pixel values ​​of corresponding pixels between the second currently predicted reconstructed image and the current real reconstructed image can be achieved, or the purpose of adjusting the differences between the pixel values ​​of corresponding pixels between the second currently predicted reconstructed image and the current real reconstructed image can be achieved based on preset adjustment parameters.

[0041] In an optional embodiment, the step of adjusting the pixel values ​​of corresponding pixels between the second currently predicted reconstructed image and the current true reconstructed image according to preset adjustment parameters to obtain two sets of adjusted differences includes: obtaining the second set of adjusted differences according to the following formula: x′ j =Tx j (1-T)μ, 1≤j≤C*H*W, where C, H, and W are the number of channels, height, and width of the second currently predicted reconstructed image, respectively, and T is the adjustment parameter, x j x is the difference between the pixel value of the j-th pixel point corresponding to the second currently predicted reconstructed image and the current true reconstructed image. j' is the j-th difference in the second set of adjusted differences, and μ is the mean of the differences in pixel values ​​between corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image. In this embodiment, analogous to the traditional knowledge distillation method, a learnable temperature parameter T is introduced, and a linear change is used as a mapping function to obtain the above-mentioned second set of adjusted differences, for example, according to formula x′ i =Tx i (1-T)μ yields the second set of adjusted differences, where T is the preset adjustment parameter mentioned above. In practical applications, a polynomial function, a learnable neural network, or metric learning can also be used as the mapping function to obtain the second set of adjusted differences. In practical applications, depending on the training needs of different network models, T can be between 0 and 1, such as 0.2, 0.5, or other values. Alternatively, T can be greater than or equal to 1, such as 1, 1.2, or other values. The parameter T can be adjusted during training until the requirements are met. Through this embodiment, the purpose of adjusting the difference distribution of each pixel between the second currently predicted reconstructed image and the current real reconstructed image by introducing the temperature parameter T is achieved.

[0042] In an optional embodiment, determining the second current loss value based on the first mean, the first variance, the second mean, and the second variance includes: determining the second current loss value according to the following formula: Among them, 2b S 2 =var(x) i (1≤i≤C*H*W),2b T 2 =var(x) j (1≤j≤C*H*W), where C, H, and W are the number of channels, height, and width of the first currently predicted reconstructed image and the second currently predicted reconstructed image, respectively, and x i x is the difference between the pixel value of the i-th pixel point corresponding to the first currently predicted reconstructed image and the current true reconstructed image. j μ is the difference between the pixel value of the j-th pixel point corresponding to the second currently predicted reconstructed image and the current true reconstructed image. S For the first mean, var(x) i ) represents the first variance value, μ T The second mean, var(x) jLet ) be the second variance value, and Loss2 be the second current loss value. In this embodiment, the second current loss value Loss2 can be determined using relative entropy (or KL divergence). In practical applications, geometric distance (or mean squared error), Jensen-Shannon divergence, or Wasserstein distance can also be used to determine the second current loss value Loss2. Through this embodiment, the purpose of measuring the distance between the difference distribution of each pixel determined by the student image reconstruction model to be trained and the difference distribution of each pixel determined by the target teacher image reconstruction model is achieved.

[0043] In an optional embodiment, determining the target loss value corresponding to the current sample image based on the first current loss value and the second current loss value includes: determining the target loss value corresponding to the current sample image as equal to the sum of the first current loss value and the second current loss value. In this embodiment, the target loss value of the current sample image includes the first current loss value and the second current loss value, and the second current loss value is a loss constraint applied to the differences between pixels determined by the target teacher image reconstruction model and the differences between pixels determined by the student image reconstruction model to be trained, which can improve the accuracy of the student image reconstruction model to be trained.

[0044] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments. The present invention will now be described in detail with reference to specific embodiments:

[0045] In related technologies, for classification problems with n categories, neural networks typically predict a one-dimensional vector of length n for a given image, and the k-th value of this vector represents the confidence score predicted by the network for the k-th category. Knowledge distillation theory posits that the distribution of confidence scores for the "teacher" network, besides the category with the highest confidence score, is also beneficial to the training of the "student" network and can improve its generalization ability; this confidence distribution is called "dark knowledge." Furthermore, during knowledge distillation training, a "temperature" parameter T is introduced into both the "teacher" and "student" networks to smooth out the predictions, allowing the "student" network to pay more attention to the "dark knowledge" of the "teacher" network during training. However, this approach cannot be directly applied to regression problems. For example, in super-resolution magnification networks, the prediction results are pixel values ​​of the image, without classification logits, making it impossible to incorporate existing knowledge distillation methods. If the output of the "teacher" network is directly used to supervise the "student" network, the poor predictions of the "teacher" network during the training process may also affect the training of the "student" network, causing a decrease in the accuracy of the "student" network.

[0046] This invention provides a novel knowledge distillation method applicable to super-resolution algorithms. This method calculates the difference Δ between the outputs and ground truth values ​​of the "teacher" network (corresponding to the aforementioned target teacher image reconstruction model) and the "student" network (corresponding to the aforementioned target student image reconstruction model, or the student image reconstruction model to be trained). T (corresponding to the aforementioned second difference) and Δ S (Corresponding to the aforementioned first difference), since super-resolution models typically use a specific image reconstruction loss L restruction Therefore, Δ can be T and Δ S The prior is called a certain distribution, such as using L1 loss as the image reconstruction loss L. restruction For training, a Laplace distribution can be used as a prior; L2 loss is used as the image reconstruction loss L. restruction Training can be performed using a priori distribution such as Gaussian distribution.

[0047] In this distribution, we believe that the distribution in the non-zero value region (i.e., the part where the "teacher" network's prediction deviates from the true value) still contains a certain amount of "knowledge" (analogous to "tacit knowledge" in classical knowledge distillation), and is an approximate solution given by the "teacher" network, which also has learning value. Therefore, we use Δ T To Δ S Loss constraints are applied, and a distribution distance metric function L is introduced. distanceThis is used to measure the distribution of both. To more effectively enable the "student" network to learn the "tacit knowledge" of the "teacher" network, a "temperature" parameter T and a mapping function U(Δ, T) are introduced, analogous to the classic knowledge distillation method. This mapping function maps the distribution Δ according to the magnitude of T; the larger T is, the smoother the transformed Δ, thus "flattening" the distribution Δ. In this way, the "student" network can use L... distance When calculating the loss, we can pay more attention to the non-zero value region of the "teacher" distribution (i.e., the "hidden knowledge" of the "teacher" network), so that the "student" network can learn and utilize the "knowledge" brought by the "teacher" network more fully. At the same time, the "temperature" parameter T is learnable, so the network can adaptively adjust this parameter during the training process, reducing the impact of noise in the "teacher" network on the "student" network.

[0048] It should be noted that in the above description, the image reconstruction loss L... restruction Including but not limited to L1 loss, L2 loss, etc.; distance metric function L distance This includes, but is not limited to, geometric distance (mean squared error), relative entropy (KL divergence), JS (Jensen-Shannon) divergence, or Wasserstein distance, etc.; the mapping function U(Δ, T) includes, but is not limited to, linear functions, polynomial functions, learnable neural networks, and metric learning, etc.

[0049] The following uses L1 loss as the image reconstruction loss of the super-resolution model. distance Let's take an example to illustrate:

[0050] Step one: Train the "teacher" network. Figure 3 This is a schematic diagram of teacher network training according to a specific embodiment of the present invention. The "teacher" network is trained using pre-prepared super-resolution training data, and L1 loss is used to calculate the loss between the network's prediction results and the actual data, in the following form:

[0051]

[0052] Where C, H, and W represent the number of channels, height, and width of the image, respectively. X is the image predicted by the network, and Y is the real image. After training with the above loss until convergence (corresponding to the satisfaction of the first preset loss condition), the corresponding "teacher" network is obtained.

[0053] Step two, train the "student" network. Figure 4 This is a schematic diagram of student network training according to a specific embodiment of the present invention. The "student" network is trained using the same super-resolution training data, and its loss mainly consists of two parts.

[0054] Lossstudent (X, Y) = L restruction (X, Y) + L distance (Δ S Δ T (2)

[0055] Among them, L1 loss L restruction (X, Y) (corresponding to the aforementioned first loss value, or the aforementioned first current loss value) has the same form as formula (1), and the second term L distance (Δ S Δ T ) represents the distillation loss (corresponding to the aforementioned second loss value, or the aforementioned second current loss value), where Δ S The distribution of the difference between the images predicted by the "student" network and the actual data, Δ T The distribution of the difference between the image predicted by the "teacher" network and the actual data is given. Since both the "teacher" and "student" networks use L1 loss calculation, we assume Δ... S Δ T The prior distribution is a Laplace distribution.

[0056]

[0057] in,

[0058] μ=mean(x) (4)

[0059] 2b 2 =var(x), b>0 (5)

[0060] Then there is,

[0061]

[0062]

[0063] In formula (6), x represents the distribution of differences between pixels in the predicted reconstructed image and the actual reconstructed image from the student network (corresponding to the aforementioned x). i Distribution, x j Distribution), in formula (4) μ corresponds to the first mean and the second mean mentioned above, and in formula (5) var(x) corresponds to the first variance and the second variance mentioned above;

[0064] At the same time, KL divergence can be used as L. distance To measure Δ S Δ T The difference between the two distributions can then be used to derive Δ. X Δ Y The KL divergence of the two Laplace distributions is:

[0065]

[0066] And in Δ T In the distribution, we believe that there is still learnable "knowledge" in the distribution of non-zero values, which is equivalent to the approximate solution given by the "teacher" network. These approximate solutions are beneficial to the training of the "student" network.

[0067] Optionally, to allow the "student" network to better focus on these approximate solutions, a learnable "temperature" parameter T (corresponding to the aforementioned preset adjustment parameter) is introduced, analogous to traditional knowledge distillation methods. Simultaneously, a linear transformation is used as the mapping function U(Δ,T) to adjust the distribution Δ. S Δ T Its form is as follows

[0068]

[0069] in,

[0070] x′=(x, T)=Tx+(1-T)μ (10)

[0071] Through the above transformation, the distribution x can be made "flattened" while keeping its mean unchanged. This allows for adaptive adjustment of the distribution Δ by training and adjusting the "temperature" parameter T. x Δ Y This enables the "student" network to better learn useful knowledge from the "teacher" network.

[0072] Finally, after the "student" network is trained to convergence (corresponding to the aforementioned satisfaction of the second preset loss condition), the trained "student" network (corresponding to the aforementioned target student image reconstruction model) can be obtained.

[0073] Once training is complete, the "student" network can be deployed to the corresponding scenario, significantly reducing the computational load while maintaining accuracy close to that of the "teacher" network.

[0074] It should be noted that this embodiment of the invention proposes a new distillation loss for knowledge distillation of super-resolution models, therefore, no specific limitations are made on the specific structure of the "teacher" network and the "student" network, as well as the network training parameters.

[0075] Through the embodiments of this invention, during the training process, the "teacher" model guides the "student" model, enabling the "student" model to achieve better results. This embodiment proposes a distillation loss method applicable to super-resolution distillation tasks. This loss method is simple and easy to use, allowing the trained "student" network to achieve accuracy comparable to the more complex "teacher" network while significantly reducing computational load. This solves the problems of high computational cost and difficulty in deployment of network models in related technologies.

[0076] This embodiment also provides a training device for an image reconstruction model. Figure 5 This is a structural block diagram of a training device for an image reconstruction model according to an embodiment of the present invention, such as... Figure 5 As shown, the device includes:

[0077] The acquisition module 502 is used to acquire a target teacher image reconstruction model, a target sample image set, and a set of real reconstructed images corresponding to the target sample image set. The target teacher image reconstruction model is a model obtained by training a teacher image reconstruction model to be trained using the target sample image set. The loss value between the predicted reconstructed image set determined by the target teacher image reconstruction model based on the target sample image set and the real reconstructed image set satisfies a first preset loss condition.

[0078] Training module 504 is used to train the student image reconstruction model to be trained using the target sample image set until the target loss value corresponding to the student image reconstruction model to be trained meets the second preset loss condition, and then the training ends to obtain the target student image reconstruction model. The target loss value is a loss value determined based on a first loss value and a second loss value. The first loss value is the loss value between the predicted reconstructed image determined by the student image reconstruction model to be trained based on the sample images in the target sample image set and the corresponding real reconstructed image in the real reconstructed image set. The second loss value is a loss value determined based on a first difference and a second difference. The first difference is the difference between each pixel point of the first predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The first predicted reconstructed image is the predicted reconstructed image determined by the student image reconstruction model to be trained based on the sample images in the target sample image set. The second difference is the difference between each pixel point of the second predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The second predicted reconstructed image is the predicted reconstructed image determined by the target teacher image reconstruction model based on the sample images in the target sample image set.

[0079] In an optional embodiment, the training module 504 includes: an execution unit, configured to repeatedly execute the following steps until the target loss value corresponding to the student image reconstruction model to be trained satisfies a second preset loss condition: obtaining a current sample image in the target sample image set and a current real reconstructed image corresponding to the current sample image in the real reconstructed image set; inputting the current sample image into the student image reconstruction model to be trained to obtain a first current predicted reconstructed image; determining a first current loss value based on the first current predicted reconstructed image and the current real reconstructed image, wherein the first current loss value represents the loss between the first current predicted reconstructed image and the current real reconstructed image; inputting the current sample image into the target teacher... The image reconstruction model obtains a second currently predicted reconstructed image; based on the differences between each pixel in the first currently predicted reconstructed image and the current real reconstructed image, and the differences between each pixel in the second currently predicted reconstructed image and the current real reconstructed image, the second current loss value corresponding to the current sample image is determined; based on the first current loss value and the second current loss value, the target loss value corresponding to the current sample image is determined; if the target loss value corresponding to the current sample image satisfies the second preset loss condition, training ends, and the target student image reconstruction model is obtained; if the target loss value corresponding to the current sample image does not satisfy the second preset loss condition, the parameters in the student image reconstruction model to be trained are adjusted.

[0080] In an optional embodiment, the execution unit may determine the second current loss value corresponding to the current sample image as follows: obtain the difference between each pixel point corresponding to the first currently predicted reconstructed image and the current real reconstructed image; determine a first mean and a first variance value based on the difference between each pixel point corresponding to the first currently predicted reconstructed image and the current real reconstructed image; obtain the difference between each pixel point corresponding to the second currently predicted reconstructed image and the current real reconstructed image; determine a second mean and a second variance value based on the difference between each pixel point corresponding to the second currently predicted reconstructed image and the current real reconstructed image; and determine the second current loss value based on the first mean, the first variance value, the second mean, and the second variance value.

[0081] In an optional embodiment, the execution unit may determine the first mean and the first variance as follows: the first mean is determined to be equal to the mean of the differences in pixel values ​​between corresponding pixels in the first currently predicted reconstructed image and the current real reconstructed image; the first variance is determined to be equal to the variance of the differences in pixel values ​​between corresponding pixels in the first currently predicted reconstructed image and the current real reconstructed image; or, according to preset adjustment parameters, the differences in pixel values ​​between corresponding pixels in the first currently predicted reconstructed image and the current real reconstructed image are adjusted to obtain a first set of adjusted differences, wherein the first set of adjusted differences corresponds to each pixel; the first mean is determined to be equal to the mean of the first set of adjusted differences, and the first variance is determined to be equal to the variance of the first set of adjusted differences.

[0082] In an optional embodiment, the execution unit may obtain the first set of adjusted differences as follows: the first set of adjusted differences is obtained according to the following formula: x′ i =Tx i +(1-T)μ, 1≤i≤C*H*W, where C, H, and W are the number of channels, height, and width of the first currently predicted reconstructed image, respectively, and T is the adjustment parameter, x i x is the difference between the pixel value of the i-th pixel point corresponding to the first currently predicted reconstructed image and the current true reconstructed image. i ' represents the i-th difference in the first set of adjusted differences, and μ represents the mean of the differences between the pixel values ​​of each corresponding pixel point between the first current predicted reconstructed image and the current true reconstructed image.

[0083] In an optional embodiment, the execution unit may determine the second mean and the second variance as follows: the second mean is determined to be equal to the mean of the differences in pixel values ​​between corresponding pixels in the second currently predicted reconstructed image and the current real reconstructed image; the second variance is determined to be equal to the variance of the differences in pixel values ​​between corresponding pixels in the second currently predicted reconstructed image and the current real reconstructed image; or, according to preset adjustment parameters, the differences in pixel values ​​between corresponding pixels in the second currently predicted reconstructed image and the current real reconstructed image are adjusted to obtain a second set of adjusted differences, wherein the second set of adjusted differences corresponds to each pixel; the second mean is determined to be equal to the mean of the second set of adjusted differences, and the second variance is determined to be equal to the variance of the second set of adjusted differences.

[0084] In an optional embodiment, the execution unit can obtain the second set of adjusted differences as follows: the second set of adjusted differences is obtained according to the following formula: x′j =Tx j +(1-T)μ, 1≤j≤C*H*W, where C, H, and W are the number of channels, height, and width of the second currently predicted reconstructed image, respectively, and T is the adjustment parameter, x j x is the difference between the pixel value of the j-th pixel point corresponding to the second currently predicted reconstructed image and the current true reconstructed image. j ' is the j-th difference in the second set of adjusted differences, and μ is the mean of the differences between the pixel values ​​of each corresponding pixel point between the second current predicted reconstructed image and the current real reconstructed image.

[0085] In an optional embodiment, the execution unit may determine the second current loss value as follows: The second current loss value is determined using the following formula: Among them, 2b S 2 =var(x) i (1≤i≤C*H*W),2b T 2 =var(x) j (1≤j≤C*H*W), where C, H, and W are the number of channels, height, and width of the first currently predicted reconstructed image and the second currently predicted reconstructed image, respectively, and x i x is the difference between the pixel value of the i-th pixel point corresponding to the first currently predicted reconstructed image and the current true reconstructed image. j μ is the difference between the pixel value of the j-th pixel point corresponding to the second currently predicted reconstructed image and the current true reconstructed image. S For the first mean, var(x) i ) represents the first variance value, μ T The second mean, var(x) j ) is the second variance value, and Loss2 is the second current loss value.

[0086] In an optional embodiment, the execution unit may determine the target loss value corresponding to the current sample image as follows: the target loss value corresponding to the current sample image is determined to be equal to the sum of the first current loss value and the second current loss value.

[0087] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0088] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0089] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0090] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0091] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0092] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0093] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0094] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A training method for an image reconstruction model, characterized in that, include: Obtain a target teacher image reconstruction model, a target sample image set, and a set of real reconstructed images corresponding to the target sample image set. The target teacher image reconstruction model is a model obtained by training a teacher image reconstruction model to be trained using the target sample image set. The loss value between the predicted reconstructed image set determined by the target teacher image reconstruction model based on the target sample image set and the real reconstructed image set satisfies a first preset loss condition. The target sample image set is used to train the student image reconstruction model until the target loss value corresponding to the student image reconstruction model meets the second preset loss condition, at which point the training ends and the target student image reconstruction model is obtained. The target loss value is a loss value determined based on a first loss value and a second loss value. The first loss value is the loss between the predicted reconstructed image determined by the student image reconstruction model based on the sample images in the target sample image set and the corresponding real reconstructed image in the real reconstructed image set. The second loss value is a loss value determined based on a first difference and a second difference. The first difference is the difference between each pixel point of the first predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The first predicted reconstructed image is the predicted reconstructed image determined by the student image reconstruction model based on the sample images in the target sample image set. The second difference is the difference between each pixel point of the second predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The second predicted reconstructed image is the predicted reconstructed image determined by the target teacher image reconstruction model based on the sample images in the target sample image set.

2. The method according to claim 1, characterized in that, The step of training the student image reconstruction model using the target sample image set until the target loss value corresponding to the student image reconstruction model satisfies the second preset loss condition includes: Repeat the following steps until the target loss value corresponding to the student image reconstruction model to be trained meets the second preset loss condition: Obtain the current sample image in the target sample image set, and the current real reconstructed image corresponding to the current sample image in the real reconstructed image set; The current sample image is input into the student image reconstruction model to be trained to obtain the first current predicted reconstruction image; A first current loss value is determined based on the first currently predicted reconstructed image and the current true reconstructed image, wherein the first current loss value represents the loss between the first currently predicted reconstructed image and the current true reconstructed image; The current sample image is input into the target teacher image reconstruction model to obtain the second current predicted reconstruction image; The second current loss value corresponding to the current sample image is determined based on the differences between each pixel between the first currently predicted reconstructed image and the current real reconstructed image, and the differences between each pixel between the second currently predicted reconstructed image and the current real reconstructed image. Based on the first current loss value and the second current loss value, determine the target loss value corresponding to the current sample image; If the target loss value corresponding to the current sample image satisfies the second preset loss condition, the training ends and the target student image reconstruction model is obtained. If the target loss value corresponding to the current sample image does not meet the second preset loss condition, the parameters in the student image reconstruction model to be trained are adjusted.

3. The method according to claim 2, characterized in that, The step of determining the second current loss value corresponding to the current sample image based on the differences between each pixel in the first currently predicted reconstructed image and the current true reconstructed image, and the differences between each pixel in the second currently predicted reconstructed image and the current true reconstructed image, includes: Obtain the difference between each pixel point corresponding to the first currently predicted reconstructed image and the current real reconstructed image; The first mean and the first variance are determined based on the differences between the corresponding pixels between the first currently predicted reconstructed image and the current real reconstructed image. Obtain the difference between each pixel point corresponding to the second currently predicted reconstructed image and the current real reconstructed image; The second mean and the second variance are determined based on the differences between the corresponding pixels between the second currently predicted reconstructed image and the current real reconstructed image. The second current loss value is determined based on the first mean, the first variance, the second mean, and the second variance.

4. The method according to claim 3, characterized in that, The step of determining the first mean and the first variance based on the differences between corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image includes: The first mean is defined as the mean of the differences between the pixel values ​​of corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image; the first variance is defined as the variance of the differences between the pixel values ​​of corresponding pixels in the first currently predicted reconstructed image and the current true reconstructed image; or According to preset adjustment parameters, the differences in pixel values ​​of corresponding pixels between the first current predicted reconstructed image and the current real reconstructed image are adjusted to obtain a first set of adjusted differences, wherein the first set of adjusted differences corresponds to each pixel; the first mean is determined to be equal to the mean of the first set of adjusted differences, and the first variance is determined to be equal to the variance of the first set of adjusted differences.

5. The method according to claim 4, characterized in that, The step involves adjusting the pixel values ​​of corresponding pixels between the first currently predicted reconstructed image and the current realistic reconstructed image according to preset adjustment parameters, to obtain a first set of adjusted differences, including: The adjusted difference for the first group is obtained using the following formula: , 1≤i≤C*H*W, Where C, H, and W are the number of channels, height, and width of the first currently predicted reconstructed image, respectively, and T is the adjustment parameter, x i x is the difference between the pixel value of the i-th pixel point corresponding to the first currently predicted reconstructed image and the current true reconstructed image. i ' represents the i-th difference in the first set of adjusted differences, and μ represents the mean of the differences between the pixel values ​​of each corresponding pixel point between the first current predicted reconstructed image and the current true reconstructed image.

6. The method according to claim 3, characterized in that, The step of determining the second mean and the second variance based on the differences between corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image includes: The second mean is defined as the mean of the differences between the pixel values ​​of corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image; the second variance is defined as the variance of the differences between the pixel values ​​of corresponding pixels in the second currently predicted reconstructed image and the current true reconstructed image; or According to preset adjustment parameters, the differences in pixel values ​​of corresponding pixels between the second current predicted reconstructed image and the current real reconstructed image are adjusted to obtain a second set of adjusted differences, wherein the second set of adjusted differences corresponds to each pixel; the second mean is determined to be equal to the mean of the second set of adjusted differences, and the second variance is determined to be equal to the variance of the second set of adjusted differences.

7. The method according to claim 6, characterized in that, The step involves adjusting the pixel values ​​of corresponding pixels between the second currently predicted reconstructed image and the current real reconstructed image according to preset adjustment parameters, resulting in two sets of adjusted differences, including: The adjusted difference for the second group is obtained using the following formula: , 1≤j≤C*H*W, Where C, H, and W are the number of channels, height, and width of the second currently predicted reconstructed image, respectively, and T is the adjustment parameter, x j x is the difference between the pixel value of the j-th pixel point corresponding to the second currently predicted reconstructed image and the current true reconstructed image. j ' is the j-th difference in the second set of adjusted differences, and μ is the mean of the differences between the pixel values ​​of each corresponding pixel point between the second current predicted reconstructed image and the current real reconstructed image.

8. The method according to claim 3, characterized in that, Determining the second current loss value based on the first mean, the first variance, the second mean, and the second variance includes: The second current loss value is determined using the following formula: ， Among them, 2b S 2 =var(x i (1≤i≤C*H*W), 2b T 2 =var(x j ) (1≤j≤C*H*W), Where C, H, and W represent the number of channels, height, and width of the first currently predicted reconstructed image and the second currently predicted reconstructed image, respectively, and x i x is the difference between the pixel value of the i-th pixel point corresponding to the first currently predicted reconstructed image and the current true reconstructed image. j μ is the difference between the pixel value of the j-th pixel point corresponding to the second currently predicted reconstructed image and the current true reconstructed image. S For the first mean, var(x) i ) represents the first variance value, μ T The second mean, var(x) j ) represents the second variance value, and Loss2 represents the second current loss value; b S b is the scale parameter between the first currently predicted reconstructed image and the current true reconstructed image. T The scale parameter is the difference between the second currently predicted reconstructed image and the current true reconstructed image.

9. The method according to claim 2, characterized in that, Determining the target loss value corresponding to the current sample image based on the first current loss value and the second current loss value includes: The target loss value corresponding to the current sample image is determined to be equal to the sum of the first current loss value and the second current loss value.

10. A training device for an image reconstruction model, characterized in that, include: The acquisition module is used to acquire a target teacher image reconstruction model, a target sample image set, and a set of real reconstructed images corresponding to the target sample image set. The target teacher image reconstruction model is a model obtained by training a teacher image reconstruction model to be trained using the target sample image set. The loss value between the predicted reconstructed image set determined by the target teacher image reconstruction model based on the target sample image set and the real reconstructed image set satisfies a first preset loss condition. The training module is used to train the student image reconstruction model to be trained using the target sample image set until the target loss value corresponding to the student image reconstruction model to be trained meets the second preset loss condition, at which point the training ends and a target student image reconstruction model is obtained. The target loss value is a loss value determined based on a first loss value and a second loss value. The first loss value is the loss value between the predicted reconstructed image determined by the student image reconstruction model to be trained based on the sample images in the target sample image set and the corresponding real reconstructed image in the real reconstructed image set. The second loss value is a loss value determined based on a first difference and a second difference. The first difference is the difference between each pixel point of the first predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The first predicted reconstructed image is the predicted reconstructed image determined by the student image reconstruction model to be trained based on the sample images in the target sample image set. The second difference is the difference between each pixel point of the second predicted reconstructed image and the corresponding real reconstructed image in the real reconstructed image set. The second predicted reconstructed image is the predicted reconstructed image determined by the target teacher image reconstruction model based on the sample images in the target sample image set.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method according to any one of claims 1 to 9.

12. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 9 through the computer program.

Images