A loss function design method and device, a storage medium and an electronic device
By introducing gradient and chromaticity information into the loss function, the loss function of the neural network model is optimized, which solves the problem of poor subjective effect of existing loss functions in image enhancement tasks and improves the subjective effect of image sharpness, signal-to-noise ratio and color.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIVIEW TECH CO LTD
- Filing Date
- 2021-12-09
- Publication Date
- 2026-07-31
AI Technical Summary
While existing loss function designs can improve objective metrics such as PSNR in image enhancement tasks, they still fall short in subjective effects such as image sharpness, signal-to-noise ratio, and color.
The design incorporates loss terms that add gradient and chromaticity information. By calculating these loss terms and combining them with the basic loss term, the loss function of the neural network model is optimized.
It improves the subjective effects of neural network models in terms of image clarity, signal-to-noise ratio, and color, while maintaining objective accuracy indicators.
Smart Images
Figure CN116257731B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of neural network technology, and in particular to a loss function design method, apparatus, storage medium, and electronic device. Background Technology
[0002] Large-scale neural networks are a representative technique of deep learning, boasting excellent performance and wide applications. Early neural networks were primarily used for higher-level visual tasks such as classification, object detection, and semantic segmentation, achieving breakthroughs and widespread application. Subsequently, neural networks entered lower-level visual tasks such as image enhancement and restoration, achieving significantly better results than traditional algorithms in areas such as super-resolution restoration, noise reduction, and low-light enhancement.
[0003] For image enhancement, the performance of neural network models depends to some extent on the design of the loss function. Currently, the mainstream loss functions mainly include MAE (Mean Absolute Error) and MSE (Mean Squared Error) for the predicted and labeled images.
[0004] This type of design can achieve excellent objective indicators such as PSNR (Peak Signal to Noise Ratio), but subjective effects such as image sharpness, signal-to-noise ratio, and color are not ideal. Summary of the Invention
[0005] This application provides a loss function design method, apparatus, storage medium, and electronic device. The design of loss terms that add gradient information and chromaticity information can effectively improve the subjective effects of neural network model prediction results in terms of image clarity, signal-to-noise ratio, and color.
[0006] In a first aspect, embodiments of this application provide a loss function design method, the method comprising:
[0007] Based on the gradient and chromaticity information of the predicted and labeled images from the neural network model, gradient loss terms and chromaticity loss terms are obtained.
[0008] The loss function value of the neural network model is obtained based on the gradient loss term, the chromaticity loss term, and the predetermined basic loss term, for use in training the neural network model; wherein the basic loss term includes at least one of the mean absolute error loss term and the mean squared error loss term.
[0009] Secondly, embodiments of this application provide a loss function design apparatus, the apparatus comprising:
[0010] The loss term acquisition module is used to obtain gradient loss terms and chromaticity loss terms based on the gradient and chromaticity information of the predicted image and the labeled image of the neural network model.
[0011] The loss function value acquisition module is used to obtain the loss function value of the neural network model based on the gradient loss term, the chromaticity loss term, and the predetermined basic loss term, for use in training the neural network model; wherein the basic loss term includes at least one of the mean absolute error loss term and the mean squared error loss term.
[0012] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the loss function design method as described in embodiments of this application.
[0013] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the loss function design method as described in embodiments of this application.
[0014] The technical solution provided in this application obtains gradient loss terms and chromaticity loss terms based on the gradient and chromaticity information of the predicted image and the labeled image from the neural network model. Then, based on the gradient loss terms, chromaticity loss terms, and pre-determined basic loss terms, the loss function value of the neural network model is obtained for training the neural network model. This technical solution, by adding loss terms for gradient and chromaticity information, can effectively improve the subjective effects of the neural network model's prediction results in terms of image clarity, signal-to-noise ratio, and color. Attached Figure Description
[0015] Figure 1 This is a flowchart of the loss function design method provided in Embodiment 1 of this application;
[0016] Figure 2 This is a schematic diagram of the loss function design process provided in Embodiment 2 of this application;
[0017] Figure 3 This is a schematic diagram of another loss function design process provided in Embodiment 3 of this application;
[0018] Figure 4 This is a schematic diagram of the loss function design device provided in Embodiment 4 of this application;
[0019] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment Six of this application. Detailed Implementation
[0020] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.
[0021] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.
[0022] Example 1
[0023] Figure 1 This is a flowchart of the loss function design method provided in Embodiment 1 of this application. This embodiment can be applied to the design of loss functions for neural network models. The method can be executed by the loss function design device provided in this embodiment. The device can be implemented by software and / or hardware and can be integrated into devices such as smart terminals used for neural network model training.
[0024] like Figure 1 As shown, the loss function design method includes:
[0025] S110. Based on the gradient and chromaticity information of the predicted image and the labeled image from the neural network model, obtain the gradient loss term and the chromaticity loss term.
[0026] In this embodiment, the gradient loss term is used to describe the magnitude of the deviation of the gradient information between the predicted image O and the label image T of the neural network model. By adding the gradient loss term to the loss function of the neural network model training, the gradient information of the predicted image O can be made as close as possible to the label image T, thereby improving the model prediction results in terms of image sharpness, signal-to-noise ratio, etc.
[0027] In this scheme, the chromaticity loss term is used to describe the deviation of chromaticity information between the predicted image O and the label image T of the neural network model. By adding the chromaticity loss term to the loss function of the neural network model training, the chromaticity information of the predicted image O can be made as close as possible to the label image T, thereby improving the subjective effects of the model's prediction results, such as color.
[0028] The loss function is the objective function of a machine learning or optimization algorithm. Minimizing the loss function value is used to estimate or optimize the parameters of a neural network model. The loss function proposed in this scheme is suitable for image enhancement applications based on neural network models, including but not limited to: image denoising, super-resolution restoration, low-light enhancement, rain and fog removal, etc.
[0029] In this scheme, the gradient loss term can be calculated using the gradient information of the predicted image O and the label image T from the neural network model; and the chromaticity loss term can be calculated using the chromaticity information of the predicted image O and the label image T from the neural network model.
[0030] S120. Based on the gradient loss term, chromaticity loss term, and predetermined basic loss term, obtain the loss function value of the neural network model for use in training the neural network model; wherein, the basic loss term includes at least one of the mean absolute error loss term and the mean squared error loss term.
[0031] The predetermined basic loss term can be the average absolute error loss or mean squared error between the predicted image O and the labeled image T of the neural network model.
[0032] Specifically, the mean absolute error loss term Loss can be calculated using the following formula. MAE Loss term and mean squared error MSE :
[0033]
[0034]
[0035] Where N is the total number of pixels in the image, and the subscript i represents the i-th pixel in the image, O i T represents the i-th pixel of the predicted image O. i This represents the i-th pixel of the label image T.
[0036] In this scheme, the gradient loss term, chromaticity loss term, and pre-determined basic loss term can be combined with different weights to obtain the loss function of the neural network model. The parameters are then tuned to obtain the loss function value, which can be used for training the neural network model.
[0037] In this technical solution, optionally, the loss function value of the neural network model is obtained based on the gradient loss term, the chromaticity loss term, and the predetermined basic loss term, including:
[0038] According to the preset third weight, the gradient loss term, chromaticity loss term, and the pre-determined basic loss term are weighted and fused to obtain the loss function value of the neural network model.
[0039] Specifically, the loss function Loss of the neural network model can be obtained using the following formula:
[0040] Loss = Loss MAE / MSE +λ1×Loss GAE +λ2×Loss CSI ;
[0041] Among them, Loss MAE / MSE Represents Loss MAE or Loss MSE That is, you can choose either one, Loss GAE For the gradient loss term, Loss CSI The chromaticity loss term is represented by two hyperparameters λ1 and λ2, which are the weights of the gradient loss term and the chromaticity loss term, respectively. Their values can be adjusted according to the training effect, and are set to λ1 = λ2 = 0.5 by default.
[0042] By adding loss terms containing gradient and chromaticity information, the subjective effects of the model's predictions in terms of image clarity, signal-to-noise ratio, and color can be effectively improved while maintaining the objective accuracy of the neural network model.
[0043] The technical solution provided in this application obtains gradient loss terms and chromaticity loss terms based on the gradient and chromaticity information of the predicted image and the labeled image from the neural network model. Then, based on the gradient loss terms, chromaticity loss terms, and pre-determined basic loss terms, the loss function value of the neural network model is obtained for training the neural network model. By implementing this technical solution and adding loss terms based on gradient and chromaticity information, the subjective effects of the neural network model's prediction results in terms of image sharpness, signal-to-noise ratio, and color can be effectively improved.
[0044] Example 2
[0045] Figure 2 This is a schematic diagram of the loss function design process provided in Embodiment 2 of this application. Embodiment 2 is a further optimization based on Embodiment 1. Specifically, the optimization involves obtaining a gradient loss term based on the gradient information of the predicted image and the label image from the neural network model. This includes: calculating the gradient components of the predicted image and the label image to obtain the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image; and calculating the gradient loss term based on the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image. For details not described in this embodiment, please refer to Embodiment 1. Figure 2 As shown, the method includes the following steps:
[0046] S210. Calculate the gradient components of the predicted image and the label image to obtain the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image.
[0047] The gradient components include horizontal gradient components and vertical gradient components, which can be calculated for the filtered predicted image O and the label image T, respectively.
[0048] In this scheme, the predicted image and the label image can be filtered versions of the predicted image and the label image. Specifically, the predicted image O and the label image T from the neural network are subjected to smoothing filtering to remove noise from the images, resulting in filtered predicted image O and label image T. The predicted image O and the label image T have the same image size.
[0049] Optionally, Gaussian filtering, mean filtering, median filtering, and other methods can be used to smooth the predicted image O and the label image T of the neural network.
[0050] Optionally, in this technical solution, gradient component calculation is performed on the predicted image and the label image to obtain the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image, including:
[0051] The predicted image and the label image are convolved with a pre-determined gradient spatial filtering kernel to obtain the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image.
[0052] The predetermined gradient spatial domain filter kernels include a horizontal gradient spatial domain filter kernel and a vertical gradient spatial domain filter kernel.
[0053] In this embodiment, the predicted image O is convolved with both the horizontal gradient spatial domain filter kernel and the vertical gradient spatial domain filter kernel to obtain the horizontal gradient component O of the predicted image. x and vertical gradient spatial domain filter kernel O y Furthermore, the label image T is convolved with both the horizontal gradient spatial domain filter kernel and the vertical gradient spatial domain filter kernel to obtain the horizontal gradient component T of the label image. x and the vertical gradient component T of the label image y Among them, the images O, T, and their gradient components O x O y T x T y They all have the same image size.
[0054] Optionally, the horizontal gradient spatial filtering kernel is:
[0055]
[0056] The vertical gradient spatial filtering kernel is:
[0057]
[0058] The horizontal gradient component O of the predicted image can be calculated using the following formula. x and vertical gradient spatial domain filter kernel O y :
[0059]
[0060]
[0061] in, Convolution operation.
[0062] The horizontal gradient component T of the label image can be calculated using the following formula. x and the vertical gradient component T of the label image y :
[0063]
[0064]
[0065] By calculating the horizontal gradient components of the predicted image, the horizontal gradient components of the label image, the vertical gradient components of the predicted image, and the vertical gradient components of the label image, the gradient loss term can be determined, making the gradient information of the model's predicted image as close as possible to the label image, thereby improving the model's prediction results in terms of image clarity, signal-to-noise ratio, etc.
[0066] S220. The gradient loss term is calculated based on the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image.
[0067] In this scheme, the horizontal gradient component O of the predicted image can be used as a basis. x Horizontal gradient component O of the label image y Predicting the vertical gradient component T of the image x and the vertical gradient component T of the label image y The gradient deviation between the predicted image O and the label image T at each pixel is calculated, and the mean and standard deviation of the gradient deviation are calculated. Then, the gradient loss term is calculated based on the mean and standard deviation of the gradient deviation.
[0068] In this technical solution, optionally, a gradient loss term is calculated based on the predicted image horizontal gradient component, the label image horizontal gradient component, the predicted image vertical gradient component, and the label image vertical gradient component, including:
[0069] The image gradient deviation is calculated based on the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image.
[0070] Calculate the mean and standard deviation of the image gradient deviation to obtain the mean and standard deviation of the gradient deviation;
[0071] The gradient loss term is calculated by weighting and fusing the mean gradient deviation and the standard deviation of gradient deviation according to a preset first weight.
[0072] Among them, the horizontal gradient component O of the predicted image can be used as a basis. x and the horizontal gradient component T of the label image x The horizontal gradient deviation at each pixel is calculated, and the vertical gradient component O of the predicted image is also calculated. y and the vertical gradient component T of the label image y The vertical gradient deviation at each pixel is calculated, and the horizontal gradient deviation and the vertical gradient deviation are added together to obtain the image gradient deviation.
[0073] Specifically, the image gradient deviation M can be calculated using the following formula. GAE :
[0074] M GAE (i)=|O x (i)-T x (i)|+|O y (i)-T y (i)|.
[0075] Among them, O x (i) represents O representing the horizontal gradient component of the predicted image. x The i-th pixel, T x (i) represents T, which represents the horizontal gradient component of the predicted image. x The i-th pixel, O y (i) represents O representing the horizontal gradient component of the predicted image. y The i-th pixel, T y (i) represents T, which represents the horizontal gradient component of the predicted image. y The image gradient deviation M for the i-th pixel GAE This reflects the gradient deviation of image O and T at each pixel. It is easy to see that M... GAE The image size is the same as that of O and T.
[0076] In this scheme, after determining the image gradient bias, the gradient loss term can be calculated based on the mean and standard deviation of the image gradient bias.
[0077] Specifically, the mean gradient deviation μ is calculated using the following formula. GAE and gradient deviation standard deviation σ GAE :
[0078]
[0079]
[0080] Furthermore, the gradient loss term Loss is calculated using the following formula. GAE :
[0081]
[0082] The above formula represents the gradient loss term Loss. GAE The mean gradient deviation μ GAE and gradient deviation standard deviation σ GAE The weighted fusion is controlled by α, which is generally set to 0.5. This calculation formula can objectively reflect the subjective perception of the human eye: that is, when the average deviation between two images is large, or the fluctuation range of the deviation is large, it reflects that the overall deviation between the two is large.
[0083] By adding a gradient loss term to the loss function during neural network model training, the gradient information of the model's predicted image can be made as close as possible to the label image, thereby improving the model's prediction results in terms of image clarity and signal-to-noise ratio.
[0084] The technical solution provided in this application calculates gradient components for the predicted image and the label image, obtaining the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image. Based on these components, a gradient loss term is calculated. By implementing this technical solution and adding a gradient information loss term, the subjective effects of the neural network model's prediction results in terms of image sharpness and signal-to-noise ratio can be effectively improved.
[0085] Example 3
[0086] Figure 3This is a schematic diagram of another loss function design process provided in Embodiment 3 of this application. Embodiment 3 further optimizes Embodiment 1. Specifically, the optimization is as follows: based on the chromaticity information of the predicted image and the label image from the neural network model, a chromaticity loss term is obtained, including: converting the predicted image and the label image to a preset color space, calculating the chromaticity components of the predicted image and the label image; and calculating the chromaticity loss term based on the chromaticity components of the predicted image and the label image. For details not described in this embodiment, please refer to Embodiment 1. Figure 3 As shown, the method includes the following steps:
[0087] S310. Convert the predicted image and the label image to a preset color space, and calculate the chromaticity components of the predicted image and the label image.
[0088] The preset color space can be a color space with brightness and chromaticity separation characteristics. For example, the color space can be YUV, YIQ, or Lab. Converting the filtered predicted image and the label image to the preset color space yields a two-dimensional chromaticity space, in which the objective deviation or similarity calculation values are closer to the subjective perception of the human eye.
[0089] In this embodiment, the predicted image and the label image can be filtered versions of the predicted image and the label image. Specifically, the predicted image O and the label image T from the neural network are subjected to smoothing filtering to remove noise from the images, resulting in filtered predicted image O and label image T. The predicted image O and the label image T have the same image size.
[0090] Optionally, Gaussian filtering, mean filtering, median filtering, and other methods can be used to smooth the predicted image O and the label image T of the neural network.
[0091] In this scheme, YIQ is used as the luminance-color separation space. RGB images are converted to YIQ space, and the I and Q components are extracted to form a two-dimensional chromaticity space (IQ). The advantages of this method are: firstly, RGB to YIQ conversion is a linear transformation with low computational complexity; secondly, the resulting IQ chromaticity space has good uniformity, making the calculated objective deviations or similarity values in this space closer to human subjective perception. The chromaticity components I and Q corresponding to any RGB color image are calculated as follows:
[0092]
[0093] Using the formula above, the chromaticity component I of the predicted image O can be calculated separately. o Q o The chromaticity component I of the label image T T QT .
[0094] S320. Calculate the chromaticity loss term based on the predicted image chromaticity components and the label image chromaticity components.
[0095] In this scheme, the chromaticity component I of the predicted image can be used as a basis. o Q o and label image chromaticity component I T Q T The image chromaticity similarity between the predicted image and the labeled image is calculated, and the chromaticity loss term is calculated based on the mean and standard deviation of the image chromaticity similarity.
[0096] In this technical solution, optionally, a chromaticity loss term is calculated based on the predicted image chromaticity components and the label image chromaticity components, including:
[0097] The image chromaticity similarity is calculated based on the predicted image chromaticity components and the label image chromaticity components.
[0098] Calculate the mean and standard deviation of the chromaticity similarity of the images to obtain the mean and standard deviation of chromaticity similarity;
[0099] The mean and standard deviation of chromaticity similarity are weighted and fused according to a preset second weight to calculate the chromaticity loss term.
[0100] In this embodiment, the image chromaticity similarity can be calculated based on the predicted image chromaticity components and the label image chromaticity components according to a preset chromaticity similarity formula.
[0101] Specifically, the image chromatic similarity M can be calculated using the following formula. CSI :
[0102]
[0103] Among them, M CSI This reflects the chromatic similarity between images O and T at each pixel, with a value ranging from -1 to 1. c1 is an adjustable parameter, which can be adjusted according to the desired effect; the default value is c1 = 0.01. This calculation method can accurately reflect the relative deviation of images O and T in terms of chromaticity, and the relative deviation of chromaticity is more in line with the subjective perception of color differences by the human eye than the absolute deviation.
[0104] Optionally, the mean chromaticity similarity μ can be calculated using the following formula. CSI and the standard deviation of chromaticity similarity σ CSI :
[0105]
[0106]
[0107] Furthermore, the chromaticity loss term Loss is calculated using the following formula. CSI :
[0108]
[0109] Among them, due to μ CSI This represents the average chromatic similarity between two images (O, T), therefore 1-μ CSI It can represent the average chromaticity deviation between two images; σ CSI This represents the fluctuation range of image chromaticity similarity, that is, the fluctuation range of image chromaticity deviation.
[0110] The above formula represents the loss term in chromaticity. CSI The average colorimetric deviation is 1-μ CSI and standard deviation σ CSI The weighted fusion is controlled by β, which is generally set to β = 2. This calculation formula can objectively reflect the subjective perception of the human eye: that is, when the average deviation between two images is large, or the fluctuation range of the deviation is large, it reflects that the overall deviation between the two is large.
[0111] By calculating the chromaticity loss term and adding it to the loss function during neural network model training, the chromaticity information of the model's predicted image can be made as close as possible to the label image, thereby improving the model's prediction results in terms of subjective effects such as color.
[0112] The technical solution provided in this application converts the predicted image and the label image to a preset color space, calculates the chromaticity components of the predicted image and the label image, and calculates the chromaticity loss term based on the chromaticity components of the predicted image and the label image. By implementing this technical solution, the design of the chromaticity information loss term is added, which can effectively improve the subjective effect of the neural network model prediction results in terms of image color, etc.
[0113] Example 4
[0114] Figure 4 This is a schematic diagram of the loss function design device provided in Embodiment 4 of this application, as shown below. Figure 4 As shown, the loss function design apparatus includes:
[0115] The loss term acquisition module 410 is used to obtain gradient loss term and chromaticity loss term based on the gradient information and chromaticity information of the predicted image and the labeled image of the neural network model.
[0116] The loss function value acquisition module 420 is used to obtain the loss function value of the neural network model based on the gradient loss term, the chromaticity loss term, and the predetermined basic loss term, for use in training the neural network model; wherein the basic loss term includes at least one of the mean absolute error loss term and the mean squared error loss term.
[0117] In this technical solution, optionally, the loss term obtaining module 410 includes:
[0118] The gradient component calculation unit is used to calculate the gradient components of the predicted image and the label image to obtain the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image.
[0119] The gradient loss term calculation unit is used to calculate the gradient loss term based on the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image.
[0120] In this technical solution, the optional gradient component calculation unit is specifically used for:
[0121] The predicted image and the label image are convolved with a pre-determined gradient spatial filtering kernel to obtain the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image.
[0122] In this technical solution, the optional gradient loss term calculation unit is specifically used for:
[0123] The image gradient deviation is calculated based on the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image.
[0124] Calculate the mean and standard deviation of the image gradient deviation to obtain the mean and standard deviation of the gradient deviation;
[0125] The gradient loss term is calculated by weighting and fusing the mean gradient deviation and the standard deviation of gradient deviation according to a preset first weight.
[0126] In this technical solution, optionally, the loss term obtaining module 410 includes:
[0127] A chromaticity component calculation unit is used to convert the predicted image and the label image to a preset color space and calculate the chromaticity components of the predicted image and the label image.
[0128] The chromaticity loss term calculation unit is used to calculate the chromaticity loss term based on the chromaticity components of the predicted image and the chromaticity components of the label image.
[0129] In this technical solution, optionally, the unit for calculating the chromaticity loss term is specifically used for:
[0130] The image chromaticity similarity is calculated based on the predicted image chromaticity components and the label image chromaticity components.
[0131] Calculate the mean and standard deviation of the chromaticity similarity of the images to obtain the mean and standard deviation of chromaticity similarity;
[0132] The mean and standard deviation of chromaticity similarity are weighted and fused according to a preset second weight to calculate the chromaticity loss term.
[0133] In this technical solution, optionally, the loss term acquisition module 410 is specifically used for:
[0134] According to the preset third weight, the gradient loss term, chromaticity loss term, and the pre-determined basic loss term are weighted and fused to obtain the loss function value of the neural network model.
[0135] The above-mentioned products can perform the methods provided in the embodiments of this application, and have the corresponding functional modules and beneficial effects of performing the methods.
[0136] Example 5
[0137] This application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a loss function design method, the method comprising:
[0138] Based on the gradient and chromaticity information of the predicted and labeled images from the neural network model, gradient loss terms and chromaticity loss terms are obtained.
[0139] The loss function value of the neural network model is obtained based on the gradient loss term, the chromaticity loss term, and the predetermined basic loss term, for use in training the neural network model; wherein the basic loss term includes at least one of the mean absolute error loss term and the mean squared error loss term.
[0140] Storage medium – any type of memory device or storage device. The term “storage medium” is intended to include: mounting media, such as CD-ROM, floppy disk, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. Storage medium may also include other types of memory or combinations thereof. Furthermore, storage medium may reside in a computer system in which a program is executed, or it may reside in a different second computer system connected to the computer system via a network (such as the Internet). The second computer system can provide program instructions to the computer for execution. The term “storage medium” can include two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). Storage medium may store program instructions (e.g., specifically implemented as a computer program) that can be executed by one or more processors.
[0141] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the loss function design operations described above, but can also execute related operations in the loss function design method provided in any embodiment of this application.
[0142] Example 6
[0143] This application provides an electronic device that can integrate the loss function design apparatus provided in this application. Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment Six of this application. Figure 5 As shown, this embodiment provides an electronic device 500, which includes: one or more processors 520; and a storage device 510 for storing one or more programs. When the one or more programs are executed by the one or more processors 520, the one or more processors 520 implement the loss function design method provided in this embodiment, the method including:
[0144] Based on the gradient and chromaticity information of the predicted and labeled images from the neural network model, gradient loss terms and chromaticity loss terms are obtained.
[0145] The loss function value of the neural network model is obtained based on the gradient loss term, the chromaticity loss term, and the predetermined basic loss term, for use in training the neural network model; wherein the basic loss term includes at least one of the mean absolute error loss term and the mean squared error loss term.
[0146] Of course, those skilled in the art will understand that the processor 520 also implements the technical solutions of the loss function design method provided in any embodiment of this application.
[0147] Figure 5 The electronic device 500 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0148] like Figure 5 As shown, the electronic device 500 includes a processor 520, a storage device 510, an input device 530, and an output device 540; the number of processors 520 in the electronic device can be one or more. Figure 5 Taking a processor 520 as an example; the processor 520, storage device 510, input device 530, and output device 540 in the electronic device can be connected via a bus or other means. Figure 5 Taking the connection via bus 550 as an example.
[0149] The storage device 510, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and module units, such as the program instructions corresponding to the loss function design method in the embodiments of this application.
[0150] Storage device 510 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, storage device 510 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, storage device 510 may further include memory remotely located relative to processor 520, which can be connected via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0151] Input device 530 can be used to receive input digital, character, or voice information, and to generate key signal inputs related to user settings and function control of the electronic device. Output device 540 may include electronic devices such as a display screen and a speaker.
[0152] The electronic device provided in this application embodiment can achieve the purpose of improving the subjective effect of neural network model prediction results in terms of image clarity, signal-to-noise ratio, and color.
[0153] The loss function design apparatus, storage medium, and electronic device provided in the above embodiments can execute the loss function design method provided in any embodiment of this application, and have the corresponding functional modules and beneficial effects of executing the method. Technical details not described in detail in the above embodiments can be found in the loss function design method provided in any embodiment of this application.
[0154] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the appended claims.
Claims
1. A loss function design method, characterized in that, include: Based on the gradient and chromaticity information of the predicted and labeled images from the neural network model, gradient loss terms and chromaticity loss terms are obtained. The loss function value of the neural network model is obtained based on the gradient loss term, the chromaticity loss term, and the predetermined basic loss term, for use in training the neural network model; wherein the basic loss term includes at least one of the mean absolute error loss term and the mean squared error loss term. The gradient loss term is obtained based on the gradient information of the predicted image and the labeled image from the neural network model, including: The predicted image and the label image are convolved with a pre-determined gradient spatial filtering kernel to obtain the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image. The image gradient deviation is calculated based on the horizontal gradient component of the predicted image, the horizontal gradient component of the label image, the vertical gradient component of the predicted image, and the vertical gradient component of the label image. Calculate the mean and standard deviation of the image gradient deviation to obtain the mean and standard deviation of the gradient deviation; The gradient loss term is calculated by weighting and fusing the mean gradient deviation and the standard deviation of gradient deviation according to a preset first weight. The chromaticity loss term is obtained based on the chromaticity information of the predicted image and the labeled image from the neural network model, including: The predicted image and the label image are converted to a preset color space, and the chromaticity components of the predicted image and the label image are calculated. The image chromaticity similarity is calculated based on the predicted image chromaticity components and the label image chromaticity components. Calculate the mean and standard deviation of the chromaticity similarity of the images to obtain the mean and standard deviation of chromaticity similarity; The mean value of chromaticity similarity and the standard deviation of chromaticity similarity are weighted and fused according to a preset second weight to calculate the chromaticity loss term; Specifically, the mean and standard deviation of chromaticity similarity are weighted and fused according to a preset second weight to calculate the chromaticity loss term, which includes: The chromaticity loss term is calculated using the following formula; ； in, This represents the mean value of chromaticity similarity. This represents the standard deviation of chromaticity similarity. This represents the second weight. This represents the chromaticity loss term.
2. The method of claim 1, wherein, Based on the gradient loss term, chromaticity loss term, and pre-determined basic loss term, the loss function value of the neural network model is obtained, including: According to the preset third weight, the gradient loss term, chromaticity loss term, and the pre-determined basic loss term are weighted and fused to obtain the loss function value of the neural network model.
3. A loss function design apparatus configured to perform the method of any one of claims 1-2, wherein, include: The loss term acquisition module is used to obtain gradient loss terms and chromaticity loss terms based on the gradient and chromaticity information of the predicted image and the labeled image of the neural network model. The loss function value acquisition module is used to obtain the loss function value of the neural network model based on the gradient loss term, the chromaticity loss term, and the predetermined basic loss term, for use in training the neural network model; wherein the basic loss term includes at least one of the mean absolute error loss term and the mean squared error loss term.
4. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the loss function design method as described in any one of claims 1-2.
5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the loss function design method as described in any one of claims 1-2.