Neural network training method and device, image processing method and device and storage medium

Abstract

The embodiment of the invention provides a neural network training method, a method and device for performing image processing by using a neural network, and a computer readable storage medium. The neural network training method according to the embodiment of the invention comprises the following steps: acquiring one or more input images for training, and acquiring one or more reference images which correspond to the one or more input images and represent true values; obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; calculating a brightness loss function for the neural network according to the values of the one or more input images, the one or more reference images and the one or more output images in the brightness channels; and training the neural network according to the brightness loss function for the neural network, and adjusting parameters of the neural network.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a neural network training method, as well as a method, device and computer-readable storage medium for performing image processing using a neural network. Background Art

[0002] Using neural networks to enhance low-light images is a crucial step in image processing. Existing techniques often require performing low-light image enhancement in multiple steps, resulting in a highly complex computational process. Furthermore, the enhancement process may require dividing high-resolution low-light images into separate regions, which can lead to inconsistent brightness and poor uniformity in the resulting stitched image.

[0003] Therefore, there is a need for an improved neural network training method and an image processing method for effectively performing image enhancement. Summary of the Invention

[0004] To solve the above technical problems, according to one aspect of the present invention, a neural network training method is provided, comprising: obtaining one or more input images for training, and obtaining one or more reference images representing true values ​​corresponding to the one or more input images; obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; calculating a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images, and the one or more output images in the brightness channel respectively; training the neural network based on the brightness loss function for the neural network, and adjusting the parameters of the neural network.

[0005] According to another aspect of the present invention, there is provided an image processing method, comprising: inputting an image to be processed; processing the image to be processed using a neural network to obtain a corresponding processed image; wherein the neural network is trained in the following manner: obtaining one or more input images for training, and obtaining one or more reference images representing true values ​​corresponding to the one or more input images; obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; calculating a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images, and the one or more output images in the brightness channel respectively; training the neural network based on the brightness loss function for the neural network, and adjusting the parameters of the neural network.

[0006] According to another aspect of the present invention, a neural network training device is provided, comprising: an acquisition unit configured to acquire one or more input images for training, and to acquire one or more reference images corresponding to the one or more input images and representing true values; a processing unit configured to acquire one or more corresponding output images generated after the one or more input images are processed by the neural network; a calculation unit configured to calculate a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images, and the one or more output images in the brightness channel; and a training unit configured to train the neural network based on the brightness loss function for the neural network and adjust the parameters of the neural network.

[0007] According to another aspect of the present invention, a neural network training device is provided, comprising: a processor; and a memory, wherein computer program instructions are stored in the memory, wherein when the computer program instructions are executed by the processor, the processor is caused to perform the following steps: obtaining one or more input images for training, and obtaining one or more reference images representing true values ​​corresponding to the one or more input images; obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; calculating a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images, and the one or more output images in the brightness channel respectively; training the neural network based on the brightness loss function for the neural network, and adjusting the parameters of the neural network.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the following steps: obtaining one or more input images for training, and obtaining one or more reference images representing true values ​​corresponding to the one or more input images; obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; calculating a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images, and the one or more output images in the brightness channel respectively; training the neural network based on the brightness loss function for the neural network, and adjusting the parameters of the neural network.

[0009] According to another aspect of the present invention, there is provided an image processing device, comprising: an input unit configured to input an image to be processed; a processing unit configured to process the image to be processed using a neural network to obtain a corresponding processed image; wherein the neural network is trained in the following manner: obtaining one or more input images for training, and obtaining one or more reference images representing true values ​​corresponding to the one or more input images; obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; calculating a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images and the one or more output images in the brightness channel respectively; training the neural network based on the brightness loss function for the neural network, and adjusting the parameters of the neural network.

[0010] According to another aspect of the present invention, a neural network training device is provided, comprising: a processor; and a memory, wherein computer program instructions are stored in the memory, wherein when the computer program instructions are executed by the processor, the processor is caused to perform the following steps: input an image to be processed; process the image to be processed using a neural network to obtain a corresponding processed image; wherein the neural network is trained in the following manner: obtain one or more input images for training, and obtain one or more reference images representing true values ​​corresponding to the one or more input images; obtain one or more corresponding output images generated after the one or more input images are processed by the neural network; calculate a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images, and the one or more output images in the brightness channel respectively; train the neural network based on the brightness loss function for the neural network, and adjust the parameters of the neural network.

[0011] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the following steps: input an image to be processed; process the image to be processed using a neural network to obtain a corresponding processed image; wherein the neural network is trained in the following manner: obtain one or more input images for training, and obtain one or more reference images representing true values ​​corresponding to the one or more input images; obtain one or more corresponding output images generated after the one or more input images are processed by the neural network; calculate a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images and the one or more output images in the brightness channel respectively; train the neural network based on the brightness loss function for the neural network, and adjust the parameters of the neural network.

[0012] According to the above-mentioned neural network training method, device and computer-readable storage medium of the present invention, as well as the method, device and computer-readable storage medium for image processing using a neural network, it is possible to generate a brightness loss function by introducing the values ​​of the brightness channels of the input image, the output image and the reference image representing the true value in the color space, and train the neural network in combination with the brightness loss function so that the generated output image can reflect the factors of the brightness channel during the processing process, thereby achieving relatively uniform brightening of the image and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other objects, features and advantages of the present invention will become more apparent by describing in detail the embodiments of the present invention with reference to the accompanying drawings.

[0014] Figure 1 A flowchart of a neural network training method according to an embodiment of the present invention is shown;

[0015] Figure 2 A flowchart showing a method for image processing using a neural network according to an embodiment of the present invention is shown;

[0016] Figure 3 An example of input images of an image batch according to an example of an embodiment of the present invention is shown;

[0017] Figure 4 An example of an embodiment of the present invention is shown. Figure 3 An example of a reference image corresponding to an input image in an image batch;

[0018] Figure 5 An example of an embodiment of the present invention is shown. Figure 3Example output images corresponding to the input images in an image batch;

[0019] Figure 6 A schematic diagram illustrating calculating a reference value of a corresponding brightness loss function using an input image and a reference image according to an example of an embodiment of the present invention is shown;

[0020] Figure 7 A schematic diagram illustrating calculating an estimated value of a corresponding brightness loss function using an input image and an output image according to an example of an embodiment of the present invention is shown;

[0021] Figure 8 A schematic diagram of neural network training according to an example of an embodiment of the present invention is shown;

[0022] Figure 9 A block diagram showing a neural network training device according to an embodiment of the present invention;

[0023] Figure 10 A block diagram showing a neural network training device according to an embodiment of the present invention;

[0024] Figure 11 A block diagram illustrating an apparatus for performing image processing using a neural network according to an embodiment of the present invention is shown;

[0025] Figure 12 A block diagram of an apparatus for performing image processing using a neural network according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0026] The following describes a neural network training method, apparatus, and computer-readable storage medium, as well as a method, apparatus, and computer-readable storage medium for image processing using a neural network, according to embodiments of the present invention, with reference to the accompanying drawings. In the accompanying drawings, like reference numerals represent like elements throughout. It should be understood that the embodiments described herein are merely illustrative and should not be construed as limiting the scope of the present invention.

[0027] When using neural networks to enhance low-light images, it's often necessary to perform the enhancement process in stages. For example, when processing high-resolution low-light images, you can first divide the image into regions, process each region separately, and then stitch the processed images together to obtain an image with improved illumination. However, this approach can easily lead to inconsistent brightness and poor uniformity in the stitched image.

[0028] To this end, embodiments of the present invention provide an improved neural network training method and an image processing method for effectively performing image enhancement.

[0029] Figure 1FIG. 1 is a flow chart of a neural network training method 100 according to an embodiment of the present invention. Figure 1 A neural network training method according to an embodiment of the present invention is described.

[0030] In step S101 , one or more input images for training are obtained, and one or more reference images corresponding to the one or more input images and representing true values ​​are obtained.

[0031] In this step, the input images required in the training process and the corresponding reference images can be obtained respectively. Optionally, the number of input images obtained for training can be one or more; accordingly, the number of reference images corresponding to the input images obtained can also be one or more. The one or more input images can be in one image batch, and accordingly, the one or more reference images can also be in the same image batch. In one example, the input image can be a low-light image with low brightness, and the reference image representing the true value can be an adjusted brightened image. In addition, in another example, the input image can be a panoramic image, and the reference image can also be a panoramic image accordingly. The above description and limitation of the input image and the corresponding reference image are only examples. In actual applications, appropriate input images and reference images can be arbitrarily selected according to the specific requirements of neural network training and image processing, and are not limited here.

[0032] In step S102, one or more corresponding output images generated after the one or more input images are processed by the neural network are obtained.

[0033] Optionally, the one or more input images can be input into a neural network to be trained for processing, thereby obtaining one or more output images. The one or more output images can be brightened images obtained by performing image enhancement processing on the one or more input images. During the training phase of the neural network, the one or more output images can be used to adjust and train the loss function of the neural network.

[0034] In step S103, a brightness loss function for the neural network is calculated based on the values ​​of the one or more input images, the one or more reference images, and the one or more output images in the brightness channel.

[0035] In this step, the reference values ​​of one or more brightness loss functions can be calculated in the following manner: convert the one or more input images and the one or more reference images into a color space having a brightness channel; obtain the values ​​of the one or more input images and the one or more reference images in the brightness channel respectively; and calculate the reference values ​​of one or more brightness loss functions according to the values ​​of the one or more input images in the brightness channel and the values ​​of the one or more reference images in the brightness channel. The color space can be an HSV space or a YCbCr space, and the brightness channel of the color space can be a V channel in the HSV space or a Y channel in the YCbCr space. In one example, the value GT of the V channel of a reference image in the HSV space can be obtained respectively. v And the corresponding input image V channel value in HSV space Input v , then, GT can be calculated v With Input v The corresponding pixel difference is taken as the reference value of the brightness loss function corresponding to the input image and the reference image, and is expressed as:

[0036] Ref v =|GT v -Input v | (1)

[0037] In addition, the estimated values ​​of one or more brightness loss functions can be calculated in the following manner: converting the one or more output images to a color space having a brightness channel; obtaining the values ​​of the one or more output images in the brightness channel respectively; and calculating the estimated values ​​of one or more brightness loss functions based on the values ​​of the one or more input images in the brightness channel and the values ​​of the one or more output images in the brightness channel. As mentioned above, the color space can also be an HSV space or a YCbCr space, and the brightness channel of the color space can be the V channel in the HSV space or the Y channel in the YCbCr space. In one example, the value of the V channel in the HSV space can be obtained as Input v The value of the V channel of the output image corresponding to the input image in HSV space Output v , then, the Output can be calculated v With Input v The corresponding pixel difference is taken as the absolute value of the estimated value of the brightness loss function corresponding to the input image and the output image, which is expressed as:

[0038] Pred v =|Output v -Input v | (2)

[0039] Obtain the reference value Ref of the brightness loss function of an input image, a corresponding reference image, and an output image respectively v and the estimated value Pred v After that, the reference value Ref of the brightness loss function corresponding to this input image and the corresponding reference image and output image can be calculated v The mean μ Ref and covariance∑ Ref , and the estimated value Pred of the brightness loss function v The mean μ Pred and covariance∑ Pred , used for the subsequent calculation of the neural network brightness loss function.

[0040] In one example, the mean μ of the reference values ​​of the brightness loss function corresponding to an input image and the corresponding reference image and output image can be calculated as follows: Ref :

[0041]

[0042] Where N is the number of pixels in the input image and i is the pixel index value.

[0043] In addition, the mean μ of the estimated values ​​of the brightness loss function corresponding to an input image and the corresponding reference image and output image can be calculated as follows Pred :

[0044]

[0045] Again, N is the number of pixels in the input image and i is the pixel index value.

[0046] On this basis, the reference value Ref of the brightness loss function can be v The covariance ∑ Ref Expressed as:

[0047]

[0048] And the estimated value Pred of the brightness loss function v The covariance ∑ Pred Expressed as:

[0049]

[0050] Where T is the matrix transpose.

[0051] After calculating the reference value Ref of the brightness loss function corresponding to an input image, a corresponding reference image, and an output image, v and the estimated value Pred vAfter the mean and covariance of , optionally, the mean μ of the reference value of the brightness loss function corresponding to this input image can be calculated respectively Ref and the mean μ of the estimated values Pred The mean square error MSE(μ Pred , μ Ref ), and the covariance ∑ of the reference value of the brightness loss function Ref and the covariance of the estimated value ∑ Pred The mean square error MSE (∑ Pred ,∑ Ref ), and weighted sum the two mean square errors. Afterwards, the two mean square errors corresponding to all input images and their corresponding reference images and output images are averaged to obtain the final brightness loss function L for calculating the neural network. brightness , expressed as:

[0052]

[0053] Where n is the number of input images previously fed into the neural network, i.e., the number of input images / reference images / output images in one batch previously fed into the neural network. λ1 and λ2 are the coefficients corresponding to the mean square error of the mean term and the mean square error of the covariance term in the brightness loss function, respectively.

[0054] In step S104, the neural network is trained according to the brightness loss function used for the neural network, and the parameters of the neural network are adjusted.

[0055] In this step, after obtaining the brightness loss function of the neural network, the brightness loss function used for the neural network can be superimposed with other loss functions used for the neural network to train the neural network and adjust the coefficients of the neural network.

[0056] Optionally, the loss function L of the neural network can be total Expressed as:

[0057] L total =σ1 L pixel +σ2 L perceptual +σ3 L GAN +...+σ n L brightness (8)

[0058] Among them L pixel is the pixel-level loss function between the reference image and the output image, L perceptual is the perceptual loss function between the reference image and the output image in the VGG space, L GANis the generative adversarial loss function between the reference image and the output image. σ1,...,σ n The coefficients corresponding to each loss function are as follows. The above is used to represent the total loss function L of the neural network. total The loss functions are all examples. In practical applications, any loss function that is suitable for the application scenario can be used without any restriction.

[0059] After calculating and obtaining the loss function of the neural network, the loss function of the neural network can be used to train the neural network to make it converge as much as possible, and the various parameters of the neural network can be continuously adjusted during the training process.

[0060] According to the above-mentioned neural network training method of the present invention, a brightness loss function can be generated by introducing the values ​​of the brightness channels of the input image, the output image and the reference image representing the true value in the color space, and the neural network can be trained in combination with the brightness loss function, so that the generated output image can reflect the factors of the brightness channel during the processing process, thereby achieving relatively uniform brightening of the image and improving the user experience.

[0061] Figure 2 FIG. 2 is a flow chart showing a method 200 for image processing using a neural network according to an embodiment of the present invention. Figure 1 The neural network trained by the process shown below is used for image processing. Figure 2 A method for image processing using a neural network according to an embodiment of the present invention is described.

[0062] In step S201 , an image to be processed is input.

[0063] In this step, the image to be processed may be input into the neural network. Optionally, the image to be processed may be a panoramic image.

[0064] In step S202, the image to be processed is processed using a neural network to obtain a corresponding processed image.

[0065] In this step, the input image to be processed can be processed as a whole using a neural network to obtain a processed image. Furthermore, the image to be processed can optionally be segmented, and each segmented portion of the image to be processed can be processed separately using a neural network, and the processed portions can be spliced ​​together to obtain a processed image. The above-mentioned methods of processing the image to be processed using a neural network are merely examples and are not intended to be limiting.

[0066] In the embodiment of the present invention, it is possible to use Figure 1The neural network trained in the process shown performs image processing, and its specific training method may include: obtaining one or more input images for training, and obtaining one or more reference images representing real values ​​corresponding to the one or more input images; obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; calculating the brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images and the one or more output images in the brightness channel respectively; training the neural network according to the brightness loss function for the neural network, and adjusting the parameters of the neural network.

[0067] The specific training operation method of the above neural network is Figure 1 The steps S101-S104 and the corresponding description are described in detail and will not be repeated here.

[0068] According to the above-mentioned image processing method of the present invention, a brightness loss function can be generated by introducing the values ​​of the brightness channels of the input image, the output image and the reference image representing the true value in the color space, and the neural network can be trained in combination with the brightness loss function so that the generated output image can reflect the factors of the brightness channel during the processing process, thereby achieving relatively uniform brightening of the image and improving the user experience.

[0069] The following describes a processing flow of a neural network training method and an image processing method according to an example of an embodiment of the present invention.

[0070] During the training process of the neural network, one or more input images for training are first obtained, and one or more reference images representing true values ​​corresponding to the one or more input images are obtained. The number of input images obtained for training can be one or more; accordingly, the number of reference images corresponding to the input images obtained can also be one or more. The one or more input images can be in one image batch, and accordingly, the one or more reference images can also be in the same image batch. In this example, the input image can be a low-light image with relatively low brightness, and can be a panoramic image or a part of a panoramic image; and the reference image representing the true value can correspondingly be an adjusted brightened panoramic image or a part of a panoramic image. Figure 3 An example of input images of an image batch according to an example of an embodiment of the present invention is shown. Figure 3 As shown, this image batch may include 8 input images. Figure 4 An example of an embodiment of the present invention is shown. Figure 3 An example of a reference image corresponding to an input image in an image batch. Figure 4 As shown, in the same image batch, 8 reference images can also be included. These 8 reference images are Figure 3 The 8 input images in are in one-to-one correspondence.

[0071] After the eight input images of this image batch are input into the neural network to be trained, one or more corresponding output images generated after processing can be obtained. Figure 5 An example of an embodiment of the present invention is shown. Figure 3 An example of the output images corresponding to the input images of an image batch in . Figure 5 As shown, it also includes 8 output images, which are Figure 3 The eight input images in are also in one-to-one correspondence, and the output image can be the image obtained by brightening the corresponding input image after being processed by the neural network.

[0072] In getting Figure 3 The 8 input images shown, Figure 4 The 8 reference images shown and Figure 5 After the 8 output images are shown, their values ​​in the brightness channel can be used to calculate the brightness loss function for the neural network.

[0073] In this example, the reference values ​​of one or more brightness loss functions can be calculated as follows: converting the one or more input images and the one or more reference images to a color space having a brightness channel; obtaining the values ​​of the one or more input images and the one or more reference images in the brightness channel; and calculating the reference values ​​of the one or more brightness loss functions based on the values ​​of the one or more input images in the brightness channel and the values ​​of the one or more reference images in the brightness channel. The color space can be, for example, an HSV space, and the brightness channel of the color space can be, for example, a V channel in the HSV space.

[0074] Figure 6 FIG. 1 shows a schematic diagram of calculating a reference value of a corresponding brightness loss function using an input image and a reference image according to an example of an embodiment of the present invention. Figure 6 As shown, a reference image can be obtained by converting the RGB space into the HSV space in the V channel value GT v , and the corresponding input image value in the V channel in HSV space Input v , then, GT can be calculated v With Input v The corresponding pixel difference of , its absolute value is used as the reference value of the brightness loss function corresponding to the input image and the reference image, and is expressed as formula (1).

[0075] Furthermore, one or more estimated values ​​of the brightness loss function may be calculated by converting the one or more output images to a color space having a brightness channel; obtaining values ​​of the one or more output images in the brightness channel; and calculating the estimated values ​​of the one or more brightness loss functions based on the values ​​of the one or more input images in the brightness channel and the values ​​of the one or more output images in the brightness channel. As previously mentioned, the color space may also be, for example, an HSV space, and the brightness channel of the color space may be, for example, a V channel in the HSV space.

[0076] Figure 7 FIG. 1 shows a schematic diagram of calculating an estimated value of a corresponding brightness loss function using an input image and an output image according to an example of an embodiment of the present invention. Figure 7 As shown, you can get the value of the V channel of an input image converted from RGB space to HSV space. v , and its corresponding output image value of the V channel in HSV space Output v , then, the Output can be calculated v With Input v , the corresponding pixel difference, its absolute value is used as the estimated value of the brightness loss function corresponding to the input image and the output image, and is expressed as formula (2).

[0077] Obtain the reference value Ref of the brightness loss function of an input image, a corresponding reference image, and an output image respectively v and the estimated value Pred v Afterwards, the reference value Ref of the brightness loss function corresponding to the input image and the corresponding reference image and output image can be calculated by equations (3) and (5) respectively: v The mean μ Ref and covariance∑ Ref , and calculate the estimated value Pred of the brightness loss function by equations (4) and (6) v The mean μ Pred and covariance∑ Pred , used for the subsequent calculation of the neural network brightness loss function.

[0078] After calculating the reference value Ref of the brightness loss function corresponding to an input image, a corresponding reference image, and an output image, v and the estimated value Pred v After the mean and covariance of , optionally, the mean μ of the reference value of the brightness loss function corresponding to this input image can be calculated respectively Ref and the mean μ of the estimated values Pred The mean square error MSE(μPred , μ Ref ), and the covariance ∑ of the reference value of the brightness loss function Ref and the covariance of the estimated value ∑ Pred The mean square error MSE (∑ Pred ,∑ Ref ), and the weighted sum of the two mean square errors. Then, the two mean square errors corresponding to all input images and their corresponding reference images and output images are averaged by formula (7) to obtain the final brightness loss function L for calculating the neural network brightness .

[0079] Finally, the neural network may be trained according to the brightness loss function used for the neural network, and the parameters of the neural network may be adjusted.

[0080] In this example, after obtaining the brightness loss function of the neural network, the brightness loss function used for the neural network can be superimposed with other loss functions used for the neural network using formula (8) to obtain the loss function of the neural network, so as to train the neural network and adjust the coefficients of the neural network.

[0081] Figure 8 FIG. 1 shows a schematic diagram of a neural network training according to an example of an embodiment of the present invention. Figure 8 In this process, multiple input images constituting an input panoramic image can be fed into a neural network to calculate multiple output images for constituting an output panoramic image. During this process, multiple reference images constituting a reference panoramic image can be used as ground truth values ​​for reference. After calculating the brightness loss function using the above method, the overall loss function of the neural network can be calculated based on this and used in the training process of the neural network.

[0082] After obtaining the trained neural network, in this example, an image to be processed may be input, and the image to be processed may be processed using the neural network to obtain a corresponding processed image.

[0083] Optionally, the image to be processed can also be segmented (such as Figure 8 Similarly, the input panoramic image to be processed is divided into multiple parts), and each part of the segmented image to be processed is processed by a neural network respectively, and the processed parts of the image are spliced ​​together to obtain the processed image.

[0084] Below, refer to Figure 9 The neural network training apparatus according to an embodiment of the present invention is described below. Figure 9 FIG. 9 is a block diagram of a neural network training apparatus 900 according to an embodiment of the present invention. Figure 9As shown, the neural network training device 900 includes an acquisition unit 910, a processing unit 920, a calculation unit 930, and a training unit 940. In addition to these units, the neural network training device 900 may also include other components. However, since these components are irrelevant to the content of the embodiment of the present invention, their illustration and description are omitted here. In addition, since the specific details of the following operations performed by the neural network training device 900 according to the embodiment of the present invention are the same as those described above with reference to Figure 1 The details described are the same, so repeated description of the same details is omitted here to avoid repetition.

[0085] Figure 9 The acquisition unit 910 of the neural network training device 900 acquires one or more input images for training, and acquires one or more reference images representing true values ​​corresponding to the one or more input images.

[0086] The acquisition unit 910 can respectively acquire the input images required in the training process and the reference images corresponding thereto. Optionally, the number of input images acquired for training can be one or more; accordingly, the number of reference images acquired corresponding to the input images can also be one or more. The one or more input images can be in one image batch, and accordingly, the one or more reference images can also be in the same image batch. In one example, the input image can be a low-light image with low brightness, and the reference image representing the true value can be an adjusted brightened image. In addition, in another example, the input image can be a panoramic image, and the reference image can also be a panoramic image accordingly. The above description and limitation of the input image and the corresponding reference image are only examples. In actual applications, appropriate input images and reference images can be arbitrarily selected according to the specific requirements of neural network training and image processing, and are not limited here.

[0087] The processing unit 920 obtains one or more corresponding output images generated after the one or more input images are processed by the neural network.

[0088] Optionally, the processing unit 920 may input the one or more input images into the neural network to be trained for processing, thereby obtaining one or more output images. The one or more output images may be brightened images obtained by performing image enhancement processing on the one or more input images. During the training phase of the neural network, the one or more output images may be used to adjust and train the loss function of the neural network.

[0089] The calculation unit 930 calculates a brightness loss function for the neural network based on the values ​​of the brightness channel of the one or more input images, the one or more reference images, and the one or more output images.

[0090] The calculation unit 930 can calculate the reference values ​​of one or more brightness loss functions in the following manner: convert the one or more input images and the one or more reference images into a color space having a brightness channel; obtain the values ​​of the one or more input images and the one or more reference images in the brightness channel respectively; and calculate the reference values ​​of one or more brightness loss functions according to the values ​​of the one or more input images in the brightness channel and the values ​​of the one or more reference images in the brightness channel. The color space can be an HSV space or a YCbCr space, and the brightness channel of the color space can be a V channel in the HSV space or a Y channel in the YCbCr space. In an example, the value GT of the V channel of a reference image in the HSV space can be obtained respectively. v And the corresponding input image V channel value in HSV space Input v , then, GT can be calculated v With Input v The corresponding pixel difference of , takes its absolute value as the reference value of the brightness loss function corresponding to the input image and the reference image, and is expressed using formula (1).

[0091] In addition, the calculation unit 930 can also calculate the estimated value of one or more brightness loss functions in the following manner: convert the one or more output images into a color space with a brightness channel; obtain the values ​​of the one or more output images in the brightness channel respectively; and calculate the estimated value of one or more brightness loss functions according to the values ​​of the one or more input images in the brightness channel and the values ​​of the one or more output images in the brightness channel. As mentioned above, the color space can also be an HSV space or a YCbCr space, and the brightness channel of the color space can be the V channel in the HSV space or the Y channel in the YCbCr space. In one example, the value of the V channel in the HSV space can be obtained as Input v The value of the V channel of the output image corresponding to the input image in HSV space Output v , then, the Output can be calculated v With Input v The corresponding pixel difference of , takes its absolute value as the estimated value of the brightness loss function corresponding to the input image and the output image, and expresses it using formula (2).

[0092] Obtain the reference value Ref of the brightness loss function of an input image, a corresponding reference image, and an output image respectively v and the estimated value Pred vAfter that, the reference value Ref of the brightness loss function corresponding to this input image and the corresponding reference image and output image can be calculated v The mean μ Ref and covariance∑ Ref , and the estimated value Pred of the brightness loss function v The mean μ Pred and covariance∑ Pred , used for the subsequent calculation of the neural network brightness loss function.

[0093] In one example, the mean μ of the reference values ​​of the brightness loss function corresponding to an input image and the corresponding reference image and output image can be calculated by formula (3): Ref , where N is the number of pixels in the input image and i is the pixel index value.

[0094] In addition, the mean μ of the estimated values ​​of the brightness loss function corresponding to an input image and the corresponding reference image and output image can be calculated by formula (4): Pred , similarly, N is the number of pixels in the input image and i is the pixel index value.

[0095] On this basis, the reference value Ref of the brightness loss function can be v The covariance ∑ Ref It is expressed in formula (5), and the estimated value Pred of the brightness loss function is v The covariance Z Pred It is expressed as formula (6), where T is the matrix transpose.

[0096] After calculating the reference value Ref of the brightness loss function corresponding to an input image, a corresponding reference image, and an output image, v and the estimated value Pred v After the mean and covariance of , optionally, the mean μ of the reference value of the brightness loss function corresponding to this input image can be calculated respectively Ref and the mean μ of the estimated values Pred The mean square error MSE(μ Pred , μ Ref ), and the covariance ∑ of the reference value of the brightness loss function Ref and the covariance of the estimated value ∑ Pred The mean square error MSE (∑ Pred ,∑ Ref ), and weighted sum the two mean square errors. Afterwards, the two mean square errors corresponding to all input images and their corresponding reference images and output images are averaged to obtain the final brightness loss function L for calculating the neural network. brightness, expressed as Equation (7). Here, n is the number of input images previously fed into the neural network, i.e., the number of input images / reference images / output images in one batch of images previously fed into the neural network. λ1 and λ2 are the coefficients corresponding to the mean square error of the mean term and the mean square error of the covariance term in the brightness loss function, respectively.

[0097] The training unit 940 trains the neural network according to the brightness loss function used for the neural network and adjusts the parameters of the neural network.

[0098] After obtaining the brightness loss function of the neural network, the training unit 940 can superimpose the brightness loss function for the neural network with other loss functions for the neural network to train the neural network and adjust the coefficients of the neural network.

[0099] Optionally, the loss function L of the neural network can be total It is expressed as formula (8). Where L pixel is the pixel-level loss function between the reference image and the output image, L perceptual is the perceptual loss function between the reference image and the output image in the VGG space, L GAN is the generative adversarial loss function between the reference image and the output image. n The coefficients corresponding to each loss function are as follows. The above is used to represent the total loss function L of the neural network. total The loss functions are all examples. In practical applications, any loss function that is suitable for the application scenario can be used without any restriction.

[0100] After calculating and obtaining the loss function of the neural network, the loss function of the neural network can be used to train the neural network to make it converge as much as possible, and the various parameters of the neural network can be continuously adjusted during the training process.

[0101] According to the above-mentioned neural network training device of the present invention, a brightness loss function can be generated by introducing the values ​​of the brightness channels of the input image, the output image and the reference image representing the true value in the color space, so as to train the neural network in combination with the brightness loss function, so that the generated output image can reflect the factors of the brightness channel during the processing process, thereby achieving relatively uniform brightening of the image and improving the user experience.

[0102] Below, refer to Figure 10 The neural network training apparatus according to an embodiment of the present invention is described below. Figure 10 FIG. 1 shows a block diagram of a neural network training apparatus 1000 according to an embodiment of the present invention. Figure 10 As shown, the device 1000 may be a computer or a server.

[0103] like Figure 10 As shown, the neural network training device 1000 includes one or more processors 1010 and a memory 1020. Of course, in addition to these, the neural network training device 1000 may also include an input device, an output device (not shown), etc. These components can be interconnected via a bus system and / or other forms of connection mechanisms. It should be noted that Figure 10 The components and structure of the neural network training device 1000 shown are merely exemplary and non-limiting. The neural network training device 1000 may also have other components and structures as needed.

[0104] The processor 1010 may be a central processing unit (CPU) or other form of processing unit having data processing capability and / or instruction execution capability, and may utilize computer program instructions stored in the memory 1020 to perform desired functions, which may include: obtaining one or more input images for training, and obtaining one or more reference images representing true values ​​corresponding to the one or more input images; obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; calculating a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images, and the one or more output images in the brightness channel; training the neural network based on the brightness loss function for the neural network, and adjusting the parameters of the neural network.

[0105] Memory 1020 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions may be stored on the computer-readable storage medium, and processor 1010 may execute the program instructions to implement the functions of the neural network training apparatus according to the embodiments of the present invention described above and / or other desired functions, and / or to execute the neural network training method according to the embodiments of the present invention. Various application programs and various data may also be stored in the computer-readable storage medium.

[0106] The following describes a computer-readable storage medium according to an embodiment of the present invention, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the following steps: obtaining one or more input images for training, and obtaining one or more reference images representing true values ​​corresponding to the one or more input images; obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; calculating a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images, and the one or more output images in the brightness channel; training the neural network based on the brightness loss function for the neural network, and adjusting the parameters of the neural network.

[0107] Below, refer to Figure 11 The apparatus for image processing using a neural network according to an embodiment of the present invention is described. Figure 11 FIG. 1 shows a block diagram of an apparatus 1100 for image processing using a neural network according to an embodiment of the present invention. Figure 11 As shown, the apparatus 1100 for image processing using a neural network includes an input unit 1110 and a processing unit 1120. In addition to these units, the apparatus 1100 may also include other components. However, since these components are irrelevant to the content of the embodiment of the present invention, their illustration and description are omitted here. In addition, since the specific details of the following operations performed by the apparatus 1100 according to the embodiment of the present invention are the same as those described above with reference to Figure 2 The details described are the same, so repeated description of the same details is omitted here to avoid repetition.

[0108] Figure 11 The input unit 1110 of the apparatus 1100 for performing image processing using a neural network inputs an image to be processed.

[0109] The input unit 1110 can input the image to be processed into the neural network. Optionally, the image to be processed can be a panoramic image.

[0110] The processing unit 1120 processes the image to be processed using a neural network to obtain a corresponding processed image.

[0111] The processing unit 1120 can process the input image to be processed as a whole using a neural network to obtain a processed image. Furthermore, the image to be processed can optionally be segmented, and each segmented portion of the image to be processed can be processed separately using a neural network, and the processed portions of the image can be spliced ​​together to obtain a processed image. The above-mentioned methods of processing the image to be processed using a neural network are merely examples and are not intended to be limiting.

[0112] In the embodiment of the present invention, it is possible to use Figure 1 The neural network trained in the process shown performs image processing, and its specific training method may include: obtaining one or more input images for training, and obtaining one or more reference images representing real values ​​corresponding to the one or more input images; obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; calculating the brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images and the one or more output images in the brightness channel respectively; training the neural network according to the brightness loss function for the neural network, and adjusting the parameters of the neural network.

[0113] The specific training operation method of the above neural network is Figure 1 It has been elaborated in detail in the corresponding description and will not be repeated here.

[0114] According to the above-mentioned image processing device of the present invention, a brightness loss function can be generated by introducing the values ​​of the brightness channels of the input image, the output image and the reference image representing the true value in the color space, and the neural network can be trained in combination with the brightness loss function so that the generated output image can reflect the factors of the brightness channel during the processing process, thereby achieving relatively uniform brightening of the image and improving the user experience.

[0115] Below, refer to Figure 12 The apparatus for image processing using a neural network according to an embodiment of the present invention is described. Figure 12 FIG. 1 shows a block diagram of an apparatus 1200 for image processing using a neural network according to an embodiment of the present invention. Figure 12 As shown, the device 1200 may be a computer or a server.

[0116] like Figure 12 As shown, the device 1200 includes one or more processors 1210 and a memory 1220. Of course, in addition to this, the device 1200 may also include an input device, an output device (not shown), etc. These components can be interconnected through a bus system and / or other forms of connection mechanisms. It should be noted that Figure 12 The components and structures of the device 1200 shown are merely exemplary and non-limiting. The device 1200 may also have other components and structures as needed.

[0117] The processor 1210 can be a central processing unit (CPU) or other form of processing unit with data processing capability and / or instruction execution capability, and can use the computer program instructions stored in the memory 1220 to perform the desired functions, which may include: inputting an image to be processed; using a neural network to process the image to be processed to obtain a corresponding processed image; wherein the neural network is trained in the following manner: obtaining one or more input images for training, and obtaining one or more reference images representing true values ​​corresponding to the one or more input images; obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; calculating the brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images and the one or more output images in the brightness channel respectively; training the neural network based on the brightness loss function for the neural network, and adjusting the parameters of the neural network.

[0118] The memory 1220 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1210 may execute the program instructions to implement the functions of the apparatus for image processing using a neural network according to the embodiment of the present invention described above and / or other desired functions, and / or may execute the method for image processing using a neural network according to the embodiment of the present invention. Various applications and various data may also be stored in the computer-readable storage medium.

[0119] The following describes a computer-readable storage medium according to an embodiment of the present invention, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the following steps: input an image to be processed; process the image to be processed using a neural network to obtain a corresponding processed image; wherein the neural network is trained in the following manner: obtain one or more input images for training, and obtain one or more reference images representing true values ​​corresponding to the one or more input images; obtain one or more corresponding output images generated after the one or more input images are processed by the neural network; calculate a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images, and the one or more output images in the brightness channel respectively; train the neural network based on the brightness loss function for the neural network, and adjust the parameters of the neural network.

[0120] Of course, the above-mentioned specific embodiments are merely examples and not limitations, and those skilled in the art can, based on the concept of the present invention, merge and combine some steps and devices from the various embodiments described separately above to achieve the effects of the present invention. Such merged and combined embodiments are also included in the present invention, and such merges and combinations are not described one by one here.

[0121] Note that the advantages, benefits, and effects mentioned in the present invention are merely illustrative and non-limiting, and should not be construed as necessarily possessed by each embodiment of the present invention. Furthermore, the specific details of the invention described above are provided for illustrative purposes and to facilitate understanding, and are not intended to be limiting. The details do not necessarily limit the present invention to being implemented using the specific details.

[0122] The block diagrams of the devices, apparatuses, equipment, and systems involved in the present invention are intended to be illustrative examples only and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems may be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and may be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and may be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and may be used interchangeably therewith.

[0123] The step flow charts and the above method descriptions in the present invention are intended to be illustrative examples only and are not intended to require or imply that the steps of the various embodiments must be performed in the order given. As will be appreciated by those skilled in the art, the order of the steps in the above embodiments can be performed in any order. Words such as "thereafter," "then," "next," and the like are not intended to limit the order of the steps; these words are only used to guide the reader through the description of these methods. In addition, any reference to an element in the singular, such as using the articles "a," "an," or "the," is not to be construed as limiting the element to the singular.

[0124] In addition, the steps and devices in the various embodiments of this document are not limited to being implemented in a certain embodiment. In fact, based on the concept of the present invention, relevant partial steps and partial devices in the various embodiments of this document can be combined to conceive new embodiments, and these new embodiments are also included in the scope of the present invention.

[0125] Each operation of the method described above may be performed by any suitable means capable of performing the corresponding functions, which may include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs), or processors.

[0126] The various illustrated logic blocks, modules, and circuits may be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an ASIC, a field programmable gate array signal (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but as an alternative, the processor may be any commercially available processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0127] The steps of the method or algorithm described in conjunction with the present invention can be directly embedded in hardware, in a software module executed by a processor, or in a combination of the two. The software module can exist in any form of tangible storage medium. Some examples of storage media that can be used include random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, etc. A storage medium can be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. In an alternative embodiment, the storage medium can be integral to the processor. A software module can be a single instruction or many instructions and can be distributed over several different code segments, between different programs, and across multiple storage media.

[0128] The methods herein include one or more actions for implementing the methods described. The methods and / or actions may be interchangeable with one another without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of the specific actions may be modified without departing from the scope of the claims.

[0129] The functions described can be implemented by hardware, software, firmware or any combination thereof. If implemented in software, the functions can be stored as one or more instructions on a tangible computer-readable medium. The storage medium can be any available tangible medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device or any other tangible medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer. As used herein, disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc.

[0130] Thus, a computer program product can perform the operations presented herein. For example, such a computer program product can be a computer-readable tangible medium having instructions tangibly stored (and / or encoded) thereon, the instructions being executable by one or more processors to perform the operations described herein. The computer program product can include packaging materials.

[0131] Software or instructions may also be transmitted via a transmission medium. For example, software may be transmitted from a website, server, or other remote source using a transmission medium such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, or microwave.

[0132] In addition, the modules and / or other appropriate means for performing the methods and techniques described herein can be downloaded and / or otherwise obtained by the user terminal and / or base station when appropriate. For example, such a device can be coupled to a server to facilitate the transmission of the means for performing the methods described herein. Alternatively, the various methods described herein can be provided via a storage component (e.g., RAM, ROM, a physical storage medium such as a CD or floppy disk) so that the user terminal and / or base station can obtain the various methods when being coupled to the device or providing a storage component to the device. In addition, any other appropriate technology for providing the methods and techniques described herein to a device can be utilized.

[0133] Other examples and implementations are within the scope and spirit of the invention and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwiring, or any combination of these. Features that implement the functions can also be physically located in various locations, including being distributed so that parts of the functions are implemented at different physical locations. Moreover, as used herein, including as used in the claims, "or" used in a list of items that begin with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). In addition, the wording "exemplary" does not mean that the example described is preferred or better than other examples.

[0134] Various changes, substitutions, and modifications may be made to the technology described herein without departing from the teachings as defined by the appended claims. Moreover, the scope of the claims is not limited to the specific aspects of the processes, machines, manufacture, compositions of things, means, methods, and actions described above. Currently existing or later developed processes, machines, manufacture, compositions of things, means, methods, or actions that perform substantially the same function or achieve substantially the same results as the corresponding aspects described herein may be utilized. Accordingly, the appended claims include within their scope such processes, machines, manufacture, compositions of things, means, methods, or actions.

[0135] The above description of the invented aspects is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features of the invention herein.

[0136] The above description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present invention to the forms invented herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A neural network training method comprising: Acquire one or more input images for training, and acquire one or more reference images corresponding to the one or more input images and representing true values; Obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; Calculating a brightness loss function for the neural network based on the values ​​of the brightness channel of the one or more input images, the one or more reference images, and the one or more output images; The neural network is trained according to the brightness loss function used for the neural network, and the parameters of the neural network are adjusted.

2. The method according to claim 1, wherein Calculating a brightness loss function for the neural network according to the values ​​of the brightness channel of the one or more input images, the one or more reference images, and the one or more output images includes: converting the one or more input images and the one or more reference images to a color space having a luma channel; Obtaining values ​​of the one or more input images and the one or more reference images in the luminance channel respectively; One or more reference values ​​of the brightness loss function are calculated based on the values ​​of the one or more input images in the brightness channel and the values ​​of the one or more reference images in the brightness channel.

3. The method according to claim 2, wherein: Calculating a brightness loss function for the neural network according to the values ​​of the brightness channel of the one or more input images, the one or more reference images, and the one or more output images includes: converting the one or more output images to a color space having a luminance channel; Obtaining values ​​of the brightness channel of the one or more output images; One or more estimated values ​​of the brightness loss function are calculated based on the values ​​of the one or more input images in the brightness channel and the values ​​of the one or more output images in the brightness channel.

4. The method according to claim 2 or 3, wherein: The color space is an HSV space or a YCbCr space.

5. The method according to claim 3, wherein: Calculating a brightness loss function for the neural network according to the values ​​of the brightness channel of the one or more input images, the one or more reference images, and the one or more output images includes: A brightness loss function for the neural network is calculated based on at least the one or more reference values ​​of the brightness loss function and the one or more estimated values ​​of the brightness loss function.

6. The method of claim 1, wherein: Training the neural network according to the brightness loss function for the neural network, and adjusting the parameters of the neural network includes: The brightness loss function for the neural network is superimposed on other loss functions for the neural network to train the neural network.

7. An image processing method, comprising: Input the image to be processed; Processing the image to be processed using a neural network to obtain a corresponding processed image; The neural network is trained in the following way: Acquire one or more input images for training, and acquire one or more reference images corresponding to the one or more input images and representing true values; Obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; Calculating a brightness loss function for the neural network based on the values ​​of the brightness channel of the one or more input images, the one or more reference images, and the one or more output images; The neural network is trained according to the brightness loss function used for the neural network, and the parameters of the neural network are adjusted.

8. A neural network training device comprising: an acquisition unit configured to acquire one or more input images for training, and acquire one or more reference images corresponding to the one or more input images and representing true values; a processing unit configured to obtain one or more corresponding output images generated after the one or more input images are processed by the neural network; a calculation unit configured to calculate a brightness loss function for the neural network based on the values ​​of the one or more input images, the one or more reference images, and the one or more output images in the brightness channel; A training unit is configured to train the neural network according to the brightness loss function used for the neural network and adjust the parameters of the neural network.

9. A neural network training device comprising: processor; and a memory having computer program instructions stored therein, When the computer program instructions are executed by the processor, the processor is caused to perform the following steps: Acquire one or more input images for training, and acquire one or more reference images corresponding to the one or more input images and representing true values; Obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; Calculating a brightness loss function for the neural network based on the values ​​of the brightness channel of the one or more input images, the one or more reference images, and the one or more output images; The neural network is trained according to the brightness loss function used for the neural network, and the parameters of the neural network are adjusted.

10. A computer-readable storage medium having computer program instructions stored thereon, wherein: When the computer program instructions are executed by a processor, the following steps are implemented: Acquire one or more input images for training, and acquire one or more reference images corresponding to the one or more input images and representing true values; Obtaining one or more corresponding output images generated after the one or more input images are processed by the neural network; Calculating a brightness loss function for the neural network based on the values ​​of the brightness channel of the one or more input images, the one or more reference images, and the one or more output images; The neural network is trained according to the brightness loss function used for the neural network, and the parameters of the neural network are adjusted.

Images