Loss value calculation method, device, electronic device and storage medium

Abstract

The present disclosure relates to a loss value calculation method, device, electronic device and storage medium. The loss value calculation method includes: obtaining a first image and a second image, wherein the first image is an image output after a preset deep learning model reconstructs the second image; performing sliding operations on the first image and the second image based on a preset sliding window and a preset step size, respectively, to determine a first self-difference value corresponding to a first pixel point in the first image and a second self-difference value corresponding to a second pixel point in the second image; determining a target loss value of a preset deep learning model based on a plurality of first self-difference values ​​and a plurality of second self-difference values, thereby being able to calculate the self-difference values ​​of the pixels in the first image and the second image respectively by means of a sliding window, and calculating the target loss value of the preset deep learning model based on the self-difference values, thereby avoiding the over-fitting phenomenon of the preset deep learning model caused by excessive attention to image position information and inability to correctly grasp the granularity.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a loss value calculation method, device, electronic device and storage medium. Background Art

[0002] As an emerging technology, deep learning is booming in the field of image processing. Deep learning models have the ability to autonomously learn the features of image data, and have accurate detection performance, high detection efficiency, and good generalization performance for various types of image tasks. This makes deep learning models widely used in the field of image processing, including image detection, image classification, image generation / reconstruction, etc.

[0003] In image generation / reconstruction tasks, how to determine the loss value of the deep learning model is a key factor in guiding the training of the deep learning model. The loss functions used by existing deep learning models for image generation / reconstruction tasks are generally mean-square error (MSE) and relative entropy. Relative entropy is also called Kullback-Leibler Divergence, or KL distance for short. The mean square error determines the loss value by calculating the difference between each pixel at the same position in two images, and the relative entropy determines the loss value by measuring the difference between two probability distributions in the same event space.

[0004] However, since the mean square error is too fine-grained and the KL distance focuses too much on the pixel distribution of the entire image and ignores the details, and the KL distance requires knowing the mean and variance, which are often unknown, it causes the deep learning model used for image generation / reconstruction tasks to overfit. Therefore, how to determine the loss value used to train the deep learning model for image generation / reconstruction tasks and avoid the overfitting of the deep learning model is a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to solve the above technical problems, the present disclosure provides a loss value calculation method, device, electronic device and storage medium.

[0006] A first aspect of an embodiment of the present disclosure provides a method for calculating a loss value, including:

[0007] Acquire a first image and a second image, where the first image is an image output after a preset deep learning model reconstructs the second image, and the second image is a real image input into the preset deep learning model;

[0008] Performing sliding operations on the first image and the second image based on a preset sliding window and a preset step size, respectively, to determine a first auto-difference value corresponding to at least one first pixel point in the first image and a second auto-difference value corresponding to at least one second pixel point in the second image;

[0009] A target loss value of a preset deep learning model is determined based on multiple first self-difference values ​​and multiple second self-difference values.

[0010] A second aspect of an embodiment of the present disclosure provides a loss value calculation device, including:

[0011] An image acquisition module, used to acquire a first image and a second image, wherein the first image is an image output after a preset deep learning model reconstructs the second image, and the second image is a real image input into the preset deep learning model;

[0012] An auto-difference value determination module is used to perform a sliding operation on the first image and the second image respectively based on a preset sliding window and a preset step size to determine a first auto-difference value corresponding to at least one first pixel point in the first image and a second auto-difference value corresponding to at least one second pixel point in the second image;

[0013] The loss value calculation module is used to determine the target loss value of the preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values.

[0014] A third aspect of the embodiments of the present disclosure provides an electronic device, including:

[0015] processor;

[0016] A memory for storing executable instructions;

[0017] Among them, the processor is used to read executable instructions from the memory and execute the executable instructions to implement the loss value calculation method provided by the first aspect above.

[0018] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the loss value calculation method provided by the first aspect above.

[0019] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0020] The loss value calculation method, device, electronic device and storage medium provided by the embodiments of the present disclosure can obtain a first image and a second image, wherein the first image is an image output after a preset deep learning model reconstructs the second image, and the second image is a real image input to the preset deep learning model; after obtaining the first image and the second image, a sliding operation is performed on the first image and the second image based on a preset sliding window and a preset step size, respectively, to determine a first self-difference value corresponding to at least one first pixel point in the first image and a second self-difference value corresponding to at least one second pixel point in the second image, and determine a target loss value of the preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values, thereby, the self-difference values ​​of the pixels in the first image and the second image can be calculated respectively by means of a sliding window, and a representation of the difference in the distribution of the image pixels corresponding to the first image and the second image can be obtained, and then the target loss value of the preset deep learning model is calculated based on the self-difference value, which draws on the invariance of the convolution operation and avoids the overfitting phenomenon of the preset deep learning model caused by excessive attention to the image position information and the inability to correctly grasp the granularity. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1 is a flow chart of a loss value calculation method provided by an embodiment of the present disclosure;

[0024] Figure 2 is a schematic diagram of a sliding window display structure provided by an embodiment of the present disclosure;

[0025] Figure 3 is another schematic diagram of a sliding window display structure provided by an embodiment of the present disclosure;

[0026] Figure 4 It is another schematic diagram of a sliding window display structure provided by an embodiment of the present disclosure;

[0027] Figure 5 is a structural schematic diagram of a loss value calculation device provided by an embodiment of the present disclosure;

[0028] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0030] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0031] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0032] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0033] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0034] In order to address the problem that the loss value of a deep learning model for image generation / reconstruction tasks is determined based on mean square error and relative entropy in the prior art, which causes overfitting of the deep learning model for image generation / reconstruction tasks, an embodiment of the present disclosure provides a method for calculating the loss value, which is introduced below in conjunction with a specific embodiment.

[0035] Figure 1It is a flowchart of a loss value calculation method provided in an embodiment of the present disclosure. The method can be executed by a loss value calculation device. The loss value calculation device can be implemented in software and / or hardware. The loss value calculation device can be configured in an electronic device, such as a server or a terminal, wherein the terminal specifically includes a computer or a tablet computer, etc.

[0036] like Figure 1 As shown, the loss value calculation method provided in this embodiment includes the following steps.

[0037] S110. Acquire a first image and a second image, where the first image is an image output after a preset deep learning model reconstructs the second image, and the second image is a real image input into the preset deep learning model.

[0038] In an embodiment of the present disclosure, when the electronic device receives a model training instruction, it obtains a first image and a second image corresponding to the first image, wherein the first image is a reconstructed image output after the second image (i.e., the real image) is input into a preset deep learning model and the preset deep learning model reconstructs the second image.

[0039] In the embodiment of the present disclosure, the preset deep learning model may be any existing deep learning model that can be used for image generation / reconstruction.

[0040] In some embodiments of the present disclosure, the model training instruction includes an input image and an output image corresponding to a preset deep learning model. After receiving the model training instruction, the electronic device parses the model training instruction, obtains the input image and output image corresponding to the preset deep learning model, and then obtains the first image and the second image.

[0041] In some other embodiments of the present disclosure, the model training instruction includes identification information of a preset deep learning model. After receiving the model training instruction, the electronic device parses the model training instruction to obtain the identification information of the preset deep learning model, and based on the identification information of the preset deep learning model, obtains the input image and output image corresponding to the preset deep learning model from a preset database, and then obtains a first image and a second image corresponding to the first image.

[0042] In some other embodiments of the present disclosure, the model training instruction includes training data corresponding to a preset deep learning model, i.e., an input image (second image). After receiving the model training instruction, the electronic device parses the model training instruction, obtains the training data corresponding to the preset deep learning model, and inputs the training data into the preset deep learning model. The preset deep learning model reconstructs the training data to obtain an output image (i.e., the first image), thereby obtaining the first image and a second image corresponding to the first image.

[0043] S120, performing sliding operations on the first image and the second image respectively based on a preset sliding window and a preset step size, and determining a first self-difference value corresponding to at least one first pixel point in the first image and a second self-difference value corresponding to at least one second pixel point in the second image.

[0044] In an embodiment of the present disclosure, after acquiring the first image and the second image, the electronic device acquires a preset sliding window, and then performs sliding operations on the first image and the second image respectively based on the preset sliding window and the preset step size, and determines the first self-difference value corresponding to at least one first pixel point in the first image and the second self-difference value corresponding to at least one second pixel point in the second image based on the position information corresponding to each cell of the preset sliding window during the sliding operation, and the pixel value corresponding to the position information.

[0045] In an embodiment of the present disclosure, the preset sliding window may be a pre-set sliding window with a fixed window size and a fixed window shape, wherein the window shape is generally a square; the preset sliding window may also be a sliding window with a new window size and a fixed window shape obtained after adjusting the pre-set fixed window size.

[0046] The preset step length may be a preset fixed step length, or a step length obtained by adjusting the preset fixed step length.

[0047] In the embodiment of the present disclosure, the number of preset sliding windows and preset step sizes may be one or more, and is not limited here.

[0048] In an embodiment of the present disclosure, the preset step size may include a preset horizontal step size and a preset vertical step size. The sliding operation on the first image based on the preset sliding window and the preset step size may specifically include: starting from the upper left corner of the first image, positioning the window center position of the preset sliding window at the position of the first pixel point in the first row of the first image, and sliding from left to right according to the preset horizontal step size; when the preset sliding window slides to the position of the last pixel point in the first row of the first image, sliding the window center position of the preset sliding window back to the position of the first pixel point in the first row, and then sliding downward to the target pixel point position according to the preset vertical step size, and sliding from left to right according to the preset horizontal step size until the last pixel point in the last row of the first image is traversed or the preset sliding window is determined to exceed the edge position of the first image based on the preset step size.

[0049] Figure 2 is a schematic diagram of a sliding window display structure provided by an embodiment of the present disclosure, such as Figure 2As shown in the figure, a large square box including 36 white small square boxes represents the first image, each small square box represents a pixel point in the first image, wherein each pixel point includes a pixel coordinate and a pixel value, and a square box including 9 gray small square boxes, i.e., a square box with a window kernel of a square and a window size of 3×3, is used as a preset sliding window. When the preset sliding window is sliding on the first image, it starts from the first pixel point in the upper left corner of the first image, i.e., pixel point 11, and slides from left to right according to a preset horizontal step length, and the first pixel point is placed at the center of the window of the preset sliding window, and the value of the cell in the preset sliding window that exceeds the first image is set to 0, and the value of the cell in the first image is set to the pixel value of the pixel point corresponding to the cell, such as the value of the cell at pixel point 12 is the pixel value of the first pixel point in the first image. The pixel value corresponding to the second pixel point in a row, when sliding according to the preset step length, when the window center position of the preset sliding window slides to the last pixel point in the first row or the preset sliding window slides to exceed the last pixel point in the first row of the first image, the preset sliding window is slid back to the position of the first pixel point in the first row, that is, the window center position of the preset sliding window is slid back to the position of the first pixel point in the first row, and then moved downward according to the preset vertical step length to the target pixel point position. If the preset vertical step length is 1, the cell of the preset sliding window slides downward by one step length, and then slides from left to right according to the preset horizontal step length until the last pixel point in the last row of the first image is traversed or the preset sliding window is determined to exceed the edge position of the first image based on the preset step length.

[0050] In the embodiment of the present disclosure, the specific implementation of performing the sliding operation on the second image based on the preset sliding window and the preset step size is similar to the above-mentioned implementation of the sliding operation on the first image, and will not be described in detail here.

[0051] In the embodiment of the present disclosure, the first pixel point is a pixel point in the first image located at the window center position of the preset sliding window during the sliding operation of the preset sliding window according to the preset step size. When the preset step size is 1, that is, the preset horizontal step size and the preset vertical step size are both 1, the first pixel point is all the pixel points in the first image. When at least one of the preset horizontal step size and the preset vertical step size is not 1, the first pixel is a portion of the pixel points in the first image, that is, the portion of the pixel points located at the window center position of the preset sliding window.

[0052] The second pixel point is a pixel point in the second image located at the window center position of the preset sliding window during the sliding operation of the preset sliding window according to the preset step size.

[0053] S130. Determine a target loss value of a preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values.

[0054] In an embodiment of the present disclosure, after acquiring multiple first self-difference values ​​and multiple second self-difference values, the electronic device determines a target loss value of a preset deep learning model based on the multiple first self-difference values ​​and the multiple second self-difference values.

[0055] In some embodiments of the present disclosure, after obtaining multiple first self-difference values ​​and multiple second self-difference values, the electronic device calculates the sum of the multiple first self-difference values ​​and calculates the sum of the multiple second self-difference values, and determines the difference between the sum of the multiple first self-difference values ​​and the sum of the multiple second self-difference values ​​as the target loss value of the preset deep learning model.

[0056] In some other embodiments of the present disclosure, after obtaining multiple first self-difference values ​​and multiple second self-difference values, the electronic device performs weighted summation on the multiple first self-difference values ​​and the multiple second self-difference values, and determines the target loss value of the preset deep learning model based on the weighted summation result. Specifically, the first weight corresponding to each first self-difference value is determined, the first product of the first weight and the first self-difference value is calculated, and the sum of the multiple first products is calculated. At the same time, the second weight corresponding to each second self-difference value is determined, the second product of the second weight and the second self-difference value is calculated, and the sum of the multiple second products is calculated, and the difference between the sum of the multiple first products and the sum of the multiple second products is determined as the target loss value of the preset deep learning model.

[0057] In the disclosed embodiment, a first image and a second image can be acquired, wherein the first image is an image output after a preset deep learning model reconstructs the second image, and the second image is a real image input to the preset deep learning model; after acquiring the first image and the second image, a sliding operation is performed on the first image and the second image based on a preset sliding window and a preset step size, respectively, to determine a first self-difference value corresponding to at least one first pixel point in the first image and a second self-difference value corresponding to at least one second pixel point in the second image, and to determine a target loss value of the preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values, thereby, the self-difference values ​​of the pixels in the first image and the second image can be calculated respectively by means of a sliding window, and a representation of the difference in the distribution of the image pixels corresponding to the first image and the second image can be obtained, and then the target loss value of the preset deep learning model is calculated based on the self-difference value, which draws on the invariance of the convolution operation and avoids the overfitting phenomenon of the preset deep learning model caused by excessive attention to the image position information and the inability to correctly grasp the granularity.

[0058] Based on the above-mentioned embodiments of the present disclosure, determining the first self-difference value corresponding to at least one first pixel point in the first image can specifically include: for each first pixel point, determining at least one calculable pixel point corresponding to the first pixel point; and calculating the first self-difference value of the first pixel point based on the pixel value corresponding to the at least one calculable pixel point.

[0059] In the embodiment of the present disclosure, the computable pixel point can be understood as a pixel point that can be used to calculate the self-difference value of the first pixel point, for example, it can be a target pixel point adjacent to the first pixel point among the pixel points corresponding to the cells of the preset sliding window.

[0060] like Figure 2 As shown, when the first pixel point is pixel point 11, at this time, the at least one calculable pixel point corresponding to pixel point 11 is pixel point 12, pixel point 21, and pixel point 22.

[0061] Among them, determining at least one computable pixel point corresponding to the first pixel point may specifically include: determining the target pixel position corresponding to each cell in the preset sliding window in the first image based on a preset hole coefficient; determining at least one computable pixel point corresponding to the first pixel point based on the positional relationship between the target pixel position and the first pixel point.

[0062] The preset hole coefficient may be 0, 1, or other integers greater than 1, and is not limited here.

[0063] Figure 3 is another schematic diagram of a sliding window display structure provided by an embodiment of the present disclosure. When the preset hole coefficient is 0, Figure 3 As shown, when the preset sliding window slides to the current position of the first image, the target pixel positions corresponding to each cell in the preset sliding window are pixel 32, pixel 33, pixel 34, pixel 42, pixel 43, pixel 44, pixel 52, pixel 53, and pixel 54 respectively. At this time, the first pixel is pixel 43, and the pixels in the target pixel positions corresponding to the cells of the preset sliding window and adjacent to the first pixel 43 are pixel 32, pixel 33, pixel 34, pixel 42, pixel 44, pixel 52, pixel 53, and pixel 54. Therefore, pixel 32, pixel 33, pixel 34, pixel 42, pixel 44, pixel 52, pixel 53, and pixel 54 are determined as the first computable pixels.

[0064] Figure 4 is another schematic diagram of a sliding window display structure provided by an embodiment of the present disclosure. When the preset hole coefficient is 1, Figure 4 As shown, when the first pixel point is pixel point 43, when the preset hole coefficient is 1, at least one calculable pixel point is pixel point 21, pixel point 23, pixel point 25, pixel point 41, pixel point 45, pixel point 61, pixel point 63, and pixel point 65.

[0065] Furthermore, a first self-difference value is calculated for a first pixel based on pixel values ​​respectively corresponding to at least one calculable pixel, which may specifically include: determining the number of at least one calculable pixel, and a first pixel value respectively corresponding to each calculable pixel and a second pixel value corresponding to the first pixel; calculating the square value of the difference between the second pixel value and each first pixel value to obtain multiple square values; and determining the ratio of the sum of the multiple square values ​​to the number of at least one calculable pixel as the first self-difference value corresponding to the first pixel.

[0066] For example, Figure 2 As shown, the at least one calculable pixel corresponding to the first pixel 11 is pixel 12, pixel 21 and pixel 22, and the number of the at least one calculable pixel is 3. At this time, the calculation formula of the first self-difference value corresponding to the first pixel 11 is:

[0067] A=[(X 11 -X 12 ) 2 +(X 11 -X 21 ) 2 +(X 11 -X 22 ) 2 ] / 3

[0068] Where A represents the self-difference value of the first pixel 11, X 11 represents the pixel value of pixel 11, X 12 represents the pixel value of pixel 12, X 21 represents the pixel value of pixel 21, X 22 Represents the pixel value of pixel 22.

[0069] In the embodiment of the present disclosure, at least one calculable pixel point corresponding to the first pixel point can be determined by a preset hole coefficient, and then the first self-difference value corresponding to the first pixel point can be determined according to the first pixel value corresponding to the at least one calculable pixel point and the second pixel value corresponding to the first pixel point, which can satisfy different receptive fields and further improve the accuracy of the obtained first self-difference value.

[0070] It should be noted that the specific implementation of determining the second self-difference value corresponding to at least one second pixel point in the second image is similar to the above-mentioned implementation of determining the first self-difference value corresponding to at least one first pixel point in the first image, and will not be repeated here.

[0071] On the basis of the above-mentioned embodiments of the present disclosure, when the first image is a color image, such as an RGB image or an HSV image, determining the first self-difference value corresponding to at least one first pixel point in the first image can specifically include: performing channel separation on the first image to obtain multiple channel images corresponding to the first image; for each channel image, determining the third pixel points in the channel image corresponding to the at least one first pixel point, and calculating the target self-difference value corresponding to the at least one third pixel point.

[0072] In the embodiment of the present disclosure, the specific implementation of channel separation for the first image is similar to the existing implementation of channel separation for a multi-channel image, and will not be described in detail here.

[0073] In the embodiment of the present disclosure, the specific implementation of calculating the target self-difference value corresponding to at least one third pixel point is similar to the above-mentioned implementation of determining the first self-difference value corresponding to at least one first pixel point in the first image, and will not be repeated here.

[0074] Furthermore, determining a target loss value of a preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values ​​can specifically include: for each channel image, adding the target self-difference values ​​corresponding to at least one third pixel point to obtain a first value; adding the first values ​​corresponding to multiple channel images corresponding to the first image and dividing them by the number of channels corresponding to the first image to obtain a second value; determining the target loss value based on the second value and multiple second self-difference values.

[0075] In the embodiment of the present disclosure, the first value is used to characterize the difference in pixel distribution of each channel image corresponding to the first image; the second value is used to characterize the difference in pixel distribution of the first image.

[0076] Specifically, for each channel image obtained by performing channel separation on the first image, the target self-difference values ​​corresponding to at least one third pixel are added to obtain a first value, the first values ​​corresponding to multiple channel images corresponding to the first image are added and divided by the number of channels corresponding to the first image to obtain a second value, and at the same time, channel separation is performed on the second image to obtain multiple channel images corresponding to the second image, and for each channel image obtained by performing channel separation on the second image, the self-difference values ​​corresponding to fourth pixel points corresponding to at least one second pixel are calculated, the self-difference values ​​corresponding to at least one fourth pixel point are added to obtain a third value, the third values ​​corresponding to multiple channel images corresponding to the second image are added and the number of channels corresponding to the second image is processed to obtain a fourth value.

[0077] In the embodiment of the present disclosure, the third value is used to characterize the difference in pixel distribution of each channel image corresponding to the second image; and the fourth value is used to characterize the difference in pixel distribution of the first image.

[0078] Determining a target loss value based on the second numerical value and multiple second self-difference values ​​may specifically include: determining a target loss value of a preset deep learning model based on the second numerical value and a fourth numerical value.

[0079] Among them, determining the target loss value of the preset deep learning model based on the second numerical value and the fourth numerical value can specifically include: subtracting the second numerical value from the fourth numerical value, and determining the fifth numerical value obtained by the subtraction process as the target loss value of the preset deep learning model.

[0080] In the embodiment of the present disclosure, the specific implementation of channel separation for the second image is similar to the existing implementation of channel separation for a multi-channel image, and will not be described in detail here.

[0081] In the embodiment of the present disclosure, the specific implementation of calculating the self-difference value corresponding to at least one fourth pixel point is similar to the above-mentioned implementation of determining the first self-difference value corresponding to at least one first pixel point in the first image, and will not be repeated here.

[0082] In the embodiments of the present disclosure, when the first image and the second image are color images, the channels of the first image and the second image can be separated to obtain multiple channel images, and then the target loss value of the preset deep learning model can be determined according to the loss values ​​corresponding to the multiple channel images respectively, thereby improving the accuracy of the target loss value of the preset deep learning model when the first image and the second image are color images.

[0083] In an embodiment of the present disclosure, when multiple groups of preset sliding windows and preset step sizes are included, determining the first self-difference value corresponding to at least one first pixel point in the first image and the second self-difference value corresponding to at least one second pixel point in the second image can specifically include: for each group of preset sliding windows and preset step sizes, determining the first self-difference value corresponding to at least one first pixel point in the first image and the second self-difference value corresponding to at least one second pixel point in the second image.

[0084] In the embodiment of the present disclosure, the window size and / or the number of steps of each set of preset sliding windows in the multiple sets of preset sliding windows and preset step sizes are different.

[0085] Furthermore, determining the target loss value of the preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values ​​can specifically include: for each group of preset sliding windows and preset step sizes, determining the group loss value corresponding to the preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values ​​of the preset sliding windows and preset step sizes; performing weighted summation of multiple group loss values ​​to obtain the target loss value.

[0086] It should be noted that the specific implementation method of determining the group loss value corresponding to the preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values ​​of a preset sliding window and a preset step size is similar to the above-mentioned implementation method of determining the target loss value of the preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values.

[0087] Exemplarily, for each set of preset sliding windows and preset step sizes, the group loss value corresponding to the preset deep learning model is determined based on multiple first self-difference values ​​and multiple second self-difference values ​​of the preset sliding windows and preset step sizes. Specifically, it may include: adding multiple first self-difference values ​​to obtain the sum of multiple first self-difference values, and adding multiple second self-difference values ​​to obtain the sum of multiple second self-difference values, and subtracting the sum of the multiple first self-difference values ​​from the sum of the multiple second self-difference values ​​to obtain the result as the group loss value.

[0088] In the embodiment of the present disclosure, the weight corresponding to each group loss value may be preset or determined based on actual conditions.

[0089] In the disclosed embodiment, sliding operations can be performed on the first image and the second image respectively through multiple sets of preset sliding windows and preset step sizes with different control parameters to determine the group loss values ​​of the first image and the second image corresponding to each set of preset sliding windows and preset step sizes, and then determine the target loss value based on the multiple group loss values. Thus, by performing sliding operations on the first image and the second image respectively through multiple sets of preset sliding windows and preset step sizes with different control parameters, loss information of the first image and the second image at different angles can be obtained, and then the target loss value of the preset deep learning model can be determined based on the loss information at different angles. The obtained target loss value can more comprehensively reflect the difference between the first image and the second image, thereby further improving the accuracy of the target loss value.

[0090] Figure 5 It is a structural schematic diagram of a loss value calculation device provided in an embodiment of the present disclosure.

[0091] In the disclosed embodiment, the loss value calculation device can be arranged in an electronic device, and is understood as a part of the functional modules in the above electronic device. Specifically, the electronic device can be a server or a terminal, wherein the terminal specifically includes a computer or a tablet computer, etc., which is not limited here.

[0092] like Figure 5 As shown, the loss value calculation device 500 may include an image acquisition module 510 , a self-difference value determination module 520 and a loss value calculation module 530 .

[0093] The image acquisition module 510 can be used to acquire a first image and a second image, where the first image is an image output after a preset deep learning model reconstructs the second image, and the second image is a real image input into the preset deep learning model.

[0094] The self-difference value determination module 520 can be used to perform sliding operations on the first image and the second image respectively based on a preset sliding window and a preset step size to determine a first self-difference value corresponding to at least one first pixel point in the first image and a second self-difference value corresponding to at least one second pixel point in the second image.

[0095] The loss value calculation module 530 can be used to determine a target loss value of a preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values.

[0096] In the disclosed embodiment, a first image and a second image can be acquired, wherein the first image is an image output after a preset deep learning model reconstructs the second image, and the second image is a real image input to the preset deep learning model; after acquiring the first image and the second image, a sliding operation is performed on the first image and the second image based on a preset sliding window and a preset step size, respectively, to determine a first self-difference value corresponding to at least one first pixel point in the first image and a second self-difference value corresponding to at least one second pixel point in the second image, and to determine a target loss value of the preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values, thereby, the self-difference values ​​of the pixels in the first image and the second image can be calculated respectively by means of a sliding window, and a representation of the difference in the distribution of the image pixels corresponding to the first image and the second image can be obtained, and then the target loss value of the preset deep learning model is calculated based on the self-difference value, which draws on the invariance of the convolution operation and avoids the overfitting phenomenon of the preset deep learning model caused by excessive attention to the image position information and the inability to correctly grasp the granularity.

[0097] In some embodiments of the present disclosure, the first pixel point is a pixel point in the first image located at the center of the preset sliding window when the preset sliding window performs a sliding operation according to a preset step size.

[0098] The self-difference value determination module 520 may include a pixel point determination unit and a first calculation unit.

[0099] The pixel point determination unit may be configured to determine, for each first pixel point, at least one calculable pixel point corresponding to the first pixel point.

[0100] The first calculation unit may be configured to perform a first self-difference calculation on a first pixel point based on a pixel value corresponding to at least one calculable pixel point.

[0101] In some embodiments of the present disclosure, the pixel point determination unit can be specifically used to determine the target pixel position corresponding to each cell in the preset sliding window in the first image based on a preset hole coefficient; and determine at least one calculable pixel point corresponding to the first pixel point based on the positional relationship between the target pixel position and the first pixel point.

[0102] In some embodiments of the present disclosure, the first calculation unit can be specifically used to determine the number of at least one calculable pixel points, as well as the first pixel value corresponding to each calculable pixel point and the second pixel value corresponding to the first pixel point; calculate the square value of the difference between the second pixel value and each first pixel value to obtain multiple square values; and determine the ratio of the sum of the multiple square values ​​to the number of at least one calculable pixel point as the first self-difference value corresponding to the first pixel point.

[0103] In some embodiments of the present disclosure, the self-difference value determination module 520 may further include a channel separation unit and a second calculation unit.

[0104] The channel separation unit may be used to perform channel separation on the first image when the first image is a color image, so as to obtain a plurality of channel images corresponding to the first image.

[0105] The second calculation unit may be used to determine, for each channel image, third pixel points in the channel image corresponding to the at least one first pixel point respectively, and calculate a target self-difference value corresponding to the at least one third pixel point respectively.

[0106] In some embodiments of the present disclosure, the loss value calculation module 530 can be specifically used to add the target self-difference values ​​corresponding to at least one third pixel point for each channel to obtain a first value; add the first values ​​corresponding to multiple channels corresponding to the first image and divide them by the number of channels corresponding to the multiple channels to obtain a second value; and determine the target loss value based on the second value and multiple second self-difference values.

[0107] In some embodiments of the present disclosure, multiple groups of preset sliding windows and preset step sizes are included.

[0108] The self-difference value determination module 520 can also be specifically used to determine, for each set of preset sliding windows and preset step sizes, a first self-difference value corresponding to at least one first pixel point in the first image and a second self-difference value corresponding to at least one second pixel point in the second image.

[0109] In some embodiments of the present disclosure, the loss value calculation module 530 can be specifically used to determine, for each group of preset sliding windows and preset step sizes, a group loss value corresponding to a preset deep learning model based on multiple first self-difference values ​​and multiple second self-difference values ​​corresponding to the preset sliding windows and preset step sizes; and perform weighted summation on the multiple group loss values ​​to obtain a target loss value.

[0110] It should be noted that Figure 5 The loss value calculation device 500 shown can execute each step in the above method embodiment and realize each process and effect in the above method embodiment, which will not be described in detail here.

[0111] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure.

[0112] In the disclosed embodiment, Figure 6 The electronic device shown may be a server or a terminal, wherein the terminal specifically includes a computer or a tablet computer, etc., which is not limited here.

[0113] like Figure 6 As shown, the electronic device may include a processor 610 and a memory 620 storing computer program instructions.

[0114] Specifically, the processor 610 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present disclosure.

[0115] The memory 620 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 620 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 620 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 620 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 620 is a non-volatile solid-state memory. In a particular embodiment, the memory 620 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically rewritable ROM (EAROM) or a flash memory, or a combination of two or more of these.

[0116] The processor 610 reads and executes the computer program instructions stored in the memory 620 to perform the steps of the loss value calculation method provided in the embodiment of the present disclosure.

[0117] In one example, the electronic device may further include a transceiver 630 and a bus 640. Figure 6 As shown, the processor 610, the memory 620 and the transceiver 630 are connected via a bus 640 and communicate with each other.

[0118] The bus 640 includes hardware, software, or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 640 may include one or more buses.

[0119] The embodiments of the present disclosure also provide a computer-readable storage medium, which may store a computer program. When the computer program is executed by a processor, the processor implements the loss value calculation method provided by the embodiments of the present disclosure.

[0120] The above-mentioned storage medium may, for example, include a memory 620 of computer program instructions, and the above-mentioned instructions may be executed by the processor 610 of the electronic device to complete the loss value calculation method provided in the embodiment of the present disclosure. Optionally, the storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc ROM, CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0121] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for calculating loss value, characterized in that: include: Acquire a first image and a second image, wherein the first image is an image output after a preset deep learning model reconstructs the second image, and the second image is a real image input into the preset deep learning model; Based on a preset sliding window and a preset step size, a sliding operation is performed on the first image and the second image respectively to determine a first auto-difference value corresponding to at least one first pixel point in the first image and a second auto-difference value corresponding to at least one second pixel point in the second image; Determine a target loss value of the preset deep learning model based on a plurality of first self-difference values ​​and a plurality of second self-difference values; The first pixel point is a pixel point in the first image located at the center of the window of the preset sliding window during the sliding operation of the preset sliding window according to the preset step size, and determining the first self-difference value corresponding to at least one first pixel point in the first image includes: For each first pixel point, determining at least one computable pixel point corresponding to the first pixel point; Determine the number of the at least one calculable pixel point, and a first pixel value corresponding to each calculable pixel point and a second pixel value corresponding to the first pixel point; Calculate the square value of the difference between the second pixel value and each of the first pixel values ​​to obtain a plurality of square values; The ratio of the sum of the multiple square values ​​to the number of the at least one calculable pixel point is determined as a first self-difference value corresponding to the first pixel point.

2. The method according to claim 1, characterized in that The determining at least one computable pixel point corresponding to the first pixel point includes: Determine a target pixel position corresponding to each cell in the preset sliding window in the first image based on a preset hole coefficient; At least one computable pixel point corresponding to the first pixel point is determined based on a positional relationship between the target pixel position and the first pixel point.

3. The method according to claim 1, characterized in that When the first image is a color image, determining first self-difference values ​​respectively corresponding to at least one first pixel point in the first image includes: Perform channel separation on the first image to obtain a plurality of channel images corresponding to the first image; For each channel image, third pixel points in the channel image that respectively correspond to the at least one first pixel point are determined, and target self-difference values ​​that respectively correspond to the at least one third pixel point are calculated.

4. The method according to claim 3, characterized in that The determining the target loss value of the preset deep learning model based on the plurality of first self-difference values ​​and the plurality of second self-difference values ​​comprises: For each channel image, adding the target self-difference values ​​corresponding to the at least one third pixel point to obtain a first value; Adding first values ​​corresponding to a plurality of channel images corresponding to the first image respectively and dividing the sum by the number of channels corresponding to the first image to obtain a second value; The target loss value is determined based on the second numerical value and a plurality of second self-difference values.

5. The method according to claim 1, characterized in that The method comprises a plurality of sets of preset sliding windows and preset step sizes, wherein determining a first auto-difference value corresponding to at least one first pixel point in the first image and a second auto-difference value corresponding to at least one second pixel point in the second image comprises: For each set of preset sliding windows and preset step sizes, determining a first auto-difference value corresponding to at least one first pixel point in the first image and a second auto-difference value corresponding to at least one second pixel point in the second image; The determining the target loss value of the preset deep learning model based on the plurality of first self-difference values ​​and the plurality of second self-difference values ​​comprises: For each set of preset sliding windows and preset step sizes, determine a group loss value corresponding to the preset deep learning model based on a plurality of first self-difference values ​​and a plurality of second self-difference values ​​corresponding to the preset sliding windows and the preset step sizes; A weighted sum is performed on multiple group loss values ​​to obtain the target loss value.

6. A loss value calculation device, characterized in that: include: An image acquisition module, used to acquire a first image and a second image corresponding to the first image, wherein the first image is an image output after a preset deep learning model reconstructs the second image, and the second image is a real image input to the preset deep learning model; An auto-difference value determination module is used to perform a sliding operation on the first image and the second image based on a preset sliding window and a preset step size, and determine a first auto-difference value corresponding to at least one first pixel point in the first image and a second auto-difference value corresponding to at least one second pixel point in the second image; A loss value calculation module, used to determine a target loss value of the preset deep learning model based on a plurality of first self-difference values ​​and a plurality of second self-difference values; The first pixel point is a pixel point in the first image located at the center of the preset sliding window during the sliding operation of the preset sliding window according to the preset step size; The self-difference value determination module is specifically used to determine, for each first pixel point, at least one calculable pixel point corresponding to the first pixel point; determine the number of the at least one calculable pixel point, and the first pixel value corresponding to each calculable pixel point and the second pixel value corresponding to the first pixel point; Calculate the square value of the difference between the second pixel value and each of the first pixel values ​​to obtain multiple square values; and determine the ratio of the sum of the multiple square values ​​to the number of the at least one calculable pixel point as the first self-difference value corresponding to the first pixel point.

7. An electronic device, characterized in that: include: processor; A memory for storing executable instructions; Wherein, the processor is used to read the executable instructions from the memory and execute the executable instructions to implement the loss value calculation method described in any one of claims 1-5 above.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the loss value calculation method described in any one of claims 1 to 5.

Images