Model training, image enhancement method and device, electronic device and storage medium

Abstract

The model training, image enhancement method and device, electronic device and storage medium provided in the present application relate to the field of deep learning technology. In the present application, first, the first image and the second image are respectively subjected to pyramid decomposition processing to obtain a multi-level first sub-band image and a multi-level second sub-band image, wherein the first image and the second image are formed based on the same image content and have different image details. Secondly, based on the image loss value between each level of the first sub-band image and the second sub-band image and a plurality of preset weight coefficients, the total image loss value is calculated, wherein there are at least two different weight coefficients among the plurality of weight coefficients. Then, based on the total image loss value, the target neural network is subjected to parameter update processing to obtain an image enhancement model. Through the above method, the problem of poor detail enhancement effect of the image enhancement model obtained based on the existing training technology can be improved.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and more specifically, to a model training, image enhancement method and device, electronic device, and storage medium. Background Art

[0002] With the continuous development of deep learning technology, its application scope is becoming wider and wider. For example, it has been well applied in the field of image processing, such as enhancing the details of the image.

[0003] The inventors have found through research that the image enhancement model trained based on the existing neural network training technology has the problem of poor detail enhancement effect when performing detail enhancement processing on images. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a model training, image enhancement method and device, electronic device and storage medium to improve the problem of poor detail enhancement effect of the image enhancement model obtained based on the existing training technology.

[0005] To achieve the above purpose, the embodiment of the present application adopts the following technical solution:

[0006] A method for training an image enhancement model, comprising:

[0007] Performing pyramid decomposition processing on the first image and the second image respectively to obtain a multi-level first sub-band image and a multi-level second sub-band image, wherein the first image and the second image are formed based on the same image content and have different image details;

[0008] Based on the image loss value between the first sub-band image and the second sub-band image of each level and a plurality of preset weight coefficients, a total image loss value is calculated, wherein at least two different weight coefficients exist in the plurality of weight coefficients;

[0009] The target neural network is subjected to parameter update processing based on the total image loss value to obtain an image enhancement model, wherein the image enhancement model is used to enhance the details of the target image.

[0010] The present application embodiment also provides another image enhancement model training method, including:

[0011] Performing pyramid decomposition processing on the first image and the second image respectively to obtain a first sub-band image and a second sub-band image, wherein the first image and the second image are formed based on the same image content and have different image details, the first sub-band image includes a first detail sub-band image and a first approximate sub-band image, and the second sub-band image includes a second detail sub-band image and a second approximate sub-band image;

[0012] Calculating a detail loss value between the first detail subband image and the second detail subband image based on a first loss function;

[0013] Calculating an approximate loss value between the first approximate subband image and the second approximate subband image based on a second loss function, wherein the second loss function is different from the first loss function;

[0014] The target neural network is subjected to parameter update processing based on the detail loss value and the approximate loss value to obtain an image enhancement model, wherein the image enhancement model is used to enhance the details of the target image.

[0015] Based on the above, the embodiment of the present application further provides an image enhancement method, including:

[0016] Inputting the obtained target image into a preset image enhancement model, wherein the image enhancement model is trained based on the above-mentioned image enhancement model training method;

[0017] The details of the target image are enhanced by using the image enhancement model.

[0018] The present application also provides an image enhancement model training device, including:

[0019] a first image decomposition module, configured to perform pyramid decomposition processing on the first image and the second image respectively, to obtain a multi-level first sub-band image and a multi-level second sub-band image, wherein the first image and the second image are formed based on the same image content and have different image details;

[0020] an image loss calculation module, configured to calculate a total image loss value based on an image loss value between a first sub-band image and a second sub-band image at each level and a plurality of preset weight coefficients, wherein at least two different weight coefficients exist among the plurality of weight coefficients;

[0021] The first network updating module is used to perform parameter updating processing on the target neural network based on the total image loss value to obtain an image enhancement model, wherein the image enhancement model is used to enhance the details of the target image.

[0022] The present application embodiment also provides another image enhancement model training device, including:

[0023] a second image decomposition module, configured to perform pyramid decomposition processing on the first image and the second image respectively, to obtain a first sub-band image and a second sub-band image, wherein the first image and the second image are formed based on the same image content and have different image details, the first sub-band image includes a first detail sub-band image and a first approximate sub-band image, and the second sub-band image includes a second detail sub-band image and a second approximate sub-band image;

[0024] a detail loss calculation module, configured to calculate a detail loss value between the first detail sub-band image and the second detail sub-band image based on a first loss function;

[0025] an approximate loss calculation module, configured to calculate an approximate loss value between the first approximate subband image and the second approximate subband image based on a second loss function, wherein the second loss function is different from the first loss function;

[0026] The second network updating module is used to perform parameter updating processing on the target neural network based on the detail loss value and the approximate loss value to obtain an image enhancement model, wherein the image enhancement model is used to enhance the details of the target image.

[0027] On the basis of the above, the embodiment of the present application further provides an image enhancement device, including:

[0028] A target image input module, used to input the obtained target image into a preset image enhancement model, wherein the image enhancement model is trained based on the above-mentioned image enhancement model training device;

[0029] The target image enhancement module is used to enhance the details of the target image through the image enhancement model.

[0030] Based on the above, an embodiment of the present application further provides an electronic device, including:

[0031] Memory for storing computer programs;

[0032] A processor connected to the memory is used to execute a computer program stored in the memory to implement the above-mentioned image enhancement model training method, or to implement the above-mentioned image enhancement method.

[0033] On the basis of the above, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed, the above-mentioned image enhancement model training method is implemented, or the above-mentioned image enhancement method is implemented.

[0034] The model training, image enhancement method and device, electronic device and storage medium provided by the present application, when calculating the total image loss value, adopt different calculation methods (such as setting different weight coefficients or different loss functions) for different calculation objects, so as to achieve targeted image loss calculation, so that the calculated total image loss value has higher accuracy. In this way, since the calculated total image loss value has higher accuracy, when the target neural network is updated with parameters (i.e., model training) based on the total image loss value, the high-precision constraints on parameter updates can be improved, so that the obtained image enhancement model has higher image processing capabilities, that is, when the target image is enhanced in detail, the effect of detail enhancement is more significant, which can effectively improve the image quality of the target image, and thus, the problem of poor detail enhancement effect in the image enhancement model obtained based on the existing training technology can be improved, and it has high practical value.

[0035] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A structural block diagram of an electronic device provided in an embodiment of the present application.

[0037] Figure 2 A flowchart of the image enhancement model training method provided in an embodiment of the present application.

[0038] Figure 3 for Figure 2 Schematic diagram of the flow chart of the sub-steps included in step S110.

[0039] Figure 4 for Figure 2 Schematic diagram of the flow chart of the sub-steps included in step S120.

[0040] Figure 5 for Figure 4 Schematic diagram of the flow chart of the sub-steps included in step S121.

[0041] Figure 6 A schematic diagram of pixel comparison between approximate sub-band images provided in an embodiment of the present application.

[0042] Figure 7 for Figure 4 Schematic diagram of the flow chart of the sub-steps included in step S122.

[0043] Figure 8 A flowchart of another image enhancement model training method provided in an embodiment of the present application.

[0044] Fig. 9A flowchart of an image enhancement method provided in an embodiment of the present application.

[0045] Fig.10 Schematic diagram of the effect of the target image after being processed based on the existing image enhancement model.

[0046] Fig.11 It is a schematic diagram of the effect of the target image after being processed by the image enhancement model based on the present application.

[0047] Fig.12 A block diagram of the functional modules included in the first image enhancement model training device provided in an embodiment of the present application.

[0048] Fig.13 A block diagram of the functional modules included in the second image enhancement model training device provided in an embodiment of the present application.

[0049] Fig.14 A block diagram of the functional modules included in the image enhancement device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present application.

[0051] like Figure 1 As shown, an embodiment of the present application provides an electronic device 10. The electronic device 10 may include a memory 12 and a processor 14.

[0052] In detail, the memory 12 and the processor 14 are electrically connected directly or indirectly to realize data transmission or interaction. For example, they can be electrically connected to each other through one or more communication buses or signal lines. The memory 12 may store at least one software function module that can exist in the form of software or firmware. The processor 14 can be used to execute an executable computer program stored in the memory 12, such as the aforementioned software function module, so as to implement the image enhancement model training method provided in the embodiment of the present application (as described later) to train the image enhancement model, or implement the image enhancement method provided in the embodiment of the present application (as described later) to enhance the details of the target image.

[0053] Optionally, the memory 12 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0054] Furthermore, the processor 14 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0055] It can be understood that the electronic device 10 can be a server with data processing capabilities.

[0056] and, Figure 1 The structure shown is for illustration only. The electronic device 10 may also include Figure 1 More or fewer components as shown, or with Figure 1 The different configurations shown, for example, may also include a communication unit for performing information interaction with other devices, such as performing information interaction with other databases to obtain a first image, or performing information interaction with a terminal device to obtain a target image.

[0057] Combination Figure 2 The present application also provides an image enhancement model training method applicable to the electronic device 10. The method steps defined in the process related to the image enhancement model training method can be implemented by the electronic device 10. Figure 2 The specific process shown is explained in detail.

[0058] Step S110 , performing pyramid decomposition processing on the first image and the second image respectively to obtain a multi-level first sub-band image and a multi-level second sub-band image.

[0059] In this embodiment, when it is necessary to train the target neural network (parameter update processing), the electronic device 10 can, on the one hand, perform pyramid decomposition processing on the obtained first image to obtain the corresponding multi-level first sub-band image, and on the other hand, can also perform pyramid decomposition processing on the obtained second image to obtain the corresponding multi-level second sub-band image.

[0060] The first image and the second image may be formed based on the same image content and may have different image details.

[0061] Step S120, calculating a total image loss value based on the image loss value between the first sub-band image and the second sub-band image at each level and a plurality of preset weight coefficients.

[0062] In this embodiment, after obtaining multiple levels of the first sub-band image and multiple levels of the second sub-band image based on step S110, the electronic device 10 can perform weighted calculation on the image loss value between each level of the first sub-band image and the second sub-band image based on a plurality of preset weight coefficients to obtain a corresponding total image loss value.

[0063] Among the multiple weight coefficients, there are at least two different weight coefficients.

[0064] Step S130, performing parameter update processing on the target neural network based on the total image loss value to obtain an image enhancement model.

[0065] In this embodiment, after obtaining the total image loss value based on step S120, the electronic device 10 can perform parameter update processing on the target neural network based on the total image loss value to obtain a corresponding image enhancement model.

[0066] The image enhancement model can be used to enhance the details of the target image.

[0067] Based on the above method, since different weight coefficients are set when calculating the total image loss value, targeted image loss calculation can be achieved, so that the calculated total image loss value has higher accuracy, so that when the target neural network is updated with parameters based on the total image loss value (ie, model training), the high-precision constraints on parameter updates can be improved, thereby making the trained image enhancement model have higher image processing capabilities, that is, when the target image is enhanced in detail, the effect of detail enhancement is more significant, which can effectively improve the image quality of the target image.

[0068] In the first aspect, it should be noted that, with respect to step S110, specific image details in the first image and the second image are not limited and can be selected according to actual application requirements.

[0069] For example, in an alternative example, the image details may be represented by noise. That is, the first image and the second image may have different noises, such as an image with greater noise has less image details, and an image with less noise has more image details.

[0070] For another example, in an alternative example, the image details may also be represented by resolution. That is, the first image and the second image may have different resolutions, such as an image with a high resolution has more image details, and an image with a low resolution has fewer image details.

[0071] It should also be noted that for step S110, the first image and the second image are formed based on the same image content, but the specific formation method is not limited and can also be selected according to actual application requirements.

[0072] For example, in an alternative example, the first image and the second image may respectively refer to two images taken of the same subject based on the same device in different resolution modes.

[0073] For example, in another alternative example, the first image and the second image may respectively refer to two images of the same subject taken based on the same device in different resolution modes, and the image in the lower resolution mode may be processed by detail enhancement of the target neural network.

[0074] For another alternative example, the first image and the second image may refer to two images obtained by performing detail enhancement processing based on the same sample image based on other trained neural networks and a target neural network that has not yet been trained.

[0075] For another example, in order to make the image loss value calculated based on the first image and the second image more effective for training the target neural network, Figure 3 , step S110 may include step S111, step S112 and step S113, the specific contents of which are as follows.

[0076] Step S111, performing detail reduction processing on the first image to obtain a training image.

[0077] In this embodiment, when the target neural network needs to be trained, detail reduction processing may be performed on the obtained first image to obtain a corresponding training image.

[0078] That is, between the first image and the second image, the first image has more image details, and the training image has fewer image details.

[0079] Step S112: performing detail enhancement processing on the training image based on the target neural network to obtain a second image.

[0080] In this embodiment, after the training image is obtained based on step S111, detail enhancement processing can be performed on the training image based on the target neural network to obtain a corresponding second image.

[0081] That is, between the training image and the second image, the second image has more image details, and the training image has fewer image details. Moreover, since the target neural network still needs to be trained, the second image also has fewer image details than the first image.

[0082] Step S113: performing pyramid decomposition processing on the first image to obtain a multi-level first sub-band image, and performing pyramid decomposition processing on the second image to obtain a multi-level second sub-band image.

[0083] In this embodiment, after obtaining the first image, the first image may be subjected to pyramid decomposition processing to obtain a corresponding multi-level first sub-band image. Furthermore, after obtaining the second image based on step S112, the second image may be subjected to pyramid decomposition processing to obtain a corresponding multi-level second sub-band image.

[0084] Optionally, the specific manner of executing step S111 to perform detail reduction processing on the first image is not limited and can be selected according to actual application requirements.

[0085] For example, in an alternative example, the first image may be subjected to noise enhancement processing so as to reduce image details of the first image, thereby obtaining a training image.

[0086] For another alternative example, the first image may be processed by reducing the resolution so as to reduce the image details of the first image, thereby obtaining a training image.

[0087] Optionally, the specific manner of executing step S112 to perform detail enhancement processing on the training image is not limited and can be selected according to actual application requirements.

[0088] For example, in an alternative example, if the training image is obtained based on noise augmentation processing, and the target neural network (such as a denoising neural network) can be used for denoising processing, then the training image can be denoised, thereby achieving detail enhancement processing on the training image.

[0089] For example, in another alternative example, if the training image is obtained based on resolution reduction processing, and the target neural network (such as a super-resolution neural network) can be used for resolution enhancement processing, then the training image can be subjected to resolution enhancement processing, thereby achieving detail enhancement processing of the training image.

[0090] Optionally, the specific manner of executing step S113 to perform pyramid decomposition processing is not limited and can be selected according to actual application requirements.

[0091] For example, in an alternative example, the pyramid decomposition processing may be a Gaussian pyramid decomposition processing, so that by performing a Gaussian pyramid decomposition processing on the first image, a multi-level first approximate sub-band image may be obtained; and by performing a Gaussian pyramid decomposition processing on the second image, a multi-level second approximate sub-band image may be obtained.

[0092] That is, in this example, each level of the first sub-band images includes a first approximate sub-band image, and each level of the second sub-band images includes a second approximate sub-band image.

[0093] For another alternative example, the pyramid decomposition processing may be a Laplacian pyramid decomposition processing. Thus, by performing a Laplacian pyramid decomposition processing on the first image, a multi-level first detail sub-band image may be obtained; and by performing a Laplacian pyramid decomposition processing on the second image, a multi-level second detail sub-band image may be obtained.

[0094] That is, in this example, each level of the first sub-band images includes a first detail sub-band image, and each level of the second sub-band images includes a second detail sub-band image.

[0095] For another example, in another alternative example, the pyramid decomposition processing may include Laplacian pyramid decomposition processing and Gaussian pyramid decomposition processing. Thus, by performing Laplacian pyramid decomposition processing and Gaussian pyramid decomposition processing on the first image respectively, a multi-level first detail subband image and a multi-level first approximate subband image can be obtained respectively; by performing Laplacian pyramid decomposition processing and Gaussian pyramid decomposition processing on the second image respectively, a multi-level second detail subband image and a multi-level second approximate subband image can be obtained respectively.

[0096] That is, in this example, each level of the first subband images includes a first detail subband image and a first approximate subband image, and each level of the second subband images includes a second detail subband image and a second approximate subband image.

[0097] It can be understood that the Gaussian pyramid decomposition processing may mean that, through continuous blurring and downsampling, multiple levels of approximate sub-band images with less and less image details can be obtained in sequence. For example, if the number of levels of the Gaussian pyramid is 6, the approximate sub-band image of the first level has the most image details, and the approximate sub-band image of the sixth level has the least image details.

[0098] Among them, the Laplacian pyramid decomposition processing may mean that a lot of image information will be lost in the process of generating each level of the Gaussian pyramid, and this information constitutes the Laplacian pyramid. For example, an image can be obtained by upsampling the approximate subband image of the sixth level in the Gaussian pyramid, and then, the approximate subband image of the fifth level in the Gaussian pyramid is subtracted from the image to obtain the detail subband image of the sixth level.

[0099] In this way, since the Laplacian pyramid decomposition processing is performed on the basis of the Gaussian pyramid decomposition processing, therefore, in the aforementioned example, when the first subband image at each level includes the first detail subband image, and the second subband image at each level includes the second detail subband image, in fact, the Gaussian pyramid decomposition processing is also performed, but only the detail subband image obtained based on the Laplacian pyramid decomposition processing can be obtained, without obtaining the approximate subband image obtained based on the Gaussian pyramid decomposition processing.

[0100] Secondly, it should be noted that for step S120, the specific method of calculating the total image loss value is not limited and can be selected according to actual application requirements.

[0101] For example, in an alternative example, if the first subband image at each level includes a first detail subband image and a first approximate subband image, and the second subband image at each level includes a second detail subband image and a second approximate subband image, a detail weight coefficient can be set for the detail loss value between the first detail subband image and the second detail subband image, and an approximate weight coefficient can be set for the approximate loss value between the first approximate subband image and the second approximate subband image.

[0102] The detail weight coefficients corresponding to the detail loss values ​​at each level may be the same, and the approximate weight coefficients corresponding to the approximate loss values ​​at each level may be the same. Furthermore, the sum of a detail weight coefficient and an approximate weight coefficient may be 1, but they are not the same.

[0103] For example, in another alternative example, in order to make the trained image enhancement model have better detail enhancement capability, Figure 4 Step S120 may include step S121 and step S122, the specific contents of which are described as follows.

[0104] Step S121 : for each level of the first sub-band image and the second sub-band image, determine the image loss value between the first sub-band image and the second sub-band image of the level, and obtain multi-level image loss values.

[0105] In this embodiment, after obtaining multiple levels of the first sub-band image and multiple levels of the second sub-band image based on step S110, the image loss value between the first sub-band image and the second sub-band image of each level can be determined, so that the total image loss value of the multiple levels can be obtained.

[0106] Step S122, obtaining a total image loss value based on the multi-level image loss values ​​and the preset multi-level weight coefficients.

[0107] In this embodiment, after the multi-level image loss value is obtained based on step S121, the multi-level image loss value can be weighted calculated based on the preset multi-level weight coefficient to obtain the corresponding total image loss value.

[0108] Among the multiple levels of level weight coefficients, there are at least two different levels of level weight coefficients. In this way, different level weight coefficients can be set for at least two levels of image loss values ​​with different importance, so that the calculated total image loss value has higher accuracy.

[0109] Optionally, the specific method of executing step S121 to determine the image loss value is not limited and can be selected according to actual application requirements.

[0110] For example, in an alternative example, in order to make the amount of computation of the electronic device 10 relatively small in the process of determining the image loss value, each level of the first sub-band image may include a first approximate sub-band image, and each level of the second sub-band image may include a second approximate sub-band image.

[0111] Based on this, in order to determine the image loss value between the first sub-band image and the second sub-band image at each level, step S121 may include the following sub-steps:

[0112] First, for each level of first approximate sub-band image and second approximate sub-band image, an approximate loss value between the first approximate sub-band image and the second approximate sub-band image of the level can be determined. Secondly, for each level of first approximate sub-band image and second approximate sub-band image, the approximate loss value between the first approximate sub-band image and the second approximate sub-band image of the level can be used as the image loss value between the corresponding first sub-band image and the second sub-band image.

[0113] That is, in this example, the sub-band images only include approximate sub-band images, and thus, the approximate loss values ​​between the approximate sub-band images can be directly used as the image loss values ​​between the sub-band images.

[0114] For example, in another alternative example, in order to make the amount of computation of the electronic device 10 relatively small in the process of determining the image loss value and further improve the detail enhancement capability of the trained image enhancement model, each level of the first sub-band image may include a first detail sub-band image, and each level of the second sub-band image may include a second detail sub-band image.

[0115] Based on this, in order to determine the image loss value between the first sub-band image and the second sub-band image at each level, step S121 may include the following sub-steps:

[0116] Firstly, for each level of first detail subband image and second detail subband image, the detail loss value between the first detail subband image and the second detail subband image of that level can be determined; secondly, for each level of first detail subband image and second detail subband image, the detail loss value between the first detail subband image and the second detail subband image of that level can be used as the image loss value between the corresponding first subband image and the second subband image.

[0117] That is, in this example, the sub-band images only include detail sub-band images, and thus, the detail loss values ​​between the detail sub-band images can be directly used as the image loss values ​​between the sub-band images.

[0118] For another alternative example, in order to fully ensure that the trained image enhancement model has a high detail enhancement capability, each level of the first sub-band image may include a first detail sub-band image and a first approximate sub-band image, and each level of the second sub-band image may include a second detail sub-band image and a second approximate sub-band image.

[0119] Based on this, in order to determine the image loss value between the first sub-band image and the second sub-band image at each level, combined with Figure 5 Step S121 may include step S121a, step S121b and step S121c, the specific contents of which are described as follows.

[0120] Step S121a: for each level of the first detail subband image and the second detail subband image, determine the detail loss value between the first detail subband image and the second detail subband image of the level.

[0121] In this example, after obtaining multiple levels of the first detail subband image and multiple levels of the second detail subband image based on step S110, the detail loss value between the first detail subband image and the second detail subband image at each level can be determined, so that multiple levels of detail loss values ​​can be obtained.

[0122] Step S121b: for each level of the first approximate sub-band image and the second approximate sub-band image, determine an approximate loss value between the first approximate sub-band image and the second approximate sub-band image of the level.

[0123] In this example, after obtaining multiple levels of the first approximate subband images and multiple levels of the second approximate subband images based on step S110, the approximate loss value between the first approximate subband image and the second approximate subband image of each level can be determined, so that multiple levels of approximate loss values ​​can be obtained.

[0124] Step S121c: for each level of the first sub-band image and the second sub-band image, based on the detail loss value and the approximate loss value between the first sub-band image and the second sub-band image of the level, obtain the image loss value between the first sub-band image and the second sub-band image of the level.

[0125] In this embodiment, after obtaining the multiple levels of detail loss values ​​and the multiple levels of approximate loss values ​​based on step S121a and step S121c respectively, the image loss value between the first subband image and the second subband image of each level can be obtained based on the detail loss value and the approximate loss value between the first subband image and the second subband image of that level.

[0126] For example, the image loss value between the first-level subband image and the second subband image can be obtained based on the detail loss value between the first subband image and the second subband image of the first level (i.e., the detail loss value between the first detail subband image and the second detail subband image of the first level) and the approximate loss value (i.e., the detail loss value between the first approximate subband image and the second approximate subband image of the first level).

[0127] Regarding step S121a and step S121b, it can be understood that the specific method of determining the detail loss value and the approximate loss value is not limited and can be selected according to actual needs.

[0128] For example, in an alternative example, the detail loss value between the detail sub-band images and the approximation loss value between the approximation sub-band images may be determined based on the same loss function.

[0129] For another alternative example, after long-term research by the inventors of the present application, it was found that the difference between detail sub-band images and the difference between approximate sub-band images represent different contents. Therefore, in order to make the determined detail loss value and approximate loss value better reflect the difference between the corresponding sub-band images, the detail loss value between the detail sub-band images and the approximate loss value between the approximate sub-band images can be determined based on different loss functions, respectively.

[0130] That is to say, on the one hand, when executing step S121a, the detail loss value between the first detail subband image and the second detail subband image of each level can be calculated based on the first loss function.

[0131] On the other hand, when executing step S121b, the approximate loss value between the first approximate subband image and the second approximate subband image of each level can be calculated based on a second loss function different from the first loss function.

[0132] The specific contents of the first loss function and the second loss function are not limited and can be selected according to actual application requirements.

[0133] For example, in an alternative example, the first loss function may be a minimum absolute deviation loss function, and the second loss function may be a minimum square error loss function.

[0134] For another alternative example, after further research by the inventors of this application, it was found that since the trained neural network model is used to pre-enhance the image details of the target image, and the minimum square error loss function has a better ability to capture the differences between sub-band images, the first loss function can be configured as a minimum square error loss function, and the second loss function can be configured as a minimum absolute deviation loss function.

[0135] That is, the detail loss value between detail subband images can be calculated based on the following formula:

[0136]

[0137] The approximation loss value between the approximate subband images can be calculated based on the following formula:

[0138]

[0139] In the above two formulas, S1 is the detail loss value, S2 is the approximate loss value, j1 is the target pixel in the detail subband image, j2 is the target pixel in the approximate subband image, n1 is the total number of pixels in the detail subband image, n2 is the total number of pixels in the approximate subband image, and Y j1 is the pixel value of the target pixel in the first detail subband image, Y j2 is the pixel value of the target pixel in the first approximate subband image, f(x j1 ) is the pixel value of the target pixel in the second detail subband image, f(x j2 ) is the pixel value of the target pixel in the second approximate subband image.

[0140] Combination Figure 6 , taking the first approximate subband image and the second approximate subband image including corresponding 4 pixel points as an example, wherein the pixel value of pixel point A1 can be 111, the pixel value of pixel point A2 can be 171, the pixel value of pixel point A3 can be 98, the pixel value of pixel point A4 can be 100, the pixel value of pixel point B1 can be 101, the pixel value of pixel point B2 can be 151, the pixel value of pixel point B3 can be 91, and the pixel value of pixel point B4 can be 87.

[0141] Based on this, the calculated approximate loss value S2 can be (111-101)+(171-151)+(98-91)+(100-87)=50.

[0142] Furthermore, it can be understood that for step S121c, the specific method of calculating the image loss value based on the detail loss value and the approximate loss value is not limited and can be selected according to actual needs.

[0143] For example, in an alternative example, for each level of the first subband image and the second subband image, the detail loss value and the approximate loss value between the first subband image and the second subband image of that level can be directly added, and the obtained sum can be used as the image loss value.

[0144] For another example, in another alternative example, considering that the importance of the detail loss value and the approximate loss value may be different based on different requirements, therefore, in order to meet different application requirements, step S121c may include the following sub-steps:

[0145] First, for the detail loss value and approximate loss value between the first sub-band image and the second sub-band image of each level, the corresponding detail weight coefficient and approximate weight coefficient can be obtained respectively; secondly, for the first sub-band image and the second sub-band image of each level, based on the corresponding detail loss value, approximate loss, detail weight coefficient and approximate weight coefficient, a weighted calculation can be performed to obtain the image loss value between the first sub-band image and the second sub-band image of that level.

[0146] That is to say, for the first sub-band image and the second sub-band image of the first level, the corresponding detail loss value and the detail weight coefficient can be multiplied to obtain a first product, and the corresponding approximate loss value and the approximate weight coefficient can be multiplied to obtain a second product; then, the first product and the second product can be added, and the obtained sum can be used as the image loss value.

[0147] Among them, the detail loss coefficient and the approximate weight coefficient are used to characterize the relative importance relationship between the detail loss value and the approximate loss value, and have no direct relationship with the pyramid levels to which they belong. Therefore, the detail weight coefficient corresponding to the detail loss value at each level can be the same, and the approximate weight coefficient corresponding to the approximate loss value at each level can also be the same.

[0148] Furthermore, the sum of a detail weight coefficient and an approximate weight coefficient is 1.

[0149] Moreover, with regard to the setting of specific numerical values ​​of the detail loss coefficient and the approximate loss coefficient, after long-term research by the inventors of the present application, it was found that the larger the numerical value of the detail loss coefficient is, the stronger the detail enhancement capability of the trained image enhancement model will be.

[0150] For example, if the image enhancement model is used to perform denoising on the target image, a detail loss coefficient with a larger value may be set so that the target image retains more image details after the denoising.

[0151] Optionally, the specific method of executing step S122 to calculate the total image loss value is not limited and can be selected according to actual application requirements.

[0152] For example, in an alternative example, among the multiple levels of image loss values, a same level weight coefficient may be set for a more important level, and another same level weight coefficient may be set for a less important level.

[0153] In detail, in a specific application example, if the number of pyramid levels is 6, then a same level weight coefficient may be set for the first three levels, and another same level weight coefficient may be set for the last three levels.

[0154] For another alternative example, in the multiple levels of image loss values, a level weight coefficient may be set for the most important level, and another same level weight coefficient may be set for other levels.

[0155] In detail, in a specific application example, if the pyramid level is 6, then a level weight coefficient may be set for the first level, and another same level weight coefficient may be set for the other 5 levels.

[0156] For another example, in order to fully ensure that the calculated total image loss value has a high accuracy so that the total image loss value can reflect the importance between different pyramid levels, Figure 7 , step S122 may include step S121a and step S122b, the specific contents of which are described as follows.

[0157] Step S122a, for each level of the image loss value, obtain the level weight coefficient corresponding to the image loss value of that level.

[0158] In this embodiment, after obtaining the multi-level image loss values ​​based on step S121, the level weight coefficient corresponding to each level of the image loss value can be obtained to obtain the multi-level level weight coefficients.

[0159] Among them, the level weight coefficient corresponding to the image loss value at each level may be different.

[0160] Step S122b, performing weighted summation calculation based on the multi-level weight coefficients and the multi-level image loss values ​​to obtain the total image loss value.

[0161] In this embodiment, after obtaining the multi-level weight coefficients based on step S122a, the multi-level image loss values ​​can be weighted summed up and processed based on the level weight coefficients to obtain the corresponding total image loss value.

[0162] That is to say, for each level of image loss value, the image loss value of that level and the corresponding level weight coefficient can be multiplied to obtain the corresponding product, so that multiple products can be obtained. Then, the multiple products are added together, and the sum obtained can be used as the total image loss value.

[0163] In the above example, for step S122a, it can be understood that the specific method of obtaining multiple levels of different hierarchical weight coefficients is not limited and can be selected according to actual application requirements.

[0164] For example, in an alternative example, multiple level weight coefficients can be set, and the multiple level weight coefficients can be randomly set as long as the values ​​are different. In this way, according to the importance of the image loss values ​​of different levels, the corresponding allocation is made according to the size of the weight coefficients of each level, such as configuring the level weight coefficient with the largest value as the weight coefficient of the most important image loss value.

[0165] For another example, in order to make the total image loss value accurately reflect the specific importance of different pyramid levels, that is, to ensure that the specific value of the level weight coefficient can represent the specific importance of the corresponding image loss value, and after long-term research by the inventors of the present application, it is found that the specific importance of image loss values ​​of different pyramid levels has a certain regularity, which can be represented by a specific functional relationship. Therefore, step S122a may include the following sub-steps:

[0166] First, for each level of image loss value, the pyramid level to which the sub-band image corresponding to the image loss value of this level belongs can be obtained; secondly, for each level of image loss value, the level weight coefficient of the image loss value of this level can be determined based on the pyramid level corresponding to the image loss value of this level and a preset corresponding function relationship.

[0167] Among them, in the corresponding function relationship, the level weight coefficient decreases as the pyramid level increases. That is, the higher the pyramid level, the lower the importance of the corresponding image loss value; the lower the pyramid level, the higher the importance of the corresponding image loss value.

[0168] In addition, the inventors of the present application have conducted long-term research on the specific degree of loss of image detail information between different pyramid levels during the pyramid decomposition process, and provided the following corresponding function relationship to accurately characterize the importance between image loss values ​​at different pyramid levels:

[0169] WO=21-i;

[0170] In the above corresponding function relationship, WO can be the level weight coefficient, and i can be the pyramid level. That is, the level weight coefficient of the first layer can be 1, the level weight coefficient of the second layer can be 0.5, the level weight coefficient of the third layer can be 0.25, ..., and the level weight coefficient of the sixth layer can be 0.03125.

[0171] Thirdly, it should be noted that for step S130, the specific method of performing parameter update processing (training) on ​​the target neural network is not limited and can be selected according to actual needs.

[0172] For example, training can be performed based on an existing neural network training method in combination with the calculated total image loss value, which will not be described in detail here.

[0173] It is understood that the specific number of times step S130 is executed is not limited. For example, in order to ensure that the final image enhancement model can have a good detail enhancement capability, step S130 can be executed multiple times to perform multiple update processes (so that in two adjacent updates, the image enhancement model obtained based on the previous update process can be used as the update object of the next update process, that is, the target neural network), until the final total image loss value is reduced to a preset value (that is, convergence).

[0174] Combination Figure 8 Based on the same technical concept as the aforementioned image model enhancement training method, different calculation methods are used for different calculation objects when calculating the total image loss value. The embodiment of the present application also provides another image enhancement model training method applicable to the above electronic device 10. Among them, the method steps defined in the process related to the image enhancement model training method can be implemented by the electronic device 10. Figure 8 The specific process shown is explained in detail.

[0175] Step S210 , performing pyramid decomposition processing on the first image and the second image respectively to obtain a first sub-band image and a second sub-band image.

[0176] In this embodiment, when it is necessary to train the target neural network (parameter update processing), the electronic device 10 can, on the one hand, perform pyramid decomposition processing on the obtained first image to obtain the corresponding first sub-band image, and on the other hand, can also perform pyramid decomposition processing on the obtained second image to obtain the corresponding second sub-band image.

[0177] The first image and the second image may be formed based on the same image content and may have different image details, and the first subband image may include a first detail subband image and a first approximate subband image, and the second subband image may include a second detail subband image and a second approximate subband image.

[0178] Step S220: calculating a detail loss value between the first detail sub-band image and the second detail sub-band image based on a first loss function.

[0179] In this embodiment, after obtaining the first detail subband image and the second detail subband image based on step S210, the electronic device 10 may calculate the detail loss value between the first detail subband image and the second detail subband image based on a first loss function.

[0180] Step S230: calculating an approximate loss value between the first approximate sub-band image and the second approximate sub-band image based on a second loss function.

[0181] In this embodiment, after obtaining the first approximate subband image and the second approximate subband image based on step S210, the electronic device 10 may calculate a detail loss value between the first approximate subband image and the second approximate subband image based on a second loss function.

[0182] The second loss function may be different from the first loss function.

[0183] Step S240, performing parameter update processing on the target neural network based on the detail loss value and the approximate loss value to obtain an image enhancement model.

[0184] In this embodiment, after obtaining the detail loss value and the approximate loss value based on step S220 and step S230 respectively, the electronic device 10 can perform parameter update processing on the target neural network based on the detail loss value and the approximate loss value to obtain an image enhancement model.

[0185] The image enhancement model can be used to enhance the details of the target image.

[0186] Based on the above method, since the detail loss value and the approximate loss value represent different contents, by setting different first loss functions and second loss functions for calculation, the obtained detail loss value and approximate loss value can be more accurate, so that when the target neural network is updated with parameters based on the detail loss value and the approximate loss value (i.e., model training), the high-precision constraints on parameter updates can be improved, thereby making the trained image enhancement model have higher image processing capabilities, that is, when the target image is enhanced in detail, the effect of detail enhancement is more significant, which can effectively improve the image quality of the target image.

[0187] For the specific processing process of step S210 to step S240, reference may be made to the related explanations of step S110 to step S130 in the previous text, which will not be described in detail here.

[0188] It should be noted that when executing step S240, the sum of the detail loss value and the approximate loss value can be calculated first, that is, the total image loss value can be obtained, and then the parameters of the target neural network can be updated based on the total image loss value to obtain an image enhancement model.

[0189] Combination Fig. 9 The present application also provides an image enhancement method applicable to the electronic device 10. The method steps defined in the process related to the image enhancement method can be implemented by the electronic device 10. Fig. 9 The specific process shown is explained in detail.

[0190] Step S310: input the obtained target image into a preset image enhancement model.

[0191] In this embodiment, when detail enhancement processing needs to be performed on the target image, the electronic device 10 may input the obtained target image into a preset image enhancement model.

[0192] Among them, the image enhancement model can be trained based on the above-mentioned image enhancement model training method (such as the image enhancement model training method corresponding to steps S110 to S130, or the image enhancement model training method corresponding to steps S210 to S240).

[0193] Step S320: enhancing the details of the target image using the image enhancement model.

[0194] In this embodiment, after the target image is input into the image enhancement model based on step S310, the electronic device 10 may enhance the details of the target image through the image enhancement model.

[0195] Based on the above method, since the image enhancement model is trained based on the aforementioned image enhancement model training method, it has a higher detail enhancement capability, so that the image details of the target image can be effectively improved, ensuring that an image with higher image quality can be obtained.

[0196] In detail, Fig.10 As shown in , it is an image after detail enhancement processing of the target image based on the existing image enhancement model; Fig.11 As shown, it is an image after the target image is enhanced by the image enhancement model improved based on the embodiment of the present application. It can be clearly seen that in the hair part and the eyelash part, the latter has more image details (higher clarity).

[0197] Combination Fig.12 The embodiment of the present application further provides a first image enhancement model training device 100, which can be applied to the above-mentioned electronic device 10. The first image enhancement model training device 100 may include a first image decomposition module 110, an image loss calculation module 120 and a first network update module 130.

[0198] The first image decomposition module 110 may be used to perform pyramid decomposition processing on the first image and the second image respectively to obtain a multi-level first sub-band image and a multi-level second sub-band image, wherein the first image and the second image are formed based on the same image content and have different image details. In this embodiment, the first image decomposition module 110 may be used to perform Figure 2 As shown in step S110, the relevant contents of the first image decomposition module 110 can refer to the above description of step S110.

[0199] The image loss calculation module 120 is used to calculate the total image loss value based on the image loss value between the first sub-band image and the second sub-band image of each level and a plurality of preset weight coefficients, wherein at least two different weight coefficients exist in the plurality of weight coefficients. In this embodiment, the image loss calculation module 120 can be used to perform Figure 2 As shown in step S120, the relevant contents of the image loss calculation module 120 can refer to the above description of step S120.

[0200] The first network update module 130 can be used to perform parameter update processing on the target neural network based on the total image loss value to obtain an image enhancement model, wherein the image enhancement model is used to enhance the details of the target image. In this embodiment, the first network update module 130 can be used to perform Figure 2As shown in step S130, the relevant contents about the first network update module 130 can refer to the above description of step S130.

[0201] Combination Fig.13 The embodiment of the present application further provides a second image enhancement model training device 200, which can be applied to the above-mentioned electronic device 10. The second image enhancement model training device 200 may include a second image decomposition module 210, a detail loss calculation module 220, an approximate loss calculation module 230 and a second network update module 240.

[0202] The second image decomposition module 210 may be used to perform pyramid decomposition processing on the first image and the second image respectively to obtain a first sub-band image and a second sub-band image, wherein the first image and the second image are formed based on the same image content and have different image details, the first sub-band image includes a first detail sub-band image and a first approximate sub-band image, and the second sub-band image includes a second detail sub-band image and a second approximate sub-band image. In this embodiment, the second image decomposition module 210 may be used to perform Figure 8 As shown in step S210, the relevant contents about the second image decomposition module 210 can refer to the above description of step S210.

[0203] The detail loss calculation module 220 may be used to calculate the detail loss value between the first detail sub-band image and the second detail sub-band image based on the first loss function. In this embodiment, the detail loss calculation module 220 may be used to perform Figure 8 As shown in step S220, the relevant contents of the detail loss calculation module 220 can refer to the above description of step S220.

[0204] The approximate loss calculation module 230 may be used to calculate the approximate loss value between the first approximate sub-band image and the second approximate sub-band image based on a second loss function, wherein the second loss function is different from the first loss function. In this embodiment, the approximate loss calculation module 230 may be used to perform Figure 8 As shown in step S230, the relevant contents of the approximate loss calculation module 230 can refer to the above description of step S230.

[0205] The second network updating module 240 can be used to perform parameter updating processing on the target neural network based on the detail loss value and the approximate loss value to obtain an image enhancement model, wherein the image enhancement model is used to enhance the details of the target image. In this embodiment, the second network updating module 240 can be used to perform Figure 8As shown in step S240, the relevant contents about the second network update module 240 can refer to the above description of step S240.

[0206] Combination Fig.14 The embodiment of the present application further provides an image enhancement device 300, which can be applied to the above-mentioned electronic device 10. The image enhancement device 300 may include a target image input module 310 and a target image enhancement module 320.

[0207] The target image input module 310 can be used to input the obtained target image into a preset image enhancement model, wherein the image enhancement model is trained based on the above-mentioned image enhancement model training device (such as the above-mentioned first image enhancement model training device 100, or the second image enhancement model training device 200). In this embodiment, the target image input module 310 can be used to perform Fig. 9 As shown in step S310, the relevant contents of the target image input module 310 can refer to the above description of step S310.

[0208] The target image enhancement module 320 can be used to enhance the details of the target image through the image enhancement model. In this embodiment, the target image enhancement module 320 can be used to perform Fig. 9 As shown in step S320, the relevant contents of the target image enhancement module 320 can refer to the above description of step S320.

[0209] In an embodiment of the present application, corresponding to the above-mentioned image enhancement model training method, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is run, each step of the above-mentioned image enhancement model training method is executed.

[0210] Among them, the steps executed when the aforementioned computer program is running will not be described one by one here, and reference can be made to the explanation of the image enhancement model training method in the previous article.

[0211] Furthermore, in an embodiment of the present application, corresponding to the above-mentioned image enhancement method, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is run, each step of the above-mentioned image enhancement method is executed.

[0212] Among them, the steps executed when the aforementioned computer program is running will not be described one by one here, and reference may be made to the explanation of the aforementioned image enhancement method in the previous text.

[0213] It should be noted that, in the above examples, multiple means two or more.

[0214] In summary, the model training, image enhancement method and device, electronic device and storage medium provided by the present application, when calculating the total image loss value, adopt different calculation methods (such as setting different weight coefficients or different loss functions) for different calculation objects, which can realize targeted image loss calculation, so that the calculated total image loss value has higher accuracy. In this way, since the calculated total image loss value has higher accuracy, when the target neural network is updated with parameters (i.e., model training) based on the total image loss value, the high-precision constraints on parameter updates can be improved, so that the obtained image enhancement model has higher image processing capabilities, that is, when the target image is enhanced in detail, the effect of detail enhancement is more significant, which can effectively improve the image quality of the target image. Therefore, the problem of poor detail enhancement effect in the image enhancement model obtained based on the existing training technology can be improved, and it has high practical value.

[0215] In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device and method embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0216] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0217] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk. It should be noted that, in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0218] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training an image enhancement model, characterized in that: include: Performing pyramid decomposition processing on the first image and the second image respectively to obtain a multi-level first sub-band image and a multi-level second sub-band image, wherein the first image and the second image are formed based on the same image content and have different image details; Based on the image loss value between the first sub-band image and the second sub-band image of each level and a plurality of preset weight coefficients, a total image loss value is calculated, wherein at least two different weight coefficients exist in the plurality of weight coefficients; Performing parameter updating processing on the target neural network based on the total image loss value to obtain an image enhancement model, wherein the image enhancement model is used to enhance the details of the target image; The step of calculating the total image loss value based on the image loss value between the first sub-band image and the second sub-band image at each level and a plurality of preset weight coefficients includes: For each level of the first sub-band image and the second sub-band image, determine an image loss value between the first sub-band image and the second sub-band image of the level, and obtain a multi-level image loss value; Based on the multi-level image loss values ​​and the preset multi-level hierarchical weight coefficients, a total image loss value is obtained, wherein there are at least two levels of different hierarchical weight coefficients in the multi-level hierarchical weight coefficients.

2. The image enhancement model training method according to claim 1, characterized in that: The step of obtaining the total image loss value based on the multi-level image loss values ​​and the preset multi-level weight coefficients includes: For each level of the image loss value, obtaining a level weight coefficient corresponding to the image loss value of the level, wherein the level weight coefficient corresponding to each level of the image loss value is different; Based on the multi-level weight coefficients and the multi-level image loss values, a weighted sum calculation process is performed to obtain a total image loss value.

3. The image enhancement model training method according to claim 2, characterized in that: The step of obtaining the level weight coefficient corresponding to the image loss value of each level comprises: For each level of the image loss value, obtaining the pyramid level to which the sub-band image corresponding to the image loss value of the level belongs; For each level of the image loss value, a level weight coefficient of the image loss value of this level is determined based on the pyramid level corresponding to the image loss value of this level and a preset corresponding functional relationship, wherein in the corresponding functional relationship, the level weight coefficient decreases as the pyramid level increases.

4. The image enhancement model training method according to claim 3, characterized in that: The corresponding functional relationship includes: WO=21-i; Wherein, WO is the level weight coefficient, and i is the pyramid level.

5. The image enhancement model training method according to any one of claims 1 to 4, characterized in that: The first sub-band images at each level include a first detail sub-band image and a first approximate sub-band image, and the second sub-band images at each level include a second detail sub-band image and a second approximate sub-band image; The step of determining the image loss value between the first sub-band image and the second sub-band image of each level comprises: For each level of first detail subband image and second detail subband image, determining a detail loss value between the first detail subband image and the second detail subband image of the level, wherein the first detail subband image and the second detail subband image are obtained based on performing Laplacian pyramid decomposition processing on the first image and the second image respectively; For each level of first approximate subband image and second approximate subband image, determine an approximate loss value between the first approximate subband image and the second approximate subband image of the level, wherein the first approximate subband image and the second approximate subband image are obtained based on performing Gaussian pyramid decomposition processing on the first image and the second image respectively; For each level of the first subband image and the second subband image, an image loss value between the first subband image and the second subband image of the level is obtained based on the detail loss value and the approximate loss value between the first subband image and the second subband image of the level.

6. The image enhancement model training method according to claim 5, characterized in that: The step of obtaining, for each level of the first sub-band image and the second sub-band image, an image loss value between the first sub-band image and the second sub-band image of the level based on the detail loss value and the approximate loss value between the first sub-band image and the second sub-band image of the level comprises: For the detail loss value and the approximate loss value between the first sub-band image and the second sub-band image at each level, respectively obtain the corresponding detail weight coefficient and the approximate weight coefficient; For each level of the first subband image and the second subband image, based on the corresponding detail loss value, approximate loss, detail weight coefficient, and approximate weight coefficient, a weighted calculation is performed to obtain the image loss value between the first subband image and the second subband image of the level.

7. The image enhancement model training method according to claim 5, characterized in that: The detail weight coefficients corresponding to the detail loss values ​​at each level are the same, and the approximate weight coefficients corresponding to the approximate loss values ​​at each level are the same; Among them, the sum of a detail weight coefficient and an approximate weight coefficient is 1.

8. The image enhancement model training method according to claim 5, characterized in that: The step of determining the detail loss value between the first detail subband image and the second detail subband image of each level comprises: For each level of the first detail subband image and the second detail subband image, based on the first loss function, calculating the detail loss value between the first detail subband image and the second detail subband image of the level; The step of determining the approximate loss value between the first approximate subband image and the second approximate subband image of each level comprises: For each level of the first approximate subband image and the second approximate subband image, based on the second loss function, calculate an approximate loss value between the first approximate subband image and the second approximate subband image of the level; The first loss function is different from the second loss function.

9. The image enhancement model training method according to claim 8, characterized in that: The first loss function is a minimum square error loss function, and the second loss function is a minimum absolute deviation loss function.

10. The image enhancement model training method according to any one of claims 1 to 4, characterized in that: The first sub-band images at each level include first detail sub-band images, and the second sub-band images at each level include second detail sub-band images; The step of determining the image loss value between the first sub-band image and the second sub-band image of each level comprises: For each level of first detail subband image and second detail subband image, determining a detail loss value between the first detail subband image and the second detail subband image of the level, wherein the first detail subband image and the second detail subband image are obtained based on performing Laplacian pyramid decomposition processing on the first image and the second image respectively; For each level of the first detail subband image and the second detail subband image, the detail loss value between the first detail subband image and the second detail subband image of the level is used as the image loss value between the corresponding first subband image and the second subband image.

11. The image enhancement model training method according to any one of claims 1 to 4, characterized in that: The first sub-band image of each level includes a first approximate sub-band image, and the second sub-band image of each level includes a second approximate sub-band image; The step of determining the image loss value between the first sub-band image and the second sub-band image of each level comprises: For each level of first approximate subband image and second approximate subband image, determine an approximate loss value between the first approximate subband image and the second approximate subband image of the level, wherein the first approximate subband image and the second approximate subband image are obtained based on performing Gaussian pyramid decomposition processing on the first image and the second image respectively; For each level of the first approximate subband image and the second approximate subband image, the approximate loss value between the first approximate subband image and the second approximate subband image of the level is used as the image loss value between the corresponding first subband image and the second subband image.

12. The image enhancement model training method according to any one of claims 1 to 4, characterized in that: The step of performing pyramid decomposition processing on the first image and the second image respectively to obtain a multi-level first sub-band image and a multi-level second sub-band image comprises: Performing detail reduction processing on the first image to obtain a training image; Performing detail enhancement processing on the training image based on the target neural network to obtain a second image; The first image is subjected to pyramid decomposition processing to obtain a multi-level first sub-band image, and the second image is subjected to pyramid decomposition processing to obtain a multi-level second sub-band image.

13. A method for training an image enhancement model, characterized in that: include: Performing pyramid decomposition processing on the first image and the second image respectively to obtain a first sub-band image and a second sub-band image, wherein the first image and the second image are formed based on the same image content and have different image details, the first sub-band image includes a first detail sub-band image and a first approximate sub-band image, and the second sub-band image includes a second detail sub-band image and a second approximate sub-band image; Calculating a detail loss value between the first detail subband image and the second detail subband image based on a first loss function; Calculating an approximate loss value between the first approximate subband image and the second approximate subband image based on a second loss function, wherein the second loss function is different from the first loss function; The target neural network is subjected to parameter update processing based on the detail loss value and the approximate loss value to obtain an image enhancement model, wherein the image enhancement model is used to enhance the details of the target image.

14. An image enhancement method, characterized in that: include: Inputting the obtained target image into a preset image enhancement model, wherein the image enhancement model is trained based on the image enhancement model training method according to any one of claims 1 to 13; The details of the target image are enhanced by using the image enhancement model.

15. An image enhancement model training device, characterized in that: include: a first image decomposition module, configured to perform pyramid decomposition processing on the first image and the second image respectively, to obtain a multi-level first sub-band image and a multi-level second sub-band image, wherein the first image and the second image are formed based on the same image content and have different image details; an image loss calculation module, configured to calculate a total image loss value based on an image loss value between a first sub-band image and a second sub-band image at each level and a plurality of preset weight coefficients, wherein at least two different weight coefficients exist among the plurality of weight coefficients; A first network updating module is used to perform parameter updating processing on a target neural network based on the total image loss value to obtain an image enhancement model, wherein the image enhancement model is used to enhance the details of the target image; The step of calculating the total image loss value based on the image loss value between the first sub-band image and the second sub-band image at each level and a plurality of preset weight coefficients includes: For each level of the first sub-band image and the second sub-band image, determine an image loss value between the first sub-band image and the second sub-band image of the level, and obtain a multi-level image loss value; Based on the multi-level image loss values ​​and the preset multi-level hierarchical weight coefficients, a total image loss value is obtained, wherein there are at least two levels of different hierarchical weight coefficients in the multi-level hierarchical weight coefficients.

16. An image enhancement model training device, characterized in that: include: a second image decomposition module, configured to perform pyramid decomposition processing on the first image and the second image respectively, to obtain a first sub-band image and a second sub-band image, wherein the first image and the second image are formed based on the same image content and have different image details, the first sub-band image includes a first detail sub-band image and a first approximate sub-band image, and the second sub-band image includes a second detail sub-band image and a second approximate sub-band image; a detail loss calculation module, configured to calculate a detail loss value between the first detail sub-band image and the second detail sub-band image based on a first loss function; an approximate loss calculation module, configured to calculate an approximate loss value between the first approximate subband image and the second approximate subband image based on a second loss function, wherein the second loss function is different from the first loss function; The second network updating module is used to perform parameter updating processing on the target neural network based on the detail loss value and the approximate loss value to obtain an image enhancement model, wherein the image enhancement model is used to enhance the details of the target image.

17. An image enhancement device, characterized in that: include: A target image input module, used to input the obtained target image into a preset image enhancement model, wherein the image enhancement model is trained based on the image enhancement model training device according to claim 15 or 16; The target image enhancement module is used to enhance the details of the target image through the image enhancement model.

18. An electronic device, characterized in that: include: Memory for storing computer programs; A processor connected to the memory is used to execute a computer program stored in the memory to implement the image enhancement model training method described in any one of claims 1 to 13, or to implement the image enhancement method described in claim 14.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, it implements the image enhancement model training method described in any one of claims 1 to 13, or implements the image enhancement method described in claim 14.

Images