Image Processing Method, Apparatus, Electronic Device, and Computer Readable Storage Medium

Abstract

The present application relates to an image processing method, apparatus, electronic device, and storage medium. The method includes: obtaining a loss image after being processed by a specified task, and obtaining a label image corresponding to the loss image; the color space types of the loss image and the label image are both YUV; based on the loss image and the label image, respectively performing a first loss calculation on the loss Y-channel map, loss U-channel map, and loss V-channel map corresponding to the loss image to obtain a first loss value of the loss image; the first loss value is used to train the specified task model to retain the low-frequency information of the U-channel map and the V-channel map when performing the specified task; based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image, performing a second loss calculation on the loss Y-channel to obtain a second loss value of the loss Y-channel map; based on the first loss value and the second loss value, determining a target loss value of the loss image. By using this method, a more accurate loss value can be determined.

Description

Technical Field

[0001] This application relates to the field of computer technologies, and particularly to an image processing method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] With the development of computer technologies, more and more electronic devices are equipped with image sensors, and images can be captured through the image sensors. Generally, a large amount of noise remains in the images, affecting the overall visual perception of the images, so image denoising is required.

[0003] In traditional denoising methods, a denoising network is usually trained based on a loss value, so that the image can be denoised to obtain a denoised image. However, in traditional image processing methods, there is a problem that the determined loss value is inaccurate. Summary of the Invention

[0004] Embodiments of this application provide an image processing method, apparatus, electronic device, and computer-readable storage medium, which can determine a more accurate loss value, thereby improving the accuracy of image processing.

[0005] An image processing method includes:

[0006] Obtaining a loss image after being processed by a specified task, and obtaining a label image corresponding to the loss image; both the loss image and the label image are in the YUV color space type;

[0007] Based on the loss image and the label image, respectively performing a first loss calculation on a loss Y-channel map, a loss U-channel map, and a loss V-channel map corresponding to the loss image to obtain a first loss value of the loss image; the first loss value is used to train a specified task model to retain low-frequency information of the U-channel map and the V-channel map when performing the specified task processing;

[0008] Based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image, performing a second loss calculation on the loss Y-channel to obtain a second loss value of the loss Y-channel map;

[0009] Based on the first loss value and the second loss value, determining a target loss value of the loss image.

[0010] An image processing apparatus includes:

[0011] An obtaining module, configured to obtain a loss image after being processed by a specified task, and obtain a label image corresponding to the loss image; both the loss image and the label image are in the YUV color space type;

[0012] A loss calculation module, configured to perform a first loss calculation on a loss Y-channel map, a loss U-channel map, and a loss V-channel map corresponding to the loss image respectively based on the loss image and the label image, so as to obtain a first loss value of the loss image; the first loss value is used to train a specified task model to retain low-frequency information of the U-channel map and the V-channel map when performing a specified task process;

[0013] The loss calculation module is further configured to perform a second loss calculation on the loss Y-channel based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image, so as to obtain a second loss value of the loss Y-channel map;

[0014] The loss calculation module is further configured to determine a target loss value of the loss image based on the first loss value and the second loss value.

[0015] An electronic device includes a memory and a processor. A computer program is stored in the memory. When the computer program is executed by the processor, the processor is caused to execute the steps of the above-mentioned image processing method.

[0016] A computer-readable storage medium has a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned method are implemented.

[0017] For the above-mentioned image processing method, apparatus, electronic device, and computer-readable storage medium, a first loss calculation is performed on a loss Y-channel map, a loss U-channel map, and a loss V-channel map corresponding to a loss image to obtain a first loss value of the loss image, and a second loss calculation is performed on the loss Y-channel to obtain a second loss value of the loss Y-channel map, so that a target loss value of the loss image can be determined based on the first loss value and the second loss value. That is to say, the target loss value not only has the second loss value for the loss Y-channel map, and the second loss value can be used to train a specified task model to more accurately reduce the luminance noise in the Y-channel map, but also has the first loss value calculated from the loss Y-channel map, the loss U-channel map, and the loss V-channel map of the loss image. The first loss value can be used to train a specified task model to retain low-frequency information of the U-channel map and the V-channel map when performing a specified task process, so as to remove high-frequency noise in the U-channel map and the V-channel map. Therefore, a more accurate target loss value can be determined. Further, a more accurate specified task model can be trained based on the more accurate target loss value. Description of the Drawings

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is an application environment diagram of the image processing method in an embodiment;

[0020] Figure 2 It is a flowchart of the image processing method in an embodiment;

[0021] Figure 3 It is a comparison diagram of the image before and after low-pass filtering in an embodiment;

[0022] Figure 4 It is a flowchart of the image processing method in another embodiment;

[0023] Figure 5 It is a structural block diagram of the image processing device in an embodiment;

[0024] Figure 6 It is an internal structure schematic diagram of an electronic device in an embodiment. Detailed implementation manners

[0025] In order to make the purpose, technical solutions and advantages of the present application clearer, the following further details the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0026] It can be understood that the terms "first", "second", etc. used in the present application can be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element. For example, without departing from the scope of the present application, the first loss value can be called the second loss value, and similarly, the second loss value can be called the first loss value. Both the first loss value and the second loss value are loss values, but they are not the same loss value.

[0027] Figure 1 It is a schematic diagram of the application environment of the image processing method in an embodiment. As Figure 1As shown in the figure, the application environment includes an electronic device 110. The electronic device obtains a loss image after specified task processing, and obtains a label image corresponding to the loss image; the color space types of the loss image and the label image are both YUV; based on the loss image and the label image, first loss calculations are respectively performed on the loss Y-channel map, loss U-channel map, and loss V-channel map corresponding to the loss image to obtain a first loss value of the loss image; the first loss value is used to train the specified task model to retain the low-frequency information of the U-channel map and the V-channel map when performing the specified task processing; based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image, a second loss calculation is performed on the loss Y-channel to obtain a second loss value of the loss Y-channel map; based on the first loss value and the second loss value, a target loss value of the loss image is determined. Training the specified task model based on the target loss value can obtain a trained specified task model; inputting an input image into the trained specified task model can more accurately perform the specified task processing on the input image to obtain a more accurate target image. Among them, the specified task may include a noise reduction task or a super-resolution task.

[0028] Among them, the electronic device 110 may be a terminal or a server. The terminal may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices, and the server may be implemented by an independent server or a server cluster composed of multiple servers.

[0029] Figure 2 It is a flowchart of an image processing method in an embodiment. In the image processing method of this embodiment, it is described by taking the electronic device running in Figure 1 as an example. As Figure 2 shown, the image processing method includes steps 202 to 208.

[0030] Step 202, obtain a loss image after specified task processing, and obtain a label image corresponding to the loss image; the color space types of the loss image and the label image are both YUV.

[0031] The specified task is a task specified by the user and can be set as needed. For example, the specified task may be a noise reduction task, a super-resolution task, a beauty task, etc. The loss image refers to an image used to compare with the label image to calculate the loss value. The label image is a pre-defined image used to calculate the loss value by reference. It can be understood that the loss image and the label image have the same shape, both being (H, W, C). Where H, W, and C are the height, width, and number of channels respectively.

[0032] YUV is a type of true-color color space. "Y" represents luminance, which is the grayscale value, and "U" and "V" represent chrominance, which is used to describe the color and saturation of the image and specify the color of pixels.

[0033] The electronic device acquires the image to be processed. After processing the image to be processed through a specified task, a loss image is obtained, and a label image corresponding to the loss image is acquired. Among them, the color space type of the image to be processed can be YUV or other color space types other than YUV. If the color space type of the image to be processed is other than YUV, the color space type of the image to be processed of non-YUV type is converted to YUV to obtain the image to be processed of YUV type. Other color space types other than YUV can specifically be RGB (Red, Green, Blue), CMY (Cyan, Magenta, Yellow), or HSV (Hue, Saturation, Value), etc.

[0034] In step 204, based on the loss image and the label image, the first loss calculation is respectively performed on the loss Y-channel map, the loss U-channel map, and the loss V-channel map corresponding to the loss image to obtain the first loss value of the loss image; the first loss value is used to train the specified task model to retain the low-frequency information of the U-channel map and the V-channel map when performing the specified task processing.

[0035] The loss Y-channel map is the Y-channel map corresponding to the loss image, and the pixels in the loss Y-channel map are all pixels representing the luminance of the loss image. The loss U-channel map is the U-channel map corresponding to the loss image, and the pixels in the loss U-channel map are all pixels representing the chrominance of the loss image. The loss V-channel map is the V-channel map corresponding to the loss image, and the pixels in the loss V-channel map are all pixels representing the chrominance of the loss image.

[0036] Similarly, the label Y-channel map corresponding to the label image is the Y-channel map corresponding to the label image, and the pixels in the label Y-channel map are all pixels representing the luminance of the label image. The label U-channel map is the U-channel map corresponding to the label image, and the pixels in the label U-channel map are all pixels representing the chrominance of the label image. The label V-channel map is the V-channel map corresponding to the label image, and the pixels in the label V-channel map are all pixels representing the chrominance of the label image.

[0037] The first loss value is the loss value calculated for the loss Y-channel map, loss U-channel map, and loss V-channel map of the loss image. The first loss value is used to train the specified task model to retain the low-frequency information of the U-channel map and V-channel map when performing the specified task processing. Among them, the low-frequency information is the information with slow color change in the image, that is, the information of the continuously gradually changing area.

[0038] Specifically, based on the loss image and the label image, the L1 (Least Abosulote Error) loss function is used to perform L1 loss calculations on the loss Y-channel map, loss U-channel map, and loss V-channel map corresponding to the loss image respectively, to obtain the first loss value of the loss image.

[0039] In another embodiment, based on the loss image and the label image, the L2 (Least Square Error) loss function is used to perform L2 loss calculations on the loss Y-channel map, loss U-channel map, and loss V-channel map corresponding to the loss image respectively, to obtain the first loss value of the loss image.

[0040] In another embodiment, based on the loss image and the label image, the TV (Total Variation) loss function is used to perform TV loss calculations on the loss Y-channel map, loss U-channel map, and loss V-channel map corresponding to the loss image respectively, to obtain the first loss value of the loss image.

[0041] In one embodiment, before performing the first loss calculation on the loss Y-channel map, loss U-channel map, and loss V-channel map corresponding to the loss image respectively based on the loss image and the label image to obtain the first loss value of the loss image, it further includes: generating the loss Y-channel map, loss U-channel map, and loss V-channel map according to the loss image respectively, and generating the label Y-channel map, label U-channel map, and label V-channel map according to the label image respectively.

[0042] Taking the generation of the loss Y-channel map, loss U-channel map, and loss V-channel map as an example for illustration:

[0043] In one implementation manner, the electronic device can extract the data of the same channel map from the loss image respectively to generate the loss Y-channel map, loss U-channel map, and loss V-channel map. In another implementation manner, the electronic device can split the loss image to obtain the loss Y-channel map, loss U-channel map, and loss V-channel map. In another implementation manner, the electronic device can also interpolate the loss Y-channel map representing brightness and the loss U-channel map and loss V-channel map representing chromaticity based on each pixel in the loss image. In other implementation manners, the electronic device can also generate the loss Y-channel map, loss U-channel map, and loss V-channel map in other ways, which are not limited herein.

[0044] It should be noted that the labeled Y-channel map, labeled U-channel map, and labeled V-channel map can also be generated in the same way as described above, and will not be elaborated here.

[0045] Step 206: Based on the loss Y-channel map corresponding to the loss image and the labeled Y-channel map of the labeled image, perform a second loss calculation on the loss Y-channel to obtain a second loss value of the loss Y-channel map.

[0046] The second loss value is the loss value calculated for the loss Y-channel map of the loss image.

[0047] Specifically, based on the loss Y-channel map corresponding to the loss image and the labeled Y-channel map of the labeled image, the electronic device uses at least one loss function among the perceptual loss function, low-pass filter loss function, traditional edge operator loss function, and multi-scale structural similarity loss function to perform a second loss calculation on the loss Y-channel map to obtain a second loss value of the loss Y-channel map.

[0048] In one embodiment, the electronic device uses any one of the perceptual loss function, low-pass filter loss function, traditional edge operator loss function, and multi-scale structural similarity loss function to perform a second loss calculation on the loss Y-channel map to obtain a second loss value of the loss Y-channel map.

[0049] In another embodiment, the electronic device uses at least two of the perceptual loss function, low-pass filter loss function, traditional edge operator loss function, and multi-scale structural similarity loss function to perform a second loss calculation on the loss Y-channel map to obtain a second loss value of the loss Y-channel map. Among them, the calculation order of at least two loss functions can be set as needed.

[0050] For example, the electronic device sequentially uses the perceptual loss function, low-pass filter loss function, traditional edge operator loss function, and multi-scale structural similarity loss function to perform a second loss calculation on the loss Y-channel map to obtain a second loss value of the loss Y-channel map. Another example is that the electronic device sequentially uses the low-pass filter loss function, perceptual loss function, and multi-scale structural similarity loss function to perform a second loss calculation on the loss Y-channel map to obtain a second loss value of the loss Y-channel map.

[0051] Step 208: Based on the first loss value and the second loss value, determine the target loss value of the loss image.

[0052] The target loss value is the final loss value obtained after comparing the loss image with the labeled image.

[0053] Specifically, the electronic device combines the first loss function and the second loss function to construct an objective loss function; substitutes the first loss value and the second loss value into the objective loss function to obtain the objective loss value of the loss image. Among them, the first loss function is used for the first loss calculation, and the second loss function is used for the second loss calculation. The combination of the first loss function and the second loss function can be addition, or can be multiplication by a weight factor first and then addition, which is not limited here.

[0054] It can be understood that for YUV-type images, the brightness information (luminance information) represented by the Y-channel map determines the details in the image, so it needs to be optimized with emphasis. For the U-channel map V and the chrominance information represented by the V-channel map, only the low-frequency information needs to be retained, and no additional constraints are required. Therefore, in the above image processing method, the first loss value of the loss image is obtained by performing the first loss calculation on the loss Y-channel map, the loss U-channel map, and the loss V-channel map corresponding to the loss image, and the second loss value of the loss Y-channel map is obtained by performing the second loss calculation on the loss Y-channel. Thus, based on the first loss value and the second loss value, the objective loss value of the loss image can be determined. That is to say, the objective loss value not only has the second loss value for the loss Y-channel map. The second loss value can be used to train the specified task model to more accurately reduce the luminance noise in the Y-channel map when performing the specified task. There is also the first loss value calculated from the loss Y-channel map, the loss U-channel map, and the loss V-channel map of the loss image. The first loss value can be used to train the specified task model to retain the low-frequency information of the U-channel map and the V-channel map when performing the specified task, so as to remove the high-frequency noise (color noise) of the U-channel map and the V-channel map. Therefore, a more accurate objective loss value can be determined. Further, based on this more accurate objective loss value, a more accurate specified task model can be trained. The specified task model can better retain the texture information while removing the color noise in the image.

[0055] In one embodiment, based on the loss image and the label image, the first loss calculation is respectively performed on the loss Y-channel map, the loss U-channel map, and the loss V-channel map corresponding to the loss image to obtain the first loss value of the loss image, including: based on the loss image and the label image, the first loss calculation is respectively performed on the loss Y-channel map, the loss U-channel map, and the loss V-channel map corresponding to the loss image to obtain the first Y-channel loss value of the loss Y-channel map, the first U-channel loss value of the loss U-channel map, and the first V-channel loss value of the loss V-channel map; based on the first Y-channel loss value, the first U-channel loss value, and the first V-channel loss value, the first loss value of the loss image is determined.

[0056] The first Y-channel loss value is the loss value obtained by performing a first loss calculation on the loss Y-channel map of the loss image. The first U-channel loss value is the loss value obtained by performing a first loss calculation on the loss U-channel map of the loss image. The first V-channel loss value is the loss value obtained by performing a first loss calculation on the loss V-channel map of the loss image.

[0057] Specifically, the electronic device calculates the L1 loss of the loss Y-channel map using the L1 loss function based on the loss Y-channel map of the loss image and the label Y-channel map of the label image, obtaining the first Y-channel loss value; calculates the L1 loss of the loss U-channel map using the L1 loss function based on the loss U-channel map of the loss image and the label U-channel map of the label image, obtaining the first U-channel loss value; calculates the L1 loss of the loss V-channel map using the L1 loss function based on the loss V-channel map of the loss image and the label V-channel map of the label image, obtaining the first V-channel loss value.

[0058] The electronic device calculates the first Y-channel loss value by performing L1 loss calculation on the loss Y-channel map using the following L1 loss function: where L Y is the first Y-channel loss value, H represents the height of the loss Y-channel map, W represents the width of the loss Y-channel map, I denoiseY is the loss Y-channel map, and I gtY is the label Y-channel map.

[0059] The electronic device calculates the first U-channel loss value by performing L1 loss calculation on the loss U-channel map using the following L1 loss function: where L U is the first U-channel loss value, H represents the height of the loss U-channel map, W represents the width of the loss U-channel map, I denoiseU is the loss U-channel map, and I gtU is the label U-channel map.

[0060] The electronic device calculates the first V-channel loss value by performing L1 loss calculation on the loss V-channel map using the following L1 loss function: where L V is the first V-channel loss value, H represents the height of the loss V-channel map, W represents the width of the loss V-channel map, I denoiseV is the loss V-channel map, and I gtV is the label V-channel map.

[0061] It should be noted that in other embodiments, the electronic device may also perform the first loss calculation using other loss functions such as the L2 loss function or the TV loss function, so as to obtain the first Y-channel loss value of the loss Y-channel map, the first U-channel loss value of the loss U-channel map, and the first V-channel loss value of the loss V-channel map.

[0062] In this embodiment, based on the loss image and the label image, first loss calculations are respectively performed on the loss Y-channel map, loss U-channel map, and loss V-channel map corresponding to the loss image to obtain the first Y-channel loss value of the loss Y-channel map, the first U-channel loss value of the loss U-channel map, and the first V-channel loss value of the loss V-channel map. Thus, based on the first Y-channel loss value, the first U-channel loss value, and the first V-channel loss value, the first loss value of the loss image can be accurately determined.

[0063] In one embodiment, the electronic device can also perform a first loss calculation based on the loss U-channel map of the loss image and the label U-channel map of the label image to obtain the first U-channel loss value of the loss U-channel map, and perform a first loss calculation based on the loss V-channel map of the loss image and the label V-channel map of the label image to obtain the first V-channel loss value of the loss V-channel map. It can be understood that both the first U-channel loss value and the first V-channel loss value are loss values for the channel maps (loss U-channel map and loss V-channel map) representing chrominance. Then, the first U-channel loss value and the first V-channel loss value can be used to optimize the chrominance processing when the specified task model performs the specified task.

[0064] Similarly, the electronic device can also perform a first loss calculation and a second loss calculation respectively based on the loss Y-channel map of the loss image and the label Y-channel map of the label image to obtain the first Y-channel loss value and the second loss value of the loss Y-channel map. The second loss value is also a loss value calculated for the loss Y-channel map. It can be understood that both the first Y-channel loss value and the second loss value are loss values for the Y-channel map representing luminance. Then, the first Y-channel loss value and the second loss value can be used to optimize the luminance processing when the specified task model performs the specified task.

[0065] In one embodiment, determining the first loss value of the loss image based on the first Y-channel loss value, the first U-channel loss value, and the first V-channel loss value includes: respectively obtaining the Y-channel loss weight corresponding to the first Y-channel loss value, the U-channel loss weight corresponding to the first U-channel loss value, and the V-channel loss weight corresponding to the first V-channel loss value; where the Y-channel loss weight and the U-channel loss weight are not the same, and the Y-channel loss weight and the V-channel loss weight are not the same; multiplying the first Y-channel loss value by the corresponding Y-channel loss weight, multiplying the first U-channel loss value by the corresponding U-channel loss weight, and multiplying the first V-channel loss value by the corresponding V-channel loss weight, and then adding the three products obtained by multiplication to obtain the first loss value of the loss image.

[0066] The weight of the Y-channel loss is the weight factor corresponding to the first Y-channel loss value. The weight of the U-channel loss is the weight factor corresponding to the first U-channel loss value. The weight of the V-channel loss is the weight factor corresponding to the first V-channel loss value. The weights of the Y-channel loss, U-channel loss, and V-channel loss can all be set according to the empirical values obtained from actual debugging or according to the actual scenario. For example, different weights of the Y-channel loss, U-channel loss, and V-channel loss can be set according to different scenarios during the day and at night.

[0067] In one embodiment, the value of the weight of the Y-channel loss is higher than the values of the weights of the U-channel loss and V-channel loss. That is, the first Y-channel loss value weighted by a higher weight can be used to train the specified task model to retain the high-frequency information of the Y-channel map when performing the specified task. Among them, the high-frequency information is the information with fast frequency changes in the image, that is, the edge information in the image.

[0068] For example, the weight of the Y-channel loss is set to 3, and the weights of the U-channel loss and V-channel loss are both set to 1. Another example is that the weight of the Y-channel loss is set to 3, the weight of the U-channel loss is set to 2, and the weight of the V-channel loss is set to 1.

[0069] It can be understood that the weight of the Y-channel loss corresponding to the first Y-channel loss value, the weight of the U-channel loss corresponding to the first U-channel loss value, and the weight of the V-channel loss corresponding to the first V-channel loss value are obtained respectively, and the weight of the Y-channel loss is inconsistent with the weight of the U-channel loss, and the weight of the Y-channel loss is inconsistent with the weight of the V-channel loss. Then, the first Y-channel loss value after weighted processing can be used to train the specified task model to retain the high-frequency information of the Y-channel map when performing the specified task.

[0070] The electronic device calculates the first loss value of the loss image using the following formula: L 1 = L Y + γ(L U + L V ). Where L 1 is the first loss value, L Y is the first Y-channel loss value, L U is the first U-channel loss value, L V is the first V-channel loss value, γ is the weight of the Y-channel loss, and the weights of the U-channel loss and V-channel loss are both 1.

[0071] In one embodiment, based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image, a second loss calculation is performed on the loss Y-channel to obtain the second loss value of the loss Y-channel map, including: based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image, a perceptual loss calculation, a low-pass filtering loss calculation, a traditional edge operator loss calculation, and a multi-scale structural similarity loss calculation are respectively performed on the loss Y-channel map to obtain the perceptual loss, the low-pass filtering loss, the traditional edge operator loss, and the multi-scale structural similarity loss of the loss Y-channel map; based on the perceptual loss, the low-pass filtering loss, the traditional edge operator loss, and the multi-scale structural similarity loss of the loss Y-channel map, the second loss value of the loss Y-channel map is determined.

[0072] Specifically, the electronic device performs a perceptual loss calculation on the loss Y-channel map to obtain the perceptual loss of the loss Y-channel map by using a perceptual loss function based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image, performs a low-pass filtering loss calculation on the loss Y-channel map to obtain the low-pass filtering loss of the loss Y-channel map by using a low-pass filtering loss function, performs a traditional edge operator loss calculation on the loss Y-channel map to obtain the traditional edge operator loss by using a traditional edge operator loss function, and performs a multi-scale structural similarity loss calculation on the loss Y-channel map to obtain the multi-scale structural similarity loss by using a multi-scale structural similarity loss function; weight factors of the perceptual loss, the low-pass filtering loss, the traditional edge operator loss, and the multi-scale structural similarity loss are respectively obtained, the perceptual loss, the low-pass filtering loss, the traditional edge operator loss, and the multi-scale structural similarity loss are respectively multiplied by the corresponding weight factors, and then the four products obtained by multiplication are added to obtain the second loss value of the loss Y-channel map.

[0073] The electronic device performs a perceptual loss calculation on the loss Y-channel map to obtain the perceptual loss of the loss Y-channel map by using a perceptual loss function based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image. Specifically, the electronic device inputs the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image into a VGG-16 (Visual Geometry Group Network) network, extracts the output features of a specified layer from the VGG-16 network, and performs a perceptual loss calculation on the output features of the specified layer to obtain the perceptual loss of the loss Y-channel map.

[0074] Among them, the VGG-16 network includes 13 convolutional layers and 3 fully connected layers. It can be understood that different layers in the VGG-16 network can extract features of different scales. The specified layer can be set as needed. In this embodiment, the specified layer can be selected as the 4th layer and the 7th layer.

[0075] Taking the loss image as the denoised image and the specified layer as the 4th layer and the 7th layer for illustration. The electronic device defines the feature shape output by the i-th layer in the VGG-16 network as (H i , Wi , C i ), H i represents the height of the feature shape output by the i-th layer, W i represents the width of the feature shape output by the i-th layer, C i represents the number of channels of the feature shape output by the i-th layer. The electronic device extracts the output features of the 4th layer and the 7th layer from the VGG-16 network. The features of the label images output by the 4th layer and the 7th layer are F4 gtY and F7 gtY , and the features of the denoised images output by the 4th layer and the 7th layer are F4 denoiseY and F7 denoiseY . Obtain the perceptual loss parameter alpha = 0.8, then the perceptual loss where L perceptual is the perceptual loss of the loss Y-channel map, C 4 *H 4 *W 4 is the feature shape output by the 4th layer, C 7 *H 7 *W 7 is the feature shape output by the 7th layer.

[0076] It can be understood that the perceptual loss only acts on the loss Y-channel map, and more semantic information and high-frequency information can be extracted by means of the VGG-16 network, so that the perceptual loss is used to train the specified task model to retain the details in the image when performing the specified task processing.

[0077] The electronic device performs low-pass filtering loss calculation on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image by calling the low-pass filtering loss function with a specified kernel size, and obtains the low-pass filtering loss of the loss Y-channel map. Among them, the specified kernel size can be set as needed. For example, the specified kernel size is 9.

[0078] It can be understood that the electronic device performs low-pass filtering loss calculation based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image, and obtains the low-pass filtering loss of the loss Y-channel map. This low-pass filtering loss can be used to train the specified task model to extract the low-frequency information in the image when performing the specified task processing, and this low-frequency information can ensure the overall contour of the image. As Figure 3 shown, it is a comparison diagram of the image before and after low-pass filtering in an embodiment.

[0079] Taking the loss image as the denoised image for illustration. The low-pass filtering loss of the loss Y-channel map is calculated by the following formula:

[0080] where Llowpass is the low-pass filtering loss of the Y-channel map of the denoised image, H is the height of the Y-channel map of the denoised image, W is the width of the Y-channel map of the denoised image, and I denoiseY is the Y-channel map of the denoised image, I gtY is the label Y-channel map.

[0081] The electronic device uses a four-directional sobel operator to extract the edge information of the loss Y-channel map and the label Y-channel map, and the traditional edge operator loss of the loss Y-channel map can be calculated.

[0082] Taking the loss image as the denoised image for illustration. The low traditional edge operator loss of the Y-channel map of the denoised image is calculated by the following formula:

[0083] where, L sobel is the low traditional edge operator loss of the Y-channel map of the denoised image, H is the height of the Y-channel map of the denoised image, W is the width of the Y-channel map of the denoised image, and I denoiseY is the Y-channel map of the denoised image, I gtY is the label Y-channel map.

[0084] It can be understood that by calculating the traditional edge operator loss of the loss Y-channel map, it can be used to train the specified task model to constrain the edge information when performing the specified task processing.

[0085] Taking the loss image as the denoised image for illustration. The multi-scale structural similarity loss of the Y-channel map of the denoised image is calculated by the following formula:

[0086] L MS-SSIM = 1 - MS-SSIM(I gtY , I denoiseY ). Where, L MS-SSIM is the multi-scale structural similarity loss, I denoiseY is the Y-channel map of the denoised image, I gtY is the label Y-channel map.

[0087] It can be understood that by calculating the multi-scale structural similarity loss of the loss Y-channel map, it can be used to train the specified task model to ensure the similarity of the spatial structure of the image when performing the specified task processing.

[0088] Furthermore, the electronic device calculates the target loss value of the loss image by the following formula:

[0089] L YUV = L 1 + 0.01 * L perceptual + 0.1 * L lowpass + 0.1 * L sobel + 0.1 * L MS-SSIM. Among them, L YUV is the target loss value, L 1 is the first loss value, L perceptual is the perceptual loss, L lowpass is the low-pass filtering loss, L sobel is the traditional edge operator loss, L MS-SSIM is the multi-scale structural similarity loss.

[0090] In this embodiment, based on the loss Y-channel map of the loss image and the label Y-channel map of the label image, the perceptual loss calculation, low-pass filtering loss calculation, traditional edge operator loss calculation, and multi-scale structural similarity loss calculation are respectively performed on the loss Y-channel map to obtain the perceptual loss, low-pass filtering loss, traditional edge operator loss, and multi-scale structural similarity loss of the loss Y-channel map. Then, based on the perceptual loss, low-pass filtering loss, traditional edge operator loss, and multi-scale structural similarity loss of the loss Y-channel map, the second loss value of the loss Y-channel map is determined. The perceptual loss included in the second loss value can be used to train the specified task model to retain details in the image when performing the specified task. The low-pass filtering loss included in the second loss value can be used to train the specified task model to extract low-frequency information in the image when performing the specified task. The traditional edge operator loss included in the second loss value can be used to train the specified task model to constrain edge information when performing the specified task. The multi-scale structural similarity loss included in the second loss value can be used to train the specified task model to ensure the similarity of the spatial structure of the image when performing the specified task. That is to say, the trained specified task model can better retain the edge information, spatial structure, and high and low-frequency important information in the image on the basis of removing color noise and brightness noise, improving the fineness of image denoising and enabling more accurate image denoising.

[0091] In one embodiment, the electronic device uses the trained specified task model to process the image obtained by motion capture for the specified task, and can perform more accurate denoising on the basis of the specified task processing to obtain a more accurate and clearer image.

[0092] In one embodiment, obtaining the loss image after the specified task processing includes: obtaining the image to be processed; performing denoising processing on the image to be processed in the denoising task to obtain a denoised image, and using the denoised image as the loss image, or performing super-resolution processing on the image to be processed in the super-resolution task to obtain a super-resolution image, and using the super-resolution image as the loss image.

[0093] The denoising task is a task of performing denoising processing on the image to be processed. The super-resolution task is a task of performing super-resolution processing on the image to be processed. The denoised image is the image obtained after the image to be processed undergoes denoising processing. The super-resolution image is the image obtained after the image to be processed undergoes super-resolution processing.

[0094] The electronic device acquires the image to be processed, inputs the image to be processed into the noise reduction network, and performs noise reduction processing on the image to be processed through a convolutional neural network to obtain a denoised image. Among them, the noise reduction network is a convolutional neural network. The electronic device acquires the image to be processed, inputs the image to be processed into the super-resolution model, and performs super-resolution processing on the image to be processed through the super-resolution model to obtain a super-resolution image.

[0095] In this embodiment, the electronic device performs noise reduction processing on the image to be processed to obtain a denoised image, or performs super-resolution processing on the image to be processed to obtain a super-resolution image, and uses the denoised image or the super-resolution image as the loss image, then the loss value between the denoised image or the super-resolution image and the label image can be accurately calculated.

[0096] In one embodiment, acquiring the label image corresponding to the loss image includes: if the loss image is a denoised image, capturing multiple frames of scene images of the same shooting scene; performing averaging processing on the multiple frames of scene images to obtain the label image corresponding to the denoised image.

[0097] Among them, the scene image is captured from the same scene and is the image used to generate the label image corresponding to the denoised image. The number of scene images captured by the electronic device for the same shooting scene can be set as needed. For example, the electronic device captures 120 frames of scene images of the same shooting scene, and performs averaging processing on the 120 frames of scene images to obtain the label image corresponding to the denoised image.

[0098] It can be understood that the noise in a single scene image conforms to a zero-mean distribution, and performing averaging processing on multiple frames of scene images can effectively eliminate the noise. And the noise level in the label image is negatively correlated with the number of scene images. That is to say, the more the number of scene images, the lower the noise level of the label image.

[0099] In one embodiment, the color space type of the scene image is YUV; performing averaging processing on multiple frames of scene images to obtain the label image corresponding to the loss image includes: converting the color space type of each frame of scene image to RGB to obtain the scene image of RGB type; performing averaging processing on multiple frames of scene images of RGB type to obtain the label image of RGB type; converting the color space type of the label image of RGB type to YUV to obtain the label image of YUV type.

[0100] It can be understood that if an electronic device performs averaging processing on multiple frames of scene images in the YUV domain, the resulting label image of the YUV type will have a color cast problem. Therefore, in this embodiment, the electronic device first converts each frame of the YUV type scene image into an RGB type scene image, performs averaging processing on multiple frames of the RGB type scene images in the RGB domain to obtain an RGB type label image, and then converts the RGB type label image into a YUV type label image, which can avoid the color cast problem of the generated YUV type label image, improve the accuracy of the generated label image, and thus can more accurately calculate the target loss value between the loss image and the label image.

[0101] In one embodiment, the above method further includes: training a specified task model based on the target loss value until a training termination condition is met to obtain a trained specified task model; processing an input image through the trained specified task model to perform a specified task to obtain a target image; where the specified task includes a noise reduction task or a super-resolution task.

[0102] If the specified task is a noise reduction task, the specified task model can be a noise reduction model, that is, a noise reduction network. Among them, the noise reduction network is a convolutional neural network. If the specified task is a super-resolution task, the specified task model can be a super-resolution model.

[0103] The training termination condition can be set as needed. In one embodiment, the training termination condition can be that the target loss value is less than a preset threshold. In another embodiment, the training termination condition can be that the number of training times reaches a preset number. In other embodiments, the training termination condition can also be that the training duration reaches a preset duration. Among them, the preset threshold, the preset number, and the preset duration can all be set as needed.

[0104] The input image is the image input to the trained specified task model. The target image is the image obtained after processing the input image through the trained specified task model to perform a specified task.

[0105] In this embodiment, the electronic device trains the specified task model based on the target loss value until the training termination condition is met, and a trained specified task model can be obtained. This trained specified task model can more accurately process the input image to perform a specified task, thereby obtaining a more accurate target image.

[0106] In one embodiment, as Figure 4As shown, the electronic device acquires the image to be processed, inputs the image to be processed into the specified task model, obtains the loss image processed by the specified task, and acquires the label image corresponding to the loss image. By using the pre-constructed objective loss function to calculate the loss image and the label image, the objective loss value of the loss image can be obtained.

[0107] It should be understood that although Figure 2 and Figure 4 the steps in the flowchart of Figure 2 and Figure 4 are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover,

[0108] Figure 5 is a structural block diagram of an image processing device according to an embodiment. As Figure 5 shown, an image processing device is provided, including: an acquisition module 502 and a loss calculation module 504, where:

[0109] The acquisition module 502 is configured to acquire the loss image after being processed by the specified task, and acquire the label image corresponding to the loss image; the color space types of the loss image and the label image are both YUV.

[0110] The loss calculation module 504 is configured to perform a first loss calculation on the loss Y-channel map, the loss U-channel map, and the loss V-channel map corresponding to the loss image respectively based on the loss image and the label image, to obtain the first loss value of the loss image; the first loss value is used to train the specified task model to retain the low-frequency information of the U-channel map and the V-channel map when performing the specified task processing.

[0111] The loss calculation module 504 is further configured to perform a second loss calculation on the loss Y-channel based on the loss Y-channel map of the loss image and the label Y-channel map of the label image, to obtain the second loss value of the loss Y-channel map.

[0112] The loss calculation module 504 is further configured to determine the objective loss value of the loss image based on the first loss value and the second loss value.

[0113] The above image processing device performs a first loss calculation on the loss Y-channel map, loss U-channel map, and loss V-channel map corresponding to the loss image to obtain the first loss value of the loss image, and performs a second loss calculation on the loss Y-channel to obtain the second loss value of the loss Y-channel map, so that the target loss value of the loss image can be determined based on the first loss value and the second loss value. That is to say, the target loss value not only has the second loss value for the loss Y-channel map, and the second loss value can be used to train the specified task model to more accurately reduce the luminance noise in the Y-channel map when performing the specified task, but also has the first loss value calculated from the loss Y-channel map, loss U-channel map, and loss V-channel map of the loss image. This first loss value can be used to train the specified task model to retain the low-frequency information of the U-channel map and V-channel map when performing the specified task, so as to remove the high-frequency noise of the U-channel map and V-channel map. Therefore, a more accurate target loss value can be determined. Further, a more accurate specified task model can be trained based on this more accurate target loss value.

[0114] In one embodiment, the loss calculation module 504 is further configured to perform a first loss calculation on the loss Y-channel map, loss U-channel map, and loss V-channel map corresponding to the loss image respectively based on the loss image and the label image, to obtain the first Y-channel loss value of the loss Y-channel map, the first U-channel loss value of the loss U-channel map, and the first V-channel loss value of the loss V-channel map; and determine the first loss value of the loss image based on the first Y-channel loss value, the first U-channel loss value, and the first V-channel loss value.

[0115] In one embodiment, the loss calculation module 504 is further configured to respectively obtain the Y-channel loss weight corresponding to the first Y-channel loss value, the U-channel loss weight corresponding to the first U-channel loss value, and the V-channel loss weight corresponding to the first V-channel loss value; wherein, the Y-channel loss weight and the U-channel loss weight are inconsistent, and the Y-channel loss weight and the V-channel loss weight are inconsistent; multiply the first Y-channel loss value by the corresponding Y-channel loss weight, multiply the first U-channel loss value by the corresponding U-channel loss weight, and multiply the first V-channel loss value by the corresponding V-channel loss weight, and then add the three products obtained by multiplication to obtain the first loss value of the loss image.

[0116] In one embodiment, the loss calculation module 504 is further configured to perform perceptual loss calculation, low-pass filtering loss calculation, traditional edge operator loss calculation, and multi-scale structural similarity loss calculation on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image respectively, to obtain the perceptual loss, low-pass filtering loss, traditional edge operator loss, and multi-scale structural similarity loss of the loss Y-channel map; and determine the second loss value of the loss Y-channel map based on the perceptual loss, low-pass filtering loss, traditional edge operator loss, and multi-scale structural similarity loss of the loss Y-channel map.

[0117] In one embodiment, the acquisition module 502 is further configured to acquire an image to be processed; perform noise reduction processing on the image to be processed in a noise reduction task to obtain a noise-reduced image, and use the noise-reduced image as the loss image, or perform super-resolution processing on the image to be processed in a super-resolution task to obtain a super-resolution image, and use the super-resolution image as the loss image.

[0118] In one embodiment, the acquisition module 502 is further configured to, if the loss image is a noise-reduced image, capture multiple frames of scene images of the same shooting scene; perform averaging processing on the multiple frames of scene images to obtain a label image corresponding to the noise-reduced image.

[0119] In one embodiment, the color space type of the scene image is YUV; the acquisition module 502 is further configured to convert the color space type of each frame of scene image into RGB to obtain a scene image of RGB type; perform averaging processing on the multiple frames of scene images of RGB type to obtain a label image of RGB type; and convert the color space type of the label image of RGB type into YUV to obtain a label image of YUV type.

[0120] In one embodiment, the above device further includes a training module, configured to train a specified task model based on a target loss value until a training cut-off condition is met, to obtain a trained specified task model; and process an input image through the trained specified task model to obtain a target image; where the specified task includes a noise reduction task or a super-resolution task.

[0121] The division of each module in the above image processing device is only for illustrative purposes. In other embodiments, the image processing device may be divided into different modules as needed to complete all or part of the functions of the above image processing device.

[0122] For the specific limitations of the image processing device, reference may be made to the limitations on the image processing method in the foregoing text, which will not be elaborated here. Each module in the foregoing image processing device may be implemented in whole or in part by software, hardware, or a combination thereof. Each of the foregoing modules may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to each of the foregoing modules.

[0123] Figure 6 FIG. is a schematic internal structure diagram of an electronic device in an embodiment. The electronic device may be any terminal device such as a mobile phone, a tablet computer, a notebook computer, a desktop computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a vehicle-mounted computer, a wearable device, etc. The electronic device includes a processor and a memory connected through a system bus. Among them, the processor may include one or more processing units. The processor may be a CPU (Central Processing Unit) or a DSP (Digital Signal Processing), etc. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program may be executed by the processor to implement an image processing method provided in each of the following embodiments. The internal memory provides a high-speed cache operating environment for the operating system computer program in the non-volatile storage medium.

[0124] In the embodiments of the present application, the implementation of each module in the image processing device may be in the form of a computer program. The computer program may run on a terminal or a server. The program module constituted by the computer program may be stored on the memory of the electronic device. When the computer program is executed by the processor, the steps of the method described in the embodiments of the present application are implemented.

[0125] The embodiments of the present application further provide a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, cause the processors to execute the steps of the image processing method.

[0126] The embodiments of the present application further provide a computer program product containing instructions, which when run on a computer, causes the computer to execute the image processing method.

[0127] Any reference to memory, storage, database, or other media used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include ROM (Read-Only Memory), PROM (Programmable Read-only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-only Memory), or flash memory. Volatile memory may include RAM (Random Access Memory), which serves as an external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), SDRAM (Synchronous Dynamic Random Access Memory), double data rate DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access memory), ESDRAM (Enhanced Synchronous Dynamic Random Access memory), SLDRAM (Sync Link Dynamic Random Access Memory), RDRAM (Rambus Dynamic Random Access Memory), DRDRAM (Direct Rambus Dynamic Random Access Memory).

[0128] The above-described embodiments merely represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all fall within the protection scope of this application. Therefore, the protection scope of the patent of this application shall be subject to the appended claims.

Claims

1. An image processing method, characterized in that, comprising: obtaining a loss image after being processed by a specified task, and obtaining a label image corresponding to the loss image; both the loss image and the label image are in the YUV color space type, the loss image refers to an image used to be compared with the label image to calculate a loss value, and the label image is a pre-defined image used as a reference to calculate the loss value; based on the loss image and the label image, respectively performing a first loss calculation on the loss Y-channel map, loss U-channel map, and loss V-channel map corresponding to the loss image to obtain a first Y-channel loss value of the loss Y-channel map, a first U-channel loss value of the loss U-channel map, and a first V-channel loss value of the loss V-channel map; based on the first Y-channel loss value, the first U-channel loss value, and the first V-channel loss value, determining a first loss value of the loss image; the first loss value is used to train the specified task model to retain the low-frequency information of the U-channel map and the V-channel map when performing the specified task processing; based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image, performing a second loss calculation on the loss Y-channel to obtain a second loss value of the loss Y-channel map; based on the first loss value and the second loss value, determining a target loss value of the loss image.

2. The method according to claim 1, characterized in that, the determining the first loss value of the loss image based on the first Y-channel loss value, the first U-channel loss value, and the first V-channel loss value includes: respectively obtaining a Y-channel loss weight corresponding to the first Y-channel loss value, a U-channel loss weight corresponding to the first U-channel loss value, and a V-channel loss weight corresponding to the first V-channel loss value; wherein, the Y-channel loss weight and the U-channel loss weight are inconsistent, and the Y-channel loss weight and the V-channel loss weight are inconsistent; multiplying the first Y-channel loss value by the corresponding Y-channel loss weight, multiplying the first U-channel loss value by the corresponding U-channel loss weight, and multiplying the first V-channel loss value by the corresponding V-channel loss weight, and then adding the three products obtained by multiplication to obtain the first loss value of the loss image.

3. The method according to claim 1, characterized in that, the performing a second loss calculation on the loss Y-channel based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image to obtain a second loss value of the loss Y-channel map includes: based on the loss Y-channel map corresponding to the loss image and the label Y-channel map of the label image, respectively performing a perceptual loss calculation, a low-pass filtering loss calculation, a traditional edge operator loss calculation, and a multi-scale structural similarity loss calculation on the loss Y-channel map to obtain a perceptual loss, a low-pass filtering loss, a traditional edge operator loss, and a multi-scale structural similarity loss of the loss Y-channel map; Determine the second loss value of the loss Y-channel map based on the perceptual loss, low-pass filtering loss, traditional edge operator loss, and multi-scale structural similarity loss of the loss Y-channel map.

4. The method according to claim 1, wherein, the obtaining of the loss image after being processed by a specified task includes: obtaining an image to be processed; performing noise reduction processing on the image to be processed in a noise reduction task to obtain a noise-reduced image, and using the noise-reduced image as the loss image, or performing super-resolution processing on the image to be processed in a super-resolution task to obtain a super-resolution image, and using the super-resolution image as the loss image.

5. The method according to claim 1, wherein, the obtaining of the label image corresponding to the loss image includes: if the loss image is a noise-reduced image, capturing multiple frames of scene images of the same shooting scene; performing averaging processing on the multiple frames of scene images to obtain the label image corresponding to the noise-reduced image.

6. The method according to claim 5, wherein, the color space type of the scene image is YUV; the performing of averaging processing on the multiple frames of scene images to obtain the label image corresponding to the noise-reduced image includes: converting the color space type of each frame of the scene image to RGB to obtain a scene image of RGB type; performing averaging processing on the multiple frames of scene images of RGB type to obtain a label image of RGB type; converting the color space type of the label image of RGB type to YUV to obtain a label image of YUV type.

7. The method according to any one of claims 1 to 6, wherein, the method further includes: training the specified task model based on the target loss value until a training cut-off condition is met to obtain a trained specified task model; processing an input image by the trained specified task model to obtain a target image; wherein, the specified task includes a noise reduction task or a super-resolution task.

8. An image processing apparatus, wherein, it includes: an obtaining module, configured to obtain a loss image after being processed by a specified task, and obtain a label image corresponding to the loss image; the color space types of the loss image and the label image are both YUV, the loss image refers to an image used to be compared with the label image to calculate a loss value, and the label image is a pre-defined image used to calculate a loss value by reference; a loss calculation module, configured to perform a first loss calculation on the loss Y-channel map, loss U-channel map, and loss V-channel map corresponding to the loss image respectively based on the loss image and the label image to obtain a first Y-channel loss value of the loss Y-channel map, a first U-channel loss value of the loss U-channel map, and a first V-channel loss value of the loss V-channel map; determine the first loss value of the loss image based on the first Y-channel loss value, the first U-channel loss value, and the first V-channel loss value; the first loss value is used to train the specified task model to retain the low-frequency information of the U-channel map and the V-channel map when performing the specified task processing. The loss calculation module is further configured to perform a second loss calculation on the loss Y channel based on the loss Y channel map corresponding to the loss image and the label Y channel map of the label image, so as to obtain a second loss value of the loss Y channel map; The loss calculation module is further configured to determine a target loss value of the loss image based on the first loss value and the second loss value.

9. An electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is caused to execute the steps of the image processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images