Image processing method and device

Abstract

The present application provides an image processing method and device, wherein the image processing method comprises: receiving a change request, wherein the change request carries image information and target color data; determining a first weight image corresponding to the image to be changed by performing style transfer on the image to be changed corresponding to the image information, and generating an inverted image based on the pixel value of each pixel in the first weight image; generating a second weight image based on the pixel value of each pixel in the inverted image; converting the image to be changed into a target image based on the first weight image, the second weight image and the target color data in the change request. The color of the entire area in the image to be changed is changed by using two weight images, so that the style change effect of the image to be changed is more in line with expectations.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and device. Background Art

[0002] Image style transfer refers to transferring the style of one image to another image, and the resulting image shows a combination of image content and image style. In the prior art, when transferring the style of an image, a style transfer model is usually used to transfer the style of the image. However, the result of style transfer may not be ideal. This is because there may be some image areas in the image that are easily affected by style transfer, but there are still some image areas that are not easily affected by style transfer. Summary of the invention

[0003] In view of this, the embodiments of the present application provide an image processing method and apparatus, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of the present application, there is provided an image processing method, comprising:

[0005] receiving a change request, wherein the change request carries image information and target color data;

[0006] By performing style migration on the image to be changed corresponding to the image information, a first weighted image corresponding to the image to be changed is determined, and an inverted image is generated according to a pixel value of each pixel in the first weighted image;

[0007] Generate a second weight image based on the pixel value of each pixel in the inverted image;

[0008] The image to be changed is converted into a target image according to the first weight image, the second weight image and the target color data.

[0009] Optionally, converting the image to be changed into a target image according to the first weight image, the second weight image and the target color data comprises:

[0010] Normalizing the pixel value of each pixel in the first weight image to obtain a first weight corresponding to each pixel in the first weight image, and normalizing the pixel value of each pixel in the second weight image to obtain a second weight corresponding to each pixel in the second weight image;

[0011] The image to be changed is converted into a target image according to the first weight, the second weight and the target color data.

[0012] Optionally, converting the image to be changed into a target image according to the first weight, the second weight and the target color data includes:

[0013] Determining, in the target color data, first color data for changing the color of the image to be changed, and second color data for changing the color of the changed first initial changed image;

[0014] Calculating the first weight, the first color data, and the pixel value of each pixel in the image to be changed according to a first color conversion algorithm to generate the first initial changed image;

[0015] The second weight, the second color data, and the pixel value of each pixel in the first initial changed image are calculated according to a second color conversion algorithm to generate a target image.

[0016] Optionally, converting the image to be changed into a target image according to the first weight, the second weight and the target color data includes:

[0017] Determining, in the target color data, third color data for changing the color of the image to be changed, and fourth color data for changing the color of the changed second initial changed image;

[0018] Calculating the second weight, the third color data, and the pixel value of each pixel in the image to be changed according to a third color conversion algorithm to generate the second initial changed image;

[0019] The first weight, the fourth color data, and the pixel value of each pixel in the second initial changed image are calculated according to a fourth color conversion algorithm to generate a target image.

[0020] Optionally, determining, according to the image information, a first weight image corresponding to the image to be changed includes:

[0021] Determining an image to be changed according to the image information, and determining a style image corresponding to the image to be changed;

[0022] Combining the style image with the image to be changed to generate an image pair;

[0023] Inputting the image pair into a style transfer model to obtain a composite image output by the style transfer model;

[0024] Performing grayscale processing on the image to be changed to generate a first grayscale image, and performing grayscale processing on the synthesized image to generate a second grayscale image;

[0025] Performing a subtraction operation based on the pixel values ​​of the pixels in the first grayscale image and the pixel values ​​of the pixels in the second grayscale image to obtain a difference grayscale image;

[0026] When the number of the difference grayscale images is one, using the difference grayscale image as a first weight map;

[0027] In the case that there are a plurality of difference grayscale images, an AND operation is performed based on the pixel values ​​of the pixels in each difference grayscale image to obtain the first weight image.

[0028] Optionally, before performing grayscale processing on the image to be changed to generate the first grayscale image, the method further includes:

[0029] Determining an image to be changed according to the image information, and determining a style image corresponding to the image to be changed;

[0030] When the length and width of the image to be changed are equal, the image to be changed is continuously expanded in four directions to obtain an expanded image to be changed;

[0031] Combining the expanded image to be changed with the style image to generate an expanded image pair;

[0032] Inputting the extended image pair into a style transfer model to obtain an extended synthetic image output by the style transfer model;

[0033] According to the image size of the image to be changed, a clipped image to be changed is obtained from the expanded image to be changed, and the clipped image to be changed is used as the image to be changed, and,

[0034] The extended composite image is cut out according to the image size to obtain a composite image.

[0035] Optionally, after performing a subtraction operation on the pixel values ​​of the pixels in the first grayscale image and the pixel values ​​of the pixels in the second grayscale image to obtain a difference grayscale image, the method further comprises:

[0036] The difference grayscale image is blurred to obtain a first blurred grayscale image, and the first blurred grayscale image is used as the difference grayscale image.

[0037] Optionally, after performing a subtraction operation on the pixel values ​​of the pixels in the first grayscale image and the pixel values ​​of the pixels in the second grayscale image to obtain a difference grayscale image, the method further includes:

[0038] Downsampling the difference grayscale image to generate a reduced grayscale image;

[0039] Blurring the reduced grayscale image to obtain a second blurred grayscale image;

[0040] The second blurred grayscale image is upsampled to generate an enlarged grayscale image, and the enlarged grayscale image is used as the difference grayscale image.

[0041] Optionally, generating a second weight image based on the pixel value of each pixel in the inverted image includes:

[0042] Calculate the pixel value of each pixel in the inverted image based on a preset algorithm to obtain a weighted pixel value corresponding to each pixel in the inverted image;

[0043] A second weight image is generated based on the weight pixel values.

[0044] Optionally, before generating the second weighted image based on the weighted pixel values, the method further includes:

[0045] In the case that there is a weighted pixel value greater than a preset pixel threshold among the absolute values ​​of the weighted pixel values, the weighted pixel value greater than the preset pixel threshold is updated to the preset pixel threshold.

[0046] According to a second aspect of an embodiment of the present application, there is provided an image processing apparatus, including:

[0047] A receiving module is configured to receive a change request, wherein the change request carries image information and target color data;

[0048] a determination module configured to determine a first weighted image corresponding to the image to be changed by performing style migration on the image to be changed corresponding to the image information, and generate an inverted image according to a pixel value of each pixel in the first weighted image;

[0049] A generating module, configured to generate a second weight image based on the pixel value of each pixel in the inverted image;

[0050] The conversion module is configured to convert the image to be changed into a target image according to the first weight image, the second weight image and the target color data.

[0051] According to a third aspect of an embodiment of the present application, a computing device is provided, comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the image processing method when executing the computer instructions.

[0052] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer instructions, and when the computer instructions are executed by a processor, the steps of the image processing method are implemented.

[0053] The image processing method provided by the embodiment of the present application receives a change request, wherein the change request carries image information and target color data, performs style transfer on the image to be changed corresponding to the image information, determines the first weight image corresponding to the image to be changed, and realizes the first weight image obtained by performing style transfer on the image to be changed, and clarifies the image area in the image to be changed that is easily affected by style transfer, and the image area that is not easily affected by style. Then, an inverted image is generated based on the pixel value of each pixel in the first weight image, and a second weight image is generated based on the pixel value of each pixel in the inverted image. Finally, the image to be changed is converted into a target image based on the first weight image, the second weight image and the target color data. The color of the image area in the image to be changed that is easily affected by style transfer is changed by the first weight image, and the color of the image area in the image to be changed that is not easily affected by style transfer is changed by the second weight image, thereby realizing the color change of the entire area in the image to be changed, so that the style change effect of the image to be changed is more in line with expectations. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a structural block diagram of a computing device provided in an embodiment of the present application;

[0055] Figure 2 is a flowchart of an image processing method provided by an embodiment of the present application;

[0056] Figure 3 is a schematic diagram of various images in the image processing method provided in the embodiment of the present application;

[0057] Figure 4 is a schematic diagram of image expansion in the image processing method provided in an embodiment of the present application;

[0058] Figure 5 is a processing flow chart of an image processing method applied in an actual scene provided by an embodiment of the present application;

[0059] Figure 6 It is a structural schematic diagram of the image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0061] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0062] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0063] First, the terms involved in one or more embodiments of the present invention are explained.

[0064] Gaussian Blur: Also known as Gaussian smoothing, it is an image blur filter that uses normal distribution to calculate the transformation of each pixel in the image. It is usually used to reduce image noise and reduce the level of detail.

[0065] Downsampling: that is, the number of sampling points is reduced. For an N*M image, if the downsampling factor is k, then every k points in each row and column of the original image are taken to form an image.

[0066] Upsampling: also known as interpolation. For images, it is two-dimensional interpolation. If the upsampling factor is k, k-1 points are inserted between points n and n+1 in the original image to form k points. Two-dimensional interpolation means that after each row is interpolated, each column is also interpolated.

[0067] POWER function: It returns the power of a given number. The syntax of the POWER function is: POWER(number, power), where the number parameter represents the base and the power parameter represents the exponent. Both parameters can be any real numbers.

[0068] Linear interpolation: refers to the interpolation method in which the interpolation function is a first-order polynomial, and the interpolation error at the interpolation node is zero. Compared with other interpolation methods, such as parabola interpolation, linear interpolation is simple and convenient. The geometric meaning of linear interpolation is to use a straight line passing through points A and B in the overview diagram to approximate the original function. Linear interpolation can be used to approximate the original function, and can also be used to calculate values ​​that are not in the table during the table lookup process.

[0069] Inversion processing: reverse the R, G, and B values ​​of the image. If the color quantization level is 256, subtract the R, G, and B values ​​from 255 respectively as the color of the image after inversion processing.

[0070] In the present application, an image processing method and apparatus, a computing device, and a computer-readable storage medium are provided, which are described in detail one by one in the following embodiments.

[0071] Figure 1 The structure block diagram of a computing device 100 according to an embodiment of the present application is shown. The components of the computing device 100 include but are not limited to a memory 110 and a processor 120. The processor 120 is connected to the memory 110 via a bus 130, and the database 150 is used to store data.

[0072] The computing device 100 also includes an access device 140 that enables the computing device 100 to communicate via one or more networks 160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 140 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0073] In one embodiment of the present application, the above components of the computing device 100 and Figure 1 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 1 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0074] The computing device 100 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 100 may also be a mobile or stationary server.

[0075] The processor 120 may execute Figure 2 Steps in the image processing method shown. Figure 2 A flowchart of an image processing method according to an embodiment of the present application is shown, which specifically includes the following steps:

[0076] Step 202: Receive a change request, wherein the change request carries image information and target color data.

[0077] Among them, the change request can be understood as a request to change the style of an image. The change request carries image information (such as image name, image identifier, image address, or the image itself, etc., which can uniquely identify an image). In addition, in order to change the color of the image, the change request also carries color data (i.e., target color data) used to change the color of the image. The target color data can include data composed of at least one color. For example, white, red, green, etc., are not limited here.

[0078] In specific implementation, in order to change the scenery, people or objects in the image to show different scenes (such as dark, snowy, rainy scenes), or to change the color preference of the image so that the changed image shows a different style (which can also be understood as a color change effect), a change interface for changing the image can be provided to the user, so that the user can submit a change request based on the interface. Based on the received change request, the style of the image to be changed is changed in response to the change request.

[0079] Taking the initial image as a ground map as an example, a change request is received, which carries the image identifier 001 of the ground map a and the target color data.

[0080] Step 204: performing style migration on the image to be changed corresponding to the image information, determining a first weighted image corresponding to the image to be changed, and generating an inverted image according to the pixel value of each pixel in the first weighted image.

[0081] Specifically, based on the above-mentioned change request, since the style transfer model is directly used to transfer the style of the image to be changed corresponding to the image information, there may be some image areas in the image to be changed that are not easily affected by the style transfer. In order to make the style (color) change of the image to be changed more thorough, it is necessary to first clarify which image areas in the image to be changed are easily affected by the style transfer and which image areas are not easily affected by the style transfer. Therefore, by performing style transfer on the image to be changed, it is determined which image areas in the image to be changed are easily affected by the style transfer and which image areas are not easily affected by the style transfer. Then obtain the first weight image corresponding to the image to be changed. So that the style of the image areas in the image to be changed that are easily affected by the style transfer can be changed through the first weight image.

[0082] Among them, the image to be changed can refer to an image of any format (such as jpg format, gif format, bmp format, png format, etc.) and any content (such as people, animals, scenery, objects, etc.), and is not limited here. Correspondingly, the first weight image is used to generate the weights needed to change the image style of the image to be changed. The weights can be understood as the weight expression for the image area in the image to be changed that is easily affected by style migration. In practical applications, the pixel values ​​of the pixels in the image to be changed can be calculated by performing preset operations, such as (averaging, or performing linear transformation, etc.). In specific implementation, each pixel in the first weight image can correspond to a weight. By changing the image to be changed through the first weight image, the image area in the image to be changed that is easily affected by style migration can be changed in color.

[0083] The inverse color image refers to an image generated by inverse color processing of the pixel value of each pixel in the first weighted image. For example, when the first weighted image is a grayscale image, the pixel value of pixel 1 in the first weighted image is 200, then the pixel value 200 is inversely processed, that is, calculated by 255-200, and the pixel value of the corresponding pixel in the inverse color image is 55. When the first weighted image is a color image, the inverse color is a color that can be turned into white when superimposed with the primary color, that is, the color of the primary color is removed by white (RGB: 255, 255, 255). For example, the inverse color of red (RGB: 255, 0, 0) is green (0, 255, 255). On the basis of obtaining the first weighted image, the pixel value of each pixel in the first weighted image is inversely processed to obtain the inverse color image.

[0084] In practical applications, when the style of the image to be changed is changed by using the corresponding weights in the first weight image, it is used to change the style of the areas in the image to be changed that are easy to change in color; and the inverted image of the first weight image is generated to facilitate the generation of another weight image (i.e., the second weight image) based on the inverted image, which is used to change the style of some color areas in the image to be changed that are not easy to change.

[0085] In specific implementation, in order to completely change the style of the image to be changed, it is necessary to clarify which image areas are easy to change the style and which image areas are not easy to change the style when changing the style of the image to be changed, so as to change the image to be changed in a targeted manner. Therefore, the style of the image to be changed can be transferred through a style transfer model, and the style transfer status of the image area in the image to be changed can be further determined based on the style transfer result, so as to change the image to be changed. The embodiment of the present application is specifically implemented in the following way:

[0086] Determining an image to be changed according to the image information, and determining a style image corresponding to the image to be changed;

[0087] Combining the style image with the image to be changed to generate an image pair;

[0088] Inputting the image pair into a style transfer model to obtain a composite image output by the style transfer model;

[0089] Performing grayscale processing on the image to be changed to generate a first grayscale image, and performing grayscale processing on the synthesized image to generate a second grayscale image;

[0090] Performing a subtraction operation based on the pixel values ​​of the pixels in the first grayscale image and the pixel values ​​of the pixels in the second grayscale image to obtain a difference grayscale image;

[0091] When the number of the difference grayscale images is one, using the difference grayscale image as a first weight map;

[0092] In the case that there are a plurality of difference grayscale images, an AND operation is performed based on the pixel values ​​of the pixels in each difference grayscale image to obtain the first weight image.

[0093] Among them, the style image refers to an image used to transfer (change) the style of the image to be changed. In specific implementation, the image to be changed and the style image are combined into an image pair and input into the style transfer model in order to make the style of the image to be changed tend toward the style image. The style transfer model refers to a model used to transfer the style of the image to be changed. The style transfer model can be a model built by an optimization-based method, a model built by a feedforward-based method, or a model built by a one network, multiple styles method, etc., and there is no limitation here.

[0094] In practical applications, there may be a situation where one or more style transfers are desired for the image to be changed. In this case, one or more style images can be pre-set for the image to be changed to change the style of the image to be changed. When there is one style image, the image to be changed and the style image can be directly combined to generate a set of image pairs; when there are multiple style images, the image to be changed and each style image can be combined separately to form multiple sets of image pairs, and each set of image pairs can be input into the style transfer model to perform style transfers of different styles on the image to be changed, and generate a synthetic image corresponding to each set of image pairs; in addition, the image to be changed and multiple style images can be combined into image pairs, and multiple styles can be directly transferred to the image to be changed through the style transfer model to generate a synthetic image corresponding to each style image. Specifically, the synthetic image refers to the image generated after the style transfer process. The synthetic image has the image content of the image to be changed and the image style of the style image.

[0095] On the basis of obtaining the composite image, the image to be changed is further gray-processed to generate a gray-scale image (i.e., the first gray-scale image), and the composite image is gray-processed to generate a gray-scale image (i.e., the second gray-scale image). This is because color images are rich in color, but it is not easy to measure the difference of the image to be changed. Therefore, it is necessary to process the image to be changed and the composite image into gray-scale images respectively, and then calculate the difference between the image to be changed and the composite image through the gray-scale images.

[0096] Among them, grayscale processing specifically refers to graying an image to generate a grayscale image. Specifically, the grayscale processing methods include: component method (averaging the brightness of the RGB three components of the pixel points in the color image to obtain a grayscale image), maximum value method (taking the maximum value of the brightness of the RGB three components in the color image as the grayscale value of the grayscale image), average method (the algorithm idea is to first take the value of each RGB component as the grayscale value of the image, so that three grayscale images of the original image are obtained, and then one of the three grayscale images is selected, that is, one of the three RGB components of a pixel point is used as the grayscale value of the point) and weighted average method (according to certain conditions, the three components of each pixel point are weighted averaged with different weights to obtain the grayscale value of each pixel point), etc. During specific implementation, any grayscale processing method can be selected according to the needs of the actual scene, and there is no limitation here. Specifically, the grayscale interval range of the grayscale image is between 0-255.

[0097] In specific implementation, the pixel values ​​of the pixels in the first grayscale image are subtracted from the pixel values ​​of the corresponding pixels in the second grayscale image, and the degree of difference between the image to be changed and the synthesized image can be determined based on the difference between the two. The larger the difference, the more obvious the difference between the synthesized image and the image to be changed, and the easier it is to change the style of the image area in the image to be changed. In practical applications, there may be a situation where the difference between the two is a negative number. In this case, the absolute value of the difference needs to be calculated to obtain the updated difference. These differences are then combined according to the arrangement of the pixels in the image to be changed to obtain a difference grayscale image.

[0098] Furthermore, when the number of the difference grayscale images is one, the difference grayscale image can reflect, in the process of style transfer for the image to be changed, which image areas are easy to change in style and which image areas are not easy to change in style. Therefore, the difference grayscale image is directly used as the first weight image to perform style change on the image areas in the image to be changed that are easy to change in style.

[0099] In the case where there are multiple difference grayscale images, since each difference grayscale image reflects which image areas are easy to change in style and which image areas are not easy to change in style in the corresponding style image in the corresponding style transfer of the image to be changed. In order to determine the image areas that are generally recognized to be easy to change in style and the image areas that are not easy to change in style in these style transfers. An AND operation can be performed based on the pixel values ​​of the pixels in each difference grayscale image. Specifically, the AND operation refers to converting the pixel values ​​of the pixels in each difference grayscale image into binary numbers, and then performing bitwise AND on the pixel values ​​of the pixels at the same position in different difference grayscale images, and then converting the binary numbers after the AND operation into decimal numbers. These decimal numbers are then combined according to the arrangement of the pixels in the difference grayscale image to obtain the first weight image.

[0100] Among them, the closer the grayscale value of the pixel point in the first weight image is to 255, it means that the image area corresponding to the grayscale value in the image to be changed is easier to change the style; the smaller the grayscale value, the less likely it is to change the style of the image area corresponding to the grayscale value in the image to be changed.

[0101] Using the above example, according to the image identifier 001 of the surface map a, the surface map a corresponding to the image identifier 001 is determined. The surface map a is specifically as follows: Figure 3 As shown in (a). And according to the image identifier 001 of the surface map a, query the style image corresponding to the image to be changed, and determine that the style images corresponding to the surface map a include style image 1, style image 2, style image 3 and style image 4. The surface map a and style image 1 form an image pair a1, the surface map a and style image 2 form an image pair a2, the surface map a and style image 3 form an image pair a3, and the surface map a and style image 4 form an image pair a4. Input the image pair a1 into the pre-trained style transfer model M to obtain the synthetic image c1 output by the style transfer model M; input the image pair a2 into the pre-trained style transfer model M to obtain the synthetic image c2 output by the style transfer model M; input the image pair a3 into the pre-trained style transfer model M to obtain the synthetic image c3 output by the style transfer model M; input the image pair a4 into the pre-trained style transfer model M to obtain the synthetic image c4 output by the style transfer model M;

[0102] The surface map a is gray-scaled. Specifically, the surface map a includes n pixels, and the pixel values ​​of these pixels (from top to bottom, from left to right) are: P1 = (RGB: pixel1_1, pixel1_2, pixel1_3), P2 = (RGB: pixel2_1, pixel2_2, pixel2_3), ..., P n=(RGB:pixel n _1, pixel n _2, pixel n _3) Calculate the gray value P corresponding to each pixel by averaging the RGB values ​​in each pixel. gg1 , P gg2 ,……,P ggn , the grayscale values ​​corresponding to these pixels are combined according to the arrangement of the pixels in the surface map a to generate a first grayscale image g;

[0103] Similarly, grayscale processing is performed on the composite images c1, c2, c3, and c4, respectively, to obtain a second grayscale image g1 corresponding to the composite image c1, a second grayscale image g2 corresponding to the composite image c2, a second grayscale image g3 corresponding to the composite image c3, and a second grayscale image g4 corresponding to the composite image c4, wherein the pixel values ​​corresponding to the pixel points in the second grayscale image g1 (from top to bottom, from left to right) are: P g11 , P g12 ,……,P g1n ; The pixel values ​​corresponding to the pixel points in the second grayscale image g2 (from top to bottom, from left to right) are: P g21 , P g22 ,……,P g2n ; The pixel values ​​corresponding to the pixel points in the second grayscale image g3 (from top to bottom, from left to right) are: P g31 , P g32 ,……,P g3n ; The pixel values ​​corresponding to the pixel points in the second grayscale image g4 (from top to bottom, from left to right) are: P g41 , P g42 ,……,P g4n ;

[0104] Subtract the pixel value of the pixel point in the first grayscale image g from the pixel value of the corresponding pixel point in the second grayscale image g1 to obtain a difference grayscale image dg1. The pixel values ​​of the pixel points in the difference grayscale image dg1 are: dg11 =(P gg1 -P g11 ), P dg12 =(P gg2 -P g12 ),……,P dg1n =(P ggn -P g1n ); Similarly, the pixel values ​​of the pixels in the first grayscale image g are subtracted from the pixel values ​​of the corresponding pixels in the second grayscale image g2 to obtain a difference grayscale image dg2, and the pixel values ​​of the pixels in the difference grayscale image dg2 are: dg21 =(Pgg1 -P g21 ), P dg22 =(P gg2 -P g22 ),……,P dg2n =(P ggn -P g2n );Subtract the pixel value of the pixel point in the first grayscale image g from the pixel value of the corresponding pixel point in the second grayscale image g3 to obtain a difference grayscale image dg3, and the pixel values ​​of the pixel points in the difference grayscale image dg3 are: dg31 =(P gg1 -P g31 ), P dg32 =(P gg2 -P g32 ),……,P dg3n =(P ggn -P g3n );Subtract the pixel value of the pixel point in the first grayscale image g from the pixel value of the corresponding pixel point in the second grayscale image g4 to obtain a difference grayscale image dg4, and the pixel values ​​of the pixel points in the difference grayscale image dg4 are: dg41 =(P gg1 -P g41 ), P dg42 =(P gg2 -P g42 ),……,P dg4n =(P ggn -P g4n ), in specific implementation, if the subtraction result is a negative number, the absolute value of the subtraction result is taken, and in addition, the absolute value of all subtraction results can be directly taken;

[0105] The pixel values ​​of the pixels in the difference grayscale images dg1, dg2, dg3, and dg4 are converted from decimal numbers to 8-bit binary numbers, and the binary numbers corresponding to the pixels at the same position in the four difference grayscale images are subjected to a phase AND operation. Specifically, only when the same binary bit in the binary bits of the corresponding pixels is 1, the phase AND operation result of the binary bit is 1, otherwise the phase AND operation result is 0. After the phase AND operation is completed, the operation result of the phase AND operation is converted to a decimal number, and the converted decimal numbers corresponding to each pixel are combined according to the original arrangement to generate a first weighted image, wherein the gray value P corresponding to each pixel in the first weighted image is g1 , P g2 ,……,P gn , the first weight image is as follows Figure 3 (c) as shown.

[0106] Furthermore, the pixel value of each pixel in the first weighted image is inverted, that is, the inverted value corresponding to each pixel is calculated in the following manner: r1 =(255-P g1 ), P r2 =(255-P g1 ),……,P rn =(255-P gn ), and then P r1 , P r2 ,……,P rn The n inverse color values ​​are combined according to the arrangement of the pixels in the first weighted image to form an inverse color image.

[0107] In summary, the image to be changed is style-transferred, and based on the difference between the composite image formed after the style transfer and the grayscale image of the image to be changed, it is determined which areas in the image to be changed are easy to style-transferred and which areas are not easy to style-transferred. In the case of performing multiple style transfers, the first weighted image obtained by performing an AND operation on the difference grayscale images determines the image areas that are recognized to be easy to style-transferred and the image areas that are not easy to style-transferred in multiple style transfers.

[0108] In actual applications, the aspect ratio of the image to be changed may be equal. In this case, when the image to be changed is applied after the style change, it may be necessary to expand the image after the style change. Directly changing the style of the image to be changed may cause discontinuity after the style change is expanded. Therefore, when the aspect ratio of the image to be changed is equal, the image to be changed is first expanded in four directions continuously, and then the style change is performed on the image to be changed. In the embodiment of the present application, before grayscale processing is performed on the image to be changed to generate the first grayscale image, it is specifically implemented in the following manner:

[0109] Determining an image to be changed according to the image information, and determining a style image corresponding to the image to be changed;

[0110] When the length and width of the image to be changed are equal, the image to be changed is continuously expanded in four directions to obtain an expanded image to be changed;

[0111] Combining the expanded image to be changed with the style image to generate an expanded image pair;

[0112] Inputting the extended image pair into a style transfer model to obtain an extended synthetic image output by the style transfer model;

[0113] According to the image size of the image to be changed, a clipped image to be changed is obtained from the expanded image to be changed, and the clipped image to be changed is used as the image to be changed, and,

[0114] The extended composite image is cut out according to the image size to obtain a composite image.

[0115] Specifically, the length and width of the image to be changed are equal, which means that the ratio of the long side to the wide side of the image to be changed is 1:1; four-way continuous extension means that the image to be changed is used as a unit pattern to be repeatedly, continuously and extended in all directions (including eight directions: up, down, left, right, upper left, upper right, lower left and lower right). Figure 4 As shown, after the image to be changed is processed continuously in four directions (i.e., expanded in four directions) in eight directions, namely, up, down, left, right, upper left, upper right, lower left and lower right, an expanded image having nine sizes of the image to be changed is generated.

[0116] During the specific implementation, the specific implementation method of generating the extended image pair is similar to the specific implementation method of generating the image pair mentioned above. You can refer to the specific implementation of generating the image pair mentioned above and will not elaborate on it here. In addition, the specific implementation of outputting the extended synthetic image is similar to the specific implementation of outputting the synthetic image mentioned above. You can refer to the specific implementation of outputting the synthetic image mentioned above and will not elaborate on it here.

[0117] Furthermore, since the image to be changed is expanded, the image size of the composite image after style transfer of the expanded image to be changed is also expanded. In this case, in order to restore the various images after style transfer to their original image sizes, the image to be changed can be cropped in the expanded image to be changed according to the original image size (i.e., image size) before expansion, to obtain a cropped image to be changed with the same size as the original image of the image to be changed, and the cropped image to be changed is used as the image to be changed (specifically, it refers to replacing the image to be changed with the cropped change image, that is, discarding the original image to be changed and using the cropped change image as the image to be changed). Similarly, the image to be changed can be cropped in the expanded composite image according to the original image size (i.e., image size) before expansion, to obtain a composite image with the same size as the original image of the image to be changed.

[0118] Using the above example, the length and width of the surface map a are both 1024 pixels in size. The surface map a is continuously expanded in four directions to obtain an expanded surface map a with a length and width of 3072 pixels. Specifically, the expanded surface map a is as follows: Figure 3(b) is shown. The extended surface map a and style image 1 form an extended image pair a1, the extended surface map a and style image 2 form an extended image pair a2, the extended surface map a and style image 3 form an extended image pair a3, and the extended surface map a and style image 4 form an extended image pair a4. The extended image pair a1 is input into the pre-trained style transfer model M to obtain the extended synthetic image c1 output by the style transfer model M; the extended image pair a2 is input into the pre-trained style transfer model M to obtain the extended synthetic image c2 output by the style transfer model M; the extended image pair a3 is input into the pre-trained style transfer model M to obtain the extended synthetic image c3 output by the style transfer model M; the extended image pair a4 is input into the pre-trained style transfer model M to obtain the extended synthetic image c4 output by the style transfer model M, wherein the length and width of the extended synthetic image c1, the extended synthetic image c2, the extended synthetic image c3, and the extended synthetic image c4 are all 3072 pixels in size.

[0119] In the expanded surface map a, a cut-off surface map with a size of 1024 pixels in length and width in the middle area is captured as the surface map a, and in the expanded composite image c1, a composite image c1 with a size of 1024 pixels in length and width in the middle area is captured; in the expanded composite image c2, a composite image c2 with a size of 1024 pixels in length and width in the middle area is captured; in the expanded composite image c3, a composite image c3 with a size of 1024 pixels in length and width in the middle area is captured; in the expanded composite image c4, a composite image c4 with a size of 1024 pixels in length and width in the middle area is captured.

[0120] In summary, when the aspect ratio of the image to be changed is the same, by continuously expanding the image to be changed and the style image in four directions and then performing style transfer, it is ensured that after the style of the image to be changed is changed, if the changed image is expanded and applied, the image continuity in the expanded application is strong.

[0121] On the basis of generating the difference grayscale image, if the difference grayscale image is directly used to change the change image without any preprocessing, the contour of the changed image may be sharp, and the changed image cannot show a smooth or gradual effect. Therefore, in order to avoid the influence of the above problems, the generated difference grayscale image can be blurred. In the embodiment of the present application, it is specifically implemented in the following manner:

[0122] The difference grayscale image is blurred to obtain a first blurred grayscale image, and the first blurred grayscale image is used as the difference grayscale image.

[0123] In specific implementation, there are various ways of blurring, such as blurring by mean blurring, Gaussian blurring, median filtering, and bilateral filtering, which are not limited here.

[0124] Continuing with the above example, on the basis of generating the difference grayscale image dg1, the difference grayscale image dg2, the difference grayscale image dg3 and the difference grayscale image dg4, Gaussian blur processing is performed on the difference grayscale image dg1, the difference grayscale image dg2, the difference grayscale image dg3 and the difference grayscale image dg4 respectively to obtain blurred grayscale image dg1, blurred grayscale image dg2, blurred grayscale image dg3 and blurred grayscale image dg4, and these four blurred grayscale images are used as difference grayscale images.

[0125] In summary, by first blurring the difference grayscale image and then performing subsequent processing on the image to be changed through the blurred difference grayscale image, the edge of the image obtained after the change of the image to be changed can be made smoother and a color gradient effect can be achieved.

[0126] In addition to the above-mentioned blurring of the difference grayscale image, image scaling can also be combined with blurring to avoid the problem that the image to be changed after subsequent change processing using the difference grayscale image is still relatively sharp and the color gradient is not obvious. After the pixel value of the pixel point in the first grayscale image is subtracted from the pixel value of the pixel point in the second grayscale image to obtain the difference grayscale image, the method further includes:

[0127] The difference grayscale image is downsampled to generate a reduced grayscale image; the reduced grayscale image is blurred to obtain a second blurred grayscale image; the second blurred grayscale image is upsampled to generate an enlarged grayscale image, and the enlarged grayscale image is used as the difference grayscale image.

[0128] Among them, downsampling can also be called downsampling. In specific implementation, the downsampling coefficient can be preset. Assuming that the downsampling coefficient is k, a pixel point is taken every k pixels in each row and column of the difference grayscale image. And the reduced grayscale image is generated through the obtained pixel points.

[0129] On the basis of downsampling the difference grayscale image, the downsampled reduced grayscale image is blurred to obtain a second blurred grayscale image. In specific implementation, the specific implementation of blurring the reduced grayscale image is similar to the specific implementation of blurring the difference grayscale image mentioned above, and the specific implementation of blurring the difference grayscale image mentioned above can be referred to. On the basis of obtaining the second blurred grayscale image, in order to generate a grayscale image of the same size as the original difference grayscale image, the second blurred grayscale image can be upsampled to generate an enlarged grayscale image (i.e., an enlarged grayscale image), and the enlarged grayscale image is used as the difference grayscale image (specifically, it means replacing the difference grayscale image with the enlarged grayscale image, that is, discarding the original difference grayscale image and using the enlarged grayscale image as the difference grayscale image).

[0130] Among them, upsampling can also be called upsampling. In specific implementation, the upsampling coefficient can be preset. Assuming that the upsampling coefficient is k, the grayscale image to be enlarged can be generated by inserting k-1 pixels into every two adjacent pixels in the second blurred grayscale image. In specific implementation, the pixel values ​​of the inserted k-1 pixels can be the same as or different from the pixel values ​​of the adjacent pixels, which is not limited here.

[0131] Continuing with the above example, Gaussian blur processing is performed on the difference grayscale image dg1, the difference grayscale image dg2, the difference grayscale image dg3 and the difference grayscale image dg4 respectively to obtain blurred grayscale image dg1, blurred grayscale image dg2, blurred grayscale image dg3 and blurred grayscale image dg4, and these four blurred grayscale images are used as difference grayscale images.

[0132] Using the above example, based on the above generation of the difference grayscale image dg1, the difference grayscale image dg2, the difference grayscale image dg3 and the difference grayscale image dg4, the difference grayscale image dg1, the difference grayscale image dg2, the difference grayscale image dg3 and the difference grayscale image dg4 are respectively downsampled, wherein the downsampling coefficient is 4, specifically, a pixel is obtained by taking one pixel for each of the four pixels in the difference grayscale image dg1, and then the obtained pixels are combined to form a reduced grayscale image dg1. The length and width of the reduced grayscale image dg1 are both 256 pixels in size. Similarly, after the difference grayscale image dg2, the difference grayscale image dg3 and the difference grayscale image dg4 are downsampled, the reduced grayscale image dg2, the reduced grayscale image dg3 and the reduced grayscale image dg4 are generated. After performing Gaussian blur processing on the reduced grayscale image dg1, the reduced grayscale image dg2, the reduced grayscale image dg3 and the reduced grayscale image dg4 respectively, the second blurred grayscale image dg1, the second blurred grayscale image dg2, the second blurred grayscale image dg3 and the second blurred grayscale image dg4 are obtained, and these four second blurred grayscale images are used as difference grayscale images to perform the subsequent step of determining the first weight image.

[0133] In summary, blurring is performed on the basis of downsampling the difference grayscale image, and the second blurred image after blurring is upsampled to generate an enlarged grayscale image, and the enlarged grayscale image is used as the difference grayscale image, so that the edge of the image obtained after changing the image to be changed is smoother and the color change effect is stronger.

[0134] Step 206: Generate a second weight image based on the pixel value of each pixel in the inverted image.

[0135] Specifically, based on the inverted image obtained above, a second weight image is determined according to the inverted image, so that the background color of the entire area of ​​the image to be changed can be changed based on the two weight images.

[0136] Among them, the second weight image is also used to generate the weights needed to be used in the image conversion process. However, the weight can be understood as a weight expression for the image area that is not easily affected by style transfer in the image to be changed. In specific implementation, each pixel in the second weight image can correspond to a weight. The second weight image is obtained to change the color of the image area that is not easily affected by style transfer in the image to be changed.

[0137] In specific implementation, since the second weight image is used to generate weights for changing the color of the image area that is not easily affected by style transfer in the image to be changed, the pixel values ​​of the pixels in the second weight image have a greater impact on the color change of the image to be changed. Therefore, it is necessary to pre-set an algorithm (i.e., a preset algorithm) for generating the pixel values ​​of the pixels in the second weight image according to actual needs, so as to change the color of the image to be changed as expected, which is specifically implemented in the following manner:

[0138] Calculate the pixel value of each pixel in the inverted image based on a preset algorithm to obtain a weighted pixel value corresponding to each pixel in the inverted image;

[0139] A second weight image is generated based on the weight pixel values.

[0140] The preset algorithm refers to an algorithm used to calculate the pixel value of each pixel in the inverted image. In practical applications, the preset algorithm may be a nonlinear algorithm, which is mainly set according to actual needs (how to change the color of the image area that is not easily affected by style transfer in the image to be changed), and is not limited here.

[0141] In specific implementation, through calculation, the pixel value corresponding to each pixel in the inverted image can be changed to obtain the changed pixel value corresponding to each pixel, that is, the weighted pixel value, and then the weighted pixel value is combined according to the arrangement of the pixels in the inverted image to generate a second weighted image.

[0142] Optionally, the preset algorithm may be an algorithm for performing a power operation on the pixel values ​​of the pixel points in the inverted image. Specifically, the preset algorithm may be as shown in the following formula 1:

[0143] P wi =POWER(P ri , a); Formula 1

[0144] Wherein, i and a are positive integers, i is less than or equal to n, n is the number of pixels in the inverted image, P ri is the pixel value of the i-th pixel in the inverted image, P wi The weighted pixel value corresponding to the i-th pixel in the inverted image. The main function of the POWER function is to return a given value P ri Raise a to the power of .

[0145] Using the above example, based on the above inverted image, the pixel value of each pixel in the inverted image is: r1 , P r2 ,……,P rn, when a is 3 in formula 1, perform a cubic operation to obtain the weighted pixel value corresponding to each pixel: P w1 , P w2 ,……,P wn Then, these n weighted pixel values ​​are combined according to the arrangement of the pixels in the inverted image to generate a second weighted image.

[0146] In summary, by performing a preset Power operation on the pixel values ​​of the pixels in the inverted image, a second weighted image is obtained. The second weighted image can be used to change the color of the image area that is not easily affected by style transfer in the image to be changed, thereby obtaining a better color change effect.

[0147] After the weighted pixel value is generated as described above, considering that the generated weighted pixel value may exceed the range of the normal pixel value [0-255], in order to ensure that the weighted pixel value is within the range of the normal pixel value, the embodiment of the present application is specifically implemented in the following manner:

[0148] In the case that there is a weighted pixel value greater than a preset pixel threshold among the absolute values ​​of the weighted pixel values, the weighted pixel value greater than the preset pixel threshold is updated to the preset pixel threshold.

[0149] The preset pixel threshold can be understood as 255. In addition, the preset pixel threshold can also take other values ​​between 0 and 255. During specific implementation, it can be adjusted according to actual needs and is not limited here.

[0150] Continuing with the above example, assume that the weighted pixel value corresponding to each pixel in the inverted image is: w1 , P w2 ,……,P wn In, P w2 If the value of is greater than 255, P w2 Update to 255. Assume that after the preset operation is performed on the pixel value of each pixel in the inverted image, the weighted pixel value P w6 If the value is less than 0, the P w6 Updated to 0.

[0151] In summary, by updating the weighted pixel values ​​whose absolute values ​​are greater than the preset pixel threshold value to the first pixel threshold value, the rationality of the pixel values ​​of the pixel points in the second weighted image is guaranteed.

[0152] Step 208: Convert the image to be changed into a target image according to the first weight image, the second weight image and the target color data in the change request.

[0153] Specifically, based on the second weighted image obtained as described above, the color of the image to be changed is converted according to the two weighted images, so that the background color of the image to be changed can achieve an overall color change effect.

[0154] In specific implementation, since the first weight image and the second weight image are used for changing weights, it is necessary to first convert the first weight image and the second weight image into weights, and then use the converted weights to change the style of the image to be changed. In the embodiment of the present application, the following method is specifically adopted:

[0155] Normalizing the pixel value of each pixel in the first weight image to obtain a first weight corresponding to each pixel in the first weight image, and normalizing the pixel value of each pixel in the second weight image to obtain a second weight corresponding to each pixel in the second weight image;

[0156] The image to be changed is converted into a target image according to the first weight, the second weight and the target color data.

[0157] The normalization process is to map the pixel value of each pixel in the first weight image / the second weight image to the range of (0, 1), thereby forming a corresponding weight value. In specific implementation, the normalization method can be implemented by a linear function or a sigmoid function, which is not limited here.

[0158] Correspondingly, the pixel value of each pixel in the first weight image is normalized to generate the first weight, and the pixel value of each pixel in the second weight image is normalized to generate the second weight. On the basis of generating the two weights, the two weights are combined with the target color data in the change request to change the image to be changed. In specific implementation, the above data can be calculated by a pre-set linear interpolation algorithm to achieve the change of the image to be changed to generate the changed image, that is, the target image.

[0159] Using the above example, based on the first weighted image and the second weighted image obtained above, the pixel value P of each pixel in the first weighted image is g1 , P g2 ,……,P gn Normalize them respectively to obtain the first weight corresponding to each pixel: W1_1, W1_2, ..., W1_n. Then calculate the pixel value P of each pixel in the second weight image. w1 , P w2 ,……,P wnNormalization is performed respectively to obtain the second weight corresponding to each pixel: W2_1, W2_2, ..., W2_n. Then, according to the first weight, the second weight and the target color data in the change request for the image a to be changed, the style of the image a to be changed is changed to obtain the target image a.

[0160] Furthermore, since the first weight and the second weight can respectively affect the change of different regions in the image to be changed, the image to be changed is changed in two stages by using the first weight and the second weight respectively, so as to achieve the overall background color change of the image to be changed. In the first optional implementation manner provided in the embodiment of the present application, it is specifically implemented in the following manner:

[0161] Determining, in the target color data, first color data for changing the color of the image to be changed, and second color data for changing the color of the changed first initial changed image;

[0162] Calculating the first weight, the first color data, and the pixel value of each pixel in the image to be changed according to a first color conversion algorithm to generate the first initial changed image;

[0163] The second weight, the second color data, and the pixel value of each pixel in the first initial changed image are calculated according to a second color conversion algorithm to generate a target image.

[0164] The first color data refers to the color of the first stage change of the image to be changed, and the second color data refers to the color of the second stage change of the initial change image (first initial change image) generated after the image to be changed is changed in the first stage.

[0165] Accordingly, the first color conversion algorithm refers to an algorithm for performing a first-stage change on the image to be changed, in which the first weight, the first color data, and the pixel value of the pixel point in the image to be changed are used for calculation. In specific implementation, the first color conversion algorithm can be shown in the following formula 2:

[0166] P ci 1=P i *W1_i+P c 1*(1-W1_i); Formula 2

[0167] Among them, P ci 1 is the pixel value of the i-th pixel in the first initial changed image; P i is the pixel value of the i-th pixel in the image to be changed; W1_i is the i-th first weight, P c 1 is the first color data. In practical applications, i and P c1Usually contains three components of RGB. In this case, the above formula can be calculated for each component separately, such as P i The R value in is P ri , P c The R value in 1 is P cr 1, then P ci The R value in is P cri =P ri *W1_i+P cr 1*(1-W1_i).

[0168] Similarly, the second color conversion algorithm refers to an algorithm for performing a second stage change on the first initial change image after the image to be changed is changed, and the second weight, the second color data, and the pixel value of the pixel point in the first initial change image are used for calculation in the algorithm. In specific implementation, the second color conversion algorithm can be shown as the following formula 3:

[0169] P pi =P ci1 *W2_i+P c 2*(1-W2_i); Formula 3

[0170] Among them, P ci 1 is the pixel value of the i-th pixel in the first initial changed image; P pi is the pixel value of the i-th pixel in the target image; W2_i is the i-th second weight, P c 2 is the second color data.

[0171] It should be noted that the first color data and the second color data may be the same color.

[0172] Using the above example, based on the first weights W1_1, W1_2, ..., W1_n and the second weights W2_1, W2_2, ..., W2_n, it is assumed that the first color data P c 1 is brick red, the second color data P c 2 is white, firstly, the pixel value of the first weight, brick red and the pixel value of the pixel point in the image a to be changed is calculated by the above formula 3 to obtain the pixel value: c1 1. P c2 1, ..., P cn 1, and then the arrangement of each pixel point in the image to be changed is combined into a first initial changed image; wherein the first initial changed image is an image after the overall background color of the image to be changed is changed to brick red. Specifically, the first initial changed image can be as follows Figure 3 Then, the second weight, white, and the pixel value of each pixel in the first initial change image are calculated by the above formula 3 to obtain the pixel value P of each pixel in the target image.p1 , P p2 ,……,P pn , and then combine the pixel values ​​of these pixels according to the arrangement of each pixel in the image to be changed to generate a target image. Specifically, the target image can be as follows Figure 3 (e) as shown.

[0173] In summary, the first weight, the first color data, and the pixel value of each pixel in the image to be changed are calculated according to the first color conversion algorithm to generate the first initial changed image; and the second weight, the second color data, and the pixel value of each pixel in the first initial changed image are calculated according to the second color conversion algorithm to generate the target image. The color of the image area that is easily affected by the style transfer in the image to be changed is first changed, and then the color of the image area that is not easily affected by the style transfer in the changed image is changed. The style transfer (color change) of the entire area of ​​the image to be changed is achieved.

[0174] In addition to the above-mentioned method of changing the image to be changed, the second optional implementation mode provided in the embodiment of the present application is specifically implemented in the following manner:

[0175] Determining, in the target color data, third color data for changing the color of the image to be changed, and fourth color data for changing the color of the changed second initial changed image;

[0176] Calculating the second weight, the third color data, and the pixel value of each pixel in the image to be changed according to a third color conversion algorithm to generate the second initial changed image;

[0177] The first weight, the fourth color data, and the pixel value of each pixel in the second initial changed image are calculated according to a fourth color conversion algorithm to generate a target image.

[0178] The third color data refers to the color of the first stage change of the image to be changed. The fourth color data refers to the color of the second stage change of the initial change image (second initial change image) generated after the image to be changed is changed in the first stage.

[0179] Accordingly, the third color conversion algorithm refers to an algorithm for performing the first stage change on the image to be changed, in which the second weight, the third color data, and the pixel value of the pixel point in the image to be changed are used for calculation. In specific implementation, the third color conversion algorithm can be shown as the following formula 4:

[0180] P ci 2=P c 3*W2_i+P i*(1-W2_i); Formula 4

[0181] Among them, P ci 2 is the pixel value of the i-th pixel in the second initial changed image; P i is the pixel value of the i-th pixel in the image to be changed; W2_i is the i-th second weight, P c 3 is the third color data.

[0182] Similarly, the second color conversion algorithm refers to an algorithm for performing a second stage change on the first initial change image after the change image is changed, and the second weight, the second color data, and the pixel value of the pixel point in the first initial change image are used for calculation in the algorithm. In specific implementation, the second color conversion algorithm can be shown as the following formula 5:

[0183] P pi =P c 4*W2_i+P ci 2*(1-W2_i); Formula 5

[0184] Among them, P ci 2 is the pixel value of the i-th pixel in the second initial changed image; P pi is the pixel value of the i-th pixel in the target image; W2_i is the i-th second weight, P c 4 is the fourth color data.

[0185] It should be noted that the third color data and the fourth color data may also be the same color.

[0186] Using the above example, based on the first weights W1_1, W1_2, ..., W1_n and the second weights W2_1, W2_2, ..., W2_n, it is assumed that the third color data P c 3 is brick red, the fourth color data P c 4 is white, firstly, the second weight, brick red and the pixel value of the pixel point in the image to be changed a are calculated by the above formula 4 to obtain the pixel value of each pixel point in the second initial changed image: c1 1. P c2 1, ..., P cn 1; wherein the second initial change image is an image after the overall background color of the image to be changed is changed to brick red. Then, the pixel value of each pixel point in the first weight, white, and the second initial change image is calculated by the above formula 5 to obtain the pixel value P of each pixel point in the target image. p1 , P p2 ,……,P pn , and then combine the pixel values ​​of these pixels according to the arrangement of each pixel in the image to be changed to generate the target image.

[0187] In summary, the second initial changed image is generated by calculating the second weight, the third color data, and the pixel value of each pixel in the image to be changed according to the third color conversion algorithm; and the target image is generated by calculating the first weight, the fourth color data, and the pixel value of each pixel in the second initial changed image according to the fourth color conversion algorithm. The color of the image area that is not easily affected by style transfer in the image to be changed is first changed, and then the color of the image area that is easily affected by style transfer in the changed image is changed. And the style transfer (color change) of the entire image to be changed is realized.

[0188] In summary, the image processing method provided by the embodiment of the present application receives a change request, wherein the change request carries image information and target color data, performs style transfer on the image to be changed corresponding to the image information, determines the first weight image corresponding to the image to be changed, and realizes the first weight image obtained by performing style transfer on the image to be changed, and clarifies the image area in the image to be changed that is easily affected by style transfer, and the image area that is not easily affected by style. Then, an inverted image is generated based on the pixel value of each pixel in the first weight image, and a second weight image is generated based on the pixel value of each pixel in the inverted image. Finally, the image to be changed is converted into a target image based on the first weight image, the second weight image and the target color data. The color change of the image area in the image to be changed that is easily affected by style transfer is realized by the first weight image, and the color change of the image area in the image to be changed that is not easily affected by style transfer is realized by the second weight image, thereby realizing the color change of the entire area in the image to be changed, so that the style change effect of the image to be changed is more in line with expectations.

[0189] Figure 5 A processing flow chart of an image processing method applied in an actual scene provided by an embodiment of the present application is shown. The updating method is described by taking an actual scene as an example, and specifically includes the following steps:

[0190] Step 502: Receive a change request, wherein the change request carries image identification information and target color data.

[0191] Specifically, the change request carries image identification information 001, and target color data includes yellow and green.

[0192] Step 504: Determine the image b1 to be changed according to the image identification information, and determine the style image s1 corresponding to the image b1 to be changed.

[0193] Specifically, by querying the correspondence between the image identification information and the image, the image b1 to be changed corresponding to the image identification information 001 can be determined according to the image identification information 001. Then, the style image s1 corresponding to the image b1 to be changed is obtained.

[0194] Step 506: When the length and width of the image to be changed b1 are equal, the image to be changed b1 is continuously expanded in four directions to obtain the expanded image to be changed b2.

[0195] Step 508: Combine the expanded image to be changed b2 with the style image s1 to generate an expanded image pair.

[0196] Step 510: Input the extended image pair into the style transfer model to obtain the extended composite image cs1 output by the style transfer model.

[0197] Step 512: According to the image size of the image to be changed b1, intercept the image to be changed in the expanded image to be changed b2 to obtain the intercepted image to be changed b3, use the intercepted image to be changed b3 as the image to be changed b1, and intercept the image to be changed in the expanded composite image cs1 according to the image size to obtain the composite image cs2.

[0198] Specifically, taking the intercepted image to be changed b3 as the image to be changed b1 can be understood as discarding the original image to be changed b1 and taking the intercepted image to be changed b3 as the image to be changed b1.

[0199] Step 514: Perform grayscale processing on the image to be changed b1 to generate a first grayscale image g1, and perform grayscale processing on the synthesized image cs2 to generate a second grayscale image g2.

[0200] Step 516: Perform a subtraction operation based on the pixel values ​​of the pixels in the first grayscale image g1 and the pixel values ​​of the pixels in the second grayscale image g2 to obtain a difference grayscale image d1.

[0201] Step 518: Downsample the difference grayscale image d1 to generate a reduced grayscale image d2.

[0202] Step 520: blur the reduced grayscale image d2 to obtain a second blurred grayscale image d3.

[0203] Step 522: Upsampling the second blurred grayscale image d3 to generate an enlarged grayscale image d4, and using the enlarged grayscale image d4 as the difference grayscale image d1.

[0204] Similarly, using the enlarged grayscale image d4 as the difference grayscale image d1 can also be understood as discarding the original difference grayscale image d1 and using the enlarged grayscale image d4 as the difference grayscale image.

[0205] Step 524: When the number of the difference grayscale image d1 is one, use the difference grayscale image d1 as the first weight image.

[0206] Step 526: Invert the pixel value of each pixel in the first weighted image to generate an inverted image.

[0207] Step 528: Calculate the pixel value of each pixel in the inverted image based on a preset algorithm to obtain a weighted pixel value corresponding to each pixel in the inverted image.

[0208] Specifically, the preset algorithm may be as shown in Formula 1 in the above method embodiment.

[0209] Step 530: Generate a second weight image based on the weight pixel values.

[0210] Specifically, generating the second weighted image based on the weighted pixel values ​​refers to combining the weighted pixel values ​​according to the arrangement of each pixel point in the inverted color image to generate the second weighted image.

[0211] Step 532: Normalize the pixel value of each pixel in the first weight image to obtain the first weight corresponding to each pixel in the first weight image, and normalize the pixel value of each pixel in the second weight image to obtain the second weight corresponding to each pixel in the second weight image.

[0212] Step 534: Determine, in the target color data of the change request, first color data for changing the color of the image to be changed b1 and second color data for changing the color of the changed first initial changed image.

[0213] Specifically, according to the color setting of the target color data (for example, setting the color data in the front position in the target color data as the first color data, and the color data in the back position as the second color data, or directly indicating which color is the first color data and which color is the second color data in the target color data, etc.), it is determined that the first color data is yellow and the second color data is green.

[0214] Step 536: Calculate the first weight, the first color data, and the pixel value of each pixel in the image to be changed b1 according to the first color conversion algorithm to generate a first initial changed image.

[0215] Specifically, by calculating the first weight, the first color data, and the pixel value of each pixel in the image b1 to be changed through the first color conversion algorithm, the color of the image region in the image b1 to be changed that is susceptible to style transfer can be changed.

[0216] Step 538: Calculate the second weight, the second color data, and the pixel value of each pixel in the first initial changed image according to the second color conversion algorithm to generate a target image.

[0217] Specifically, the second weight, the second color data, and the pixel value of each pixel in the first initial changed image are calculated by the second color conversion algorithm, so that the color of the image region that is not easily affected by the style transfer in the image b1 to be changed can be changed.

[0218] In summary, the image processing method provided by the embodiment of the present application receives a change request, wherein the change request carries image identification information and target color data, performs style transfer on the image b1 to be changed corresponding to the image identification information, determines the first weight image corresponding to the image to be changed, and realizes the first weight image obtained by performing style transfer on the image b1 to be changed, and clarifies the image area that is easily affected by style transfer and the image area that is not easily affected by style in the image b1 to be changed. Then, an inverted image is generated based on the pixel value of each pixel in the first weight image, and a second weight image is generated based on the pixel value of each pixel in the inverted image. Finally, the image b1 to be changed is converted into the target image according to the first weight image, the second weight image and the target color data. The color change of the image area that is easily affected by style transfer in the image b1 to be changed is realized by the first weight image, and the color change of the image area that is not easily affected by style transfer in the image to be changed is realized by the second weight image b1, thereby realizing the color change of the entire area in the image b1 to be changed, so that the style change effect of the image b1 to be changed is more in line with expectations.

[0219] Corresponding to the above method embodiment, the present application also provides an image processing device embodiment, Figure 6 FIG. 1 is a schematic diagram showing the structure of an image processing device according to an embodiment of the present application. Figure 6 As shown, the device 600 includes:

[0220] The receiving module 602 is configured to receive a change request, wherein the change request carries image information and target color data;

[0221] The determination module 604 is configured to determine a first weighted image corresponding to the image to be changed by performing style migration on the image to be changed corresponding to the image information, and generate an inverted image according to a pixel value of each pixel in the first weighted image;

[0222] A generating module 606 is configured to generate a second weight image based on the pixel value of each pixel in the inverted image;

[0223] The conversion module 608 is configured to convert the image to be changed into a target image according to the first weight image, the second weight image and the target color data.

[0224] Optionally, the conversion module 608 includes:

[0225] a normalization processing submodule, configured to perform normalization processing on the pixel value of each pixel in the first weight image to obtain a first weight corresponding to each pixel in the first weight image, and to perform normalization processing on the pixel value of each pixel in the second weight image to obtain a second weight corresponding to each pixel in the second weight image;

[0226] The conversion submodule is configured to convert the image to be changed into a target image according to the first weight, the second weight and the target color data.

[0227] Optionally, the conversion submodule is further configured to:

[0228] Determining, in the target color data, first color data for changing the color of the image to be changed, and second color data for changing the color of the changed first initial changed image;

[0229] Calculating the first weight, the first color data, and the pixel value of each pixel in the image to be changed according to a first color conversion algorithm to generate the first initial changed image;

[0230] The second weight, the second color data, and the pixel value of each pixel in the first initial changed image are calculated according to a second color conversion algorithm to generate a target image.

[0231] Optionally, the conversion submodule is further configured to:

[0232] Determining, in the target color data, third color data for changing the color of the image to be changed, and fourth color data for changing the color of the changed second initial changed image;

[0233] Calculating the second weight, the third color data, and the pixel value of each pixel in the image to be changed according to a third color conversion algorithm to generate the second initial changed image;

[0234] The first weight, the fourth color data, and the pixel value of each pixel in the second initial changed image are calculated according to a fourth color conversion algorithm to generate a target image.

[0235] Optionally, the determining module 604 is further configured to:

[0236] Determining an image to be changed according to the image information, and determining a style image corresponding to the image to be changed;

[0237] Combining the style image with the image to be changed to generate an image pair;

[0238] Inputting the image pair into a style transfer model to obtain a composite image output by the style transfer model;

[0239] Performing grayscale processing on the image to be changed to generate a first grayscale image, and performing grayscale processing on the synthesized image to generate a second grayscale image;

[0240] Performing a subtraction operation based on the pixel values ​​of the pixels in the first grayscale image and the pixel values ​​of the pixels in the second grayscale image to obtain a difference grayscale image;

[0241] When the number of the difference grayscale images is one, using the difference grayscale image as a first weight map;

[0242] In the case that there are a plurality of difference grayscale images, an AND operation is performed based on the pixel values ​​of the pixels in each difference grayscale image to obtain the first weight image.

[0243] Optionally, the determining module 604 is further configured to:

[0244] Determining an image to be changed according to the image information, and determining a style image corresponding to the image to be changed;

[0245] When the length and width of the image to be changed are equal, the image to be changed is continuously expanded in four directions to obtain an expanded image to be changed;

[0246] Combining the expanded image to be changed with the style image to generate an expanded image pair;

[0247] Inputting the extended image pair into a style transfer model to obtain an extended synthetic image output by the style transfer model;

[0248] According to the image size of the image to be changed, a clipped image to be changed is obtained from the expanded image to be changed, and the clipped image to be changed is used as the image to be changed, and,

[0249] The extended composite image is cut out according to the image size to obtain a composite image.

[0250] Optionally, the determining module 604 is further configured to:

[0251] The difference grayscale image is blurred to obtain a first blurred grayscale image, and the first blurred grayscale image is used as the difference grayscale image.

[0252] Optionally, the determining module 604 is further configured to:

[0253] Downsampling the difference grayscale image to generate a reduced grayscale image;

[0254] Blurring the reduced grayscale image to obtain a second blurred grayscale image;

[0255] The second blurred grayscale image is upsampled to generate an enlarged grayscale image, and the enlarged grayscale image is used as the difference grayscale image.

[0256] Optionally, the generating module 606 is further configured to:

[0257] Calculate the pixel value of each pixel in the inverted image based on a preset algorithm to obtain a weighted pixel value corresponding to each pixel in the inverted image;

[0258] A second weight image is generated based on the weight pixel values.

[0259] Optionally, the generating module 606 is further configured to:

[0260] In the case that there is a weighted pixel value greater than a preset pixel threshold among the absolute values ​​of the weighted pixel values, the weighted pixel value greater than the preset pixel threshold is updated to the preset pixel threshold.

[0261] In summary, the image processing device provided by the embodiment of the present application receives a change request, wherein the change request carries image information and target color data, performs style transfer on the image to be changed corresponding to the image information, determines the first weight image corresponding to the image to be changed, and realizes the first weight image obtained by performing style transfer on the image to be changed, and clarifies the image area in the image to be changed that is easily affected by style transfer, and the image area that is not easily affected by style. Then, an inverted image is generated based on the pixel value of each pixel in the first weight image, and a second weight image is generated based on the pixel value of each pixel in the inverted image. Finally, the image to be changed is converted into a target image based on the first weight image, the second weight image and the target color data. The color change of the image area in the image to be changed that is easily affected by style transfer is realized by the first weight image, and the color change of the image area in the image to be changed that is not easily affected by style transfer is realized by the second weight image, thereby realizing the color change of the entire area in the image to be changed, so that the style change effect of the image to be changed is more in line with expectations.

[0262] The above is a schematic scheme of an image processing device of this embodiment. It should be noted that the technical scheme of the image processing device and the technical scheme of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical scheme of the image processing device can be referred to the description of the technical scheme of the above-mentioned image processing method.

[0263] In one embodiment of the present application, a computing device is also provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the image processing method when executing the computer instructions.

[0264] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned image processing method.

[0265] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which implement the steps of the image processing method as described above when executed by a processor.

[0266] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned image processing method.

[0267] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0268] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0269] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0270] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0271] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. An image processing method, characterized in that: include: receiving a change request, wherein the change request carries image information and target color data, and the target color data is color data used to change the color of the image; By performing style migration on the image to be changed corresponding to the image information, a first weighted image corresponding to the image to be changed is determined, and an inverted image is generated according to a pixel value of each pixel in the first weighted image; Generate a second weight image based on the pixel value of each pixel in the inverted image; converting the image to be changed into a target image according to the first weight image, the second weight image and the target color data; The determining of the first weight image corresponding to the image to be changed by performing style migration on the image to be changed corresponding to the image information comprises: performing style migration on the image to be changed to obtain a composite image, determining a first grayscale image corresponding to the image to be changed and a second grayscale image corresponding to the composite image, and determining the first weight image based on a difference in pixel values ​​of pixels between the first grayscale image and the second grayscale image; The method of converting the image to be changed into a target image based on the first weight image, the second weight image and the target color data includes: performing operations on pixel values ​​of pixels in the image to be changed, a first weight corresponding to each pixel in the first weight image, a second weight corresponding to each pixel in the second weight image and the target color data to obtain the target image after the style of the image to be changed is changed.

2. The image processing method according to claim 1, characterized in that: The step of calculating the pixel values ​​of the pixels in the image to be changed, the first weight corresponding to each pixel in the first weight image, the second weight corresponding to each pixel in the second weight image, and the target color data to obtain the target image after the style of the image to be changed includes: Normalizing the pixel value of each pixel in the first weight image to obtain a first weight corresponding to each pixel in the first weight image, and normalizing the pixel value of each pixel in the second weight image to obtain a second weight corresponding to each pixel in the second weight image; The image to be changed is converted into a target image according to the first weight, the second weight and the target color data.

3. The image processing method according to claim 2, characterized in that: The converting the image to be changed into a target image according to the first weight, the second weight and the target color data comprises: Determining, in the target color data, first color data for changing the color of the image to be changed, and second color data for changing the color of the changed first initial changed image; Calculating the first weight, the first color data, and the pixel value of each pixel in the image to be changed according to a first color conversion algorithm to generate the first initial changed image; The second weight, the second color data, and the pixel value of each pixel in the first initial changed image are calculated according to a second color conversion algorithm to generate a target image.

4. The image processing method according to claim 2, characterized in that: The converting the image to be changed into a target image according to the first weight, the second weight and the target color data comprises: Determining, in the target color data, third color data for changing the color of the image to be changed, and fourth color data for changing the color of the changed second initial changed image; Calculating the second weight, the third color data, and the pixel value of each pixel in the image to be changed according to a third color conversion algorithm to generate the second initial changed image; The first weight, the fourth color data, and the pixel value of each pixel in the second initial changed image are calculated according to a fourth color conversion algorithm to generate a target image.

5. The image processing method according to claim 1, characterized in that: The step of performing style migration on the image to be changed to obtain a composite image, determining a first grayscale image corresponding to the image to be changed and a second grayscale image corresponding to the composite image, and determining the first weight image based on a difference in pixel values ​​of pixels between the first grayscale image and the second grayscale image, comprises: Determining an image to be changed according to the image information, and determining a style image corresponding to the image to be changed; Combining the style image with the image to be changed to generate an image pair; Inputting the image pair into a style transfer model to obtain a composite image output by the style transfer model; Performing grayscale processing on the image to be changed to generate a first grayscale image, and performing grayscale processing on the synthesized image to generate a second grayscale image; Performing a subtraction operation based on pixel values ​​of pixels in the first grayscale image and pixel values ​​of pixels in the second grayscale image to obtain a difference grayscale image; When the number of the difference grayscale images is one, using the difference grayscale image as a first weight map; In the case that there are a plurality of difference grayscale images, an AND operation is performed based on pixel values ​​of pixels in each difference grayscale image to obtain the first weight image.

6. The image processing method according to claim 5, characterized in that: Before performing grayscale processing on the image to be changed to generate the first grayscale image, the method further includes: Determining an image to be changed according to the image information, and determining a style image corresponding to the image to be changed; When the length and width of the image to be changed are equal, the image to be changed is continuously expanded in four directions to obtain an expanded image to be changed; Combining the expanded image to be changed with the style image to generate an expanded image pair; Inputting the extended image pair into a style transfer model to obtain an extended synthetic image output by the style transfer model; According to the image size of the image to be changed, a clipped image to be changed is captured from the expanded image to be changed, and the clipped image to be changed is used as the image to be changed, and, The extended composite image is cut out according to the image size to obtain a composite image.

7. The image processing method according to claim 5 or 6, characterized in that: After performing a subtraction operation based on the pixel values ​​of the pixels in the first grayscale image and the pixel values ​​of the pixels in the second grayscale image to obtain a difference grayscale image, the method includes: The difference grayscale image is blurred to obtain a first blurred grayscale image, and the first blurred grayscale image is used as the difference grayscale image.

8. The image processing method according to claim 5 or 6, characterized in that: After performing a subtraction operation based on the pixel values ​​of the pixels in the first grayscale image and the pixel values ​​of the pixels in the second grayscale image to obtain a difference grayscale image, the method further includes: Downsampling the difference grayscale image to generate a reduced grayscale image; Blurring the reduced grayscale image to obtain a second blurred grayscale image; The second blurred grayscale image is upsampled to generate an enlarged grayscale image, and the enlarged grayscale image is used as the difference grayscale image.

9. The image processing method according to claim 1, characterized in that: The step of generating a second weight image based on the pixel value of each pixel in the inverted image comprises: Calculate the pixel value of each pixel in the inverted image based on a preset algorithm to obtain a weighted pixel value corresponding to each pixel in the inverted image; A second weight image is generated based on the weight pixel values.

10. The image processing method according to claim 9, characterized in that: Before generating the second weighted image based on the weighted pixel value, the method further includes: In the case that there is a weighted pixel value greater than a preset pixel threshold among the absolute values ​​of the weighted pixel values, the weighted pixel value greater than the preset pixel threshold is updated to the preset pixel threshold.

11. An image processing device, characterized in that: include: A receiving module is configured to receive a change request, wherein the change request carries image information and target color data, and the target color data is color data used to change the color of the image; a determination module configured to determine a first weighted image corresponding to the image to be changed by performing style migration on the image to be changed corresponding to the image information, and generate an inverted image according to a pixel value of each pixel in the first weighted image; A generating module, configured to generate a second weight image based on the pixel value of each pixel in the inverted image; a conversion module, configured to convert the image to be changed into a target image according to the first weight image, the second weight image and the target color data; The determination module is further configured to perform style migration on the image to be changed to obtain a composite image, determine a first grayscale image corresponding to the image to be changed and a second grayscale image corresponding to the composite image, and determine the first weight image based on the difference in pixel values ​​of pixels between the first grayscale image and the second grayscale image; The conversion module is further configured to perform operations on the pixel values ​​of the pixels in the image to be changed, the first weight corresponding to each pixel in the first weight image, the second weight corresponding to each pixel in the second weight image, and the target color data to obtain the target image after the style of the image to be changed is changed.

12. A computing device comprising a memory, a processor, and computer instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer instructions, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 10.

Images