Image processing method, device, electronic device and storage medium

Abstract

The present disclosure relates to an image processing method, device, electronic device and storage medium. The method includes: obtaining a face image, the face image includes a target object's area to be repaired; selecting a repair reference area of ​​the area to be repaired, the repair reference area includes the area to be repaired and an area adjacent to the area to be repaired; updating the pixel value of the area to be repaired to 0; dividing the repair reference area into multiple candidate areas of equal size to the area to be repaired, and screening out a target candidate area from the multiple candidate areas, the pixel average value of the target candidate area, the arrangement position is centered within the pixel average values ​​of each candidate area arranged in a target order, the target order includes a large to small order or a small to large order; using the pixel value of each pixel in the target candidate area to update the pixel value of the pixel at the corresponding position in the area to be repaired, and obtaining a target face image. The present disclosure improves the processing speed of defect removal and repair of face images.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] After performing a shooting operation such as a selfie, a user often needs to perform a beautification process on the face image obtained by the shooting operation to obtain a target image. For example, a blemish removal and beautification process is performed on the blemish part of the face to obtain a beautified target image.

[0003] In the related art, the PatchMatch algorithm can be used to fill the defective area in the face image to achieve the repair and beautification of the defective part. Among them, the PatchMatch algorithm method requires multiple downsampling and upsampling of the face image to obtain multiple reduced images corresponding to the face images and multiple enlarged images corresponding to the face images. Then, the defective area is searched and matched from all non-defective areas in the obtained multiple reduced images and multiple enlarged images, and the defective area is filled with the matched image blocks.

[0004] However, the method of using the PatchMatch algorithm to fill the defective area has a huge amount of calculation, and as the number of defective parts in the face image increases, the processing speed of removing the defect and repairing the face image is slow. Summary of the invention

[0005] The present disclosure provides an image processing method, device and system to at least solve the problem of slow processing speed of defect removal and repair processing of face images in related technologies. The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, the method comprising:

[0007] Acquire a face image, wherein the face image includes a to-be-repaired area of ​​the target object;

[0008] Selecting a repair reference area of ​​the area to be repaired, wherein the repair reference area includes the area to be repaired and an area adjacent to the area to be repaired;

[0009] Update the pixel value of the area to be repaired to 0;

[0010] Divide the repair reference area into a plurality of candidate areas of equal size to the area to be repaired, and screen a target candidate area from the plurality of candidate areas, wherein the pixel average value of the target candidate area is centered within the pixel average values ​​of the candidate areas arranged in a target order, wherein the target order includes an order from large to small or from small to large;

[0011] The pixel values ​​of the pixels in the target candidate area are used to update the pixel values ​​of the pixels in the corresponding positions in the area to be repaired, so as to obtain the target face image.

[0012] In a possible implementation, the screening out a target candidate region from the plurality of candidate regions includes:

[0013] Calculating the average pixel value of each candidate area;

[0014] Arrange the pixel averages according to the target order;

[0015] The candidate area corresponding to the target number of pixel average values ​​located in the middle position among the arranged pixel average values ​​is taken as the target candidate area.

[0016] In a possible implementation, there are multiple target candidate regions, and using the pixel value of each pixel in the target candidate region to update the pixel value of the pixel at the corresponding position in the to-be-repaired region to obtain the target face image includes:

[0017] Obtaining the grayscale value of each pixel in each of the target candidate areas;

[0018] From the grayscale values ​​of pixels at corresponding positions in each of the target candidate regions, select a pixel value corresponding to a maximum grayscale value as a target pixel value at the corresponding position in the region to be repaired;

[0019] The pixel value of each pixel in the to-be-repaired area is updated to the corresponding target pixel value to obtain a target face image.

[0020] In a possible implementation, dividing the repair reference area into a plurality of candidate areas having the same size as the area to be repaired includes:

[0021] A sliding window having the same size as the area to be repaired is used to extract multiple candidate areas from the repair reference area with a set step size.

[0022] In a possible implementation, before dividing the repair reference area into a plurality of candidate areas having the same size as the area to be repaired and selecting a target candidate area from the plurality of candidate areas, the method further includes:

[0023] The pixel value of each pixel in the area where the non-target object is located in the face image is updated to 0.

[0024] In a possible implementation, the pixel average value of the target candidate region is centered in the pixel average values ​​of the candidate regions arranged in the target order, including:

[0025] The pixel average value of the red channel of each pixel in the target candidate area is centered within the pixel average values ​​of the red channel of each candidate area arranged in the target order.

[0026] In a possible implementation, after the pixel values ​​of pixels at corresponding positions in the to-be-repaired area are updated by using the pixel values ​​of each pixel in the target candidate area to obtain the target face image, the method further includes:

[0027] The target face image is fused with the face image to obtain a fused target image.

[0028] In a possible implementation, the selecting a repair reference area of ​​the area to be repaired includes:

[0029] Determine the minimum bounding rectangle of the area to be repaired in the face image;

[0030] Enlarging the minimum circumscribed rectangle by a set multiple to obtain an expanded rectangle;

[0031] The area where the expanded rectangle is located is used as the repair reference area.

[0032] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, the apparatus comprising:

[0033] An acquisition module, used for acquiring a face image, wherein the face image includes a to-be-repaired area of ​​a target object;

[0034] A selection module, used for selecting a repair reference area of ​​the area to be repaired, wherein the repair reference area includes the area to be repaired and an area adjacent to the area to be repaired;

[0035] An updating module, used for updating the pixel value of the to-be-repaired area to 0;

[0036] A screening module, used for dividing the repair reference area into a plurality of candidate areas of equal size to the area to be repaired, and screening a target candidate area from the plurality of candidate areas, wherein the pixel average value of the target candidate area is centered within the pixel average values ​​of the candidate areas arranged in a target order, wherein the target order includes an order from large to small or an order from small to large;

[0037] The updating module is further used to use the pixel value of each pixel in the target candidate area to update the pixel value of the pixel at the corresponding position in the area to be repaired, so as to obtain the target face image.

[0038] In a possible implementation, the screening module is further used to:

[0039] Calculating the average pixel value of each candidate area;

[0040] Arrange the pixel averages according to the target order;

[0041] The candidate area corresponding to the target number of pixel average values ​​located in the middle position among the arranged pixel average values ​​is taken as the target candidate area.

[0042] In a possible implementation manner, the number of the target candidate regions is multiple, and the updating module is further used to:

[0043] Obtaining the grayscale value of each pixel in each of the target candidate areas;

[0044] From the grayscale values ​​of pixels at corresponding positions in each of the target candidate regions, select a pixel value corresponding to a maximum grayscale value as a target pixel value at the corresponding position in the region to be repaired;

[0045] The pixel value of each pixel in the to-be-repaired area is updated to the corresponding target pixel value to obtain a target face image.

[0046] In a possible implementation, the screening module is further configured to use a sliding window having a size equal to that of the region to be repaired to extract a plurality of candidate regions from the repair reference region with a set step size.

[0047] In a possible implementation, the updating module is further configured to update the pixel value of each pixel in the area where the non-target object is located in the facial image to 0.

[0048] In a possible implementation, the pixel average value of the target candidate region is centered in the pixel average values ​​of the candidate regions arranged in the target order, including:

[0049] The pixel average value of the red channel of each pixel in the target candidate area is centered within the pixel average values ​​of the red channel of each candidate area arranged in the target order.

[0050] In a possible implementation, the device further includes:

[0051] The fusion processing module is used to fuse the target face image with the face image to obtain a fused target image.

[0052] In a possible implementation, the selection module is further used to:

[0053] Determine the minimum bounding rectangle of the area to be repaired in the face image;

[0054] Enlarging the minimum circumscribed rectangle by a set multiple to obtain an expanded rectangle;

[0055] The area where the expanded rectangle is located is used as the repair reference area.

[0056] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0057] processor;

[0058] a memory for storing instructions executable by the processor;

[0059] Wherein, the processor is configured to execute the instructions to implement any image processing method described in the first aspect.

[0060] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute any image processing method described in the first aspect.

[0061] According to a fifth aspect of an embodiment of the present disclosure, there is provided a computer program product, comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the image processing method described in any one of the first aspects is implemented.

[0062] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0063] By acquiring a face image of the area to be repaired including the target object, a repair reference area of ​​the area to be repaired is selected, and the repair reference area includes the area to be repaired and the area adjacent to the area to be repaired. After updating the pixel value of the area to be repaired to 0, the repair reference area is divided into multiple candidate areas of equal size to the area to be repaired, and the target candidate area is screened from the multiple candidate areas. The pixel value of each pixel in the target candidate area is used to update the pixel value of the pixel at the corresponding position in the area to be repaired, so as to obtain the target face image. In this technical solution, by updating the pixel value of the area to be repaired to 0 and ensuring that the pixel average value of the selected target candidate area is centered in the pixel average values ​​of each candidate area arranged in a descending order or a descending order, it is ensured that the target candidate area screened for updating the area to be repaired comes from the area around the area to be repaired of the face image. And by directly using the pixel value of each pixel in the screened target candidate area to update the area to be repaired, the effect of repairing the area to be repaired is achieved. Compared with the related art, the processing process of repairing the area to be repaired is simple, the processing rate is improved, and the processing efficiency is improved.

[0064] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0066] Figure 1 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0067] Figure 2 is a schematic diagram showing a repair reference area according to an exemplary embodiment.

[0068] Figure 3 The figure is a flow chart of a method for selecting a target pixel value according to an exemplary embodiment.

[0069] Figure 4 is a flowchart of another image processing method according to an exemplary embodiment.

[0070] Figure 5 The figure is a flowchart of an image processing apparatus according to an exemplary embodiment.

[0071] Figure 6 The invention is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0072] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0073] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0074] Please refer to Figure 1 , which shows a flow chart of an image processing method provided by an embodiment of the present disclosure. The image processing method can be applied to an electronic device. The electronic device can be a terminal. Figure 1 As shown, the method includes:

[0075] In step 101, a face image is acquired, where the face image includes a to-be-repaired area of ​​a target object.

[0076] In the embodiment of the present disclosure, if a user wants to perform beauty processing on the area to be repaired that includes a target object in the facial image taken by the user, the user can perform a setting operation on the terminal to trigger the beauty processing. The terminal receives the setting operation and obtains the facial image in response to the setting operation. The target object included in the area to be repaired can be an object such as skin, accessories, or facial features. The area to be repaired that includes the target object can be called a defect area of ​​the target object in the facial image.

[0077] In step 102, a repair reference region of the area to be repaired is selected, where the repair reference region includes the area to be repaired and an area adjacent to the area to be repaired.

[0078] In the embodiment of the present disclosure, the area adjacent to the area to be repaired may include an area adjacent to at least part of the edge area of ​​the area to be repaired. For example, the area adjacent to the area to be repaired is an area adjacent to all edge areas of the area to be repaired, that is, the area adjacent to the area to be repaired is an area surrounding and adjacent to the area to be repaired.

[0079] Optionally, the process of selecting a repair reference area for the area to be repaired may include the following steps 1021 to 1023 .

[0080] In step 1021, the minimum bounding rectangle of the area to be repaired in the face image is determined.

[0081] Optionally, the terminal can determine the minimum bounding rectangle of the area to be repaired by calling the minAreaRect function in the opencv software library and passing the pixels of the area to be repaired in the face image to the minAreaRect function. The minimum bounding rectangle is identified by the pixel coordinates of the center point of the rectangle, the width of the rectangle, the height of the rectangle, and the rotation angle of the rectangle. For example, the terminal can obtain the pixel coordinates of the center point of the minimum bounding rectangle, the width of the minimum bounding rectangle, the height of the minimum bounding rectangle, and the rotation angle of the minimum bounding rectangle by inputting the pixel coordinates of all pixels in the area to be repaired, or the pixel coordinates of the pixels on the outer contour of the area to be repaired to the minAreaRect function.

[0082] In step 1022, the minimum bounding rectangle is enlarged by a set multiple to obtain an expanded rectangle.

[0083] Optionally, the terminal may use any pixel in the minimum bounding rectangle as a fixed point, magnify the minimum bounding rectangle by a set multiple, and obtain an expanded rectangle. For example, the terminal may use the center point of the minimum bounding rectangle as a fixed point, magnify the minimum bounding rectangle by a set multiple, and obtain an expanded rectangle. That is, the terminal may use the center point of the minimum bounding rectangle as a fixed point, magnify the minimum bounding rectangle by a set multiple compared to the minimum bounding rectangle, and generate an expanded rectangle.

[0084] In the expanded rectangle obtained in this way, other areas except the area to be repaired surround the area to be repaired. It is convenient to subsequently select pixels for updating the pixels in the area to be repaired from the area surrounding the area to be repaired in the expanded rectangle, and compared with selecting pixels for updating the pixels in the area to be repaired from the adjacent area of ​​a single edge area of ​​the area to be repaired, the comprehensiveness of candidate pixels for updating the pixels in the area to be repaired is improved.

[0085] In the disclosed embodiment, the value of the setting multiple can be set by the user or pre-stored by the terminal. The value range of the setting multiple can be determined according to the size of the minimum bounding rectangle and the face image. For example, the value range of the setting multiple w is [u1, u2]. u1 is 1, and u2 is the divisor of the size of the face image and the size of the minimum bounding rectangle. For example, u is 3, 4, etc.

[0086] Based on this, Figure 2 As shown, when the extended rectangle is a rectangle that takes the center point of the minimum circumscribed rectangle as a fixed point and enlarges the minimum circumscribed rectangle by a set multiple, the coordinates (x2, y2) of the vertex located at the upper left corner of the extended rectangle satisfy:

[0087] x2 = max(0,x1-expand_wd);

[0088] y2=max(0,y1-expand_ht);

[0089] Where x1 is the first coordinate value of the vertex located at the upper left corner in the minimum bounding rectangle of the area to be repaired. y1 is the second coordinate value of the vertex located at the upper left corner in the minimum bounding rectangle. expand_wd is the distance from the first side of the minimum bounding rectangle of the area to be repaired to the corresponding side of the expanded rectangle. expand_ht is the distance from the second side of the minimum bounding rectangle of the area to be repaired to the corresponding side of the expanded rectangle, and the first side and the second side are two adjacent sides of the minimum bounding rectangle.

[0090] The length w2 and width h2 of the expanded rectangle satisfy:

[0091] w2=min(w-1,x1+w1+expand_wd)-x2;

[0092] h2=min(h-1,y1+h1+expand_ht)-y2;

[0093] Wherein, w is the length of the face image, h is the width of the face image, w1 is the length of the minimum bounding rectangle of the area to be repaired, and h1 is the width of the minimum bounding rectangle of the area to be repaired.

[0094] In step 1023, the area where the expanded rectangle is located is used as a repair reference area.

[0095] In the disclosed embodiment, by determining the minimum enclosing rectangle of the area to be repaired in the face image, and using the area where the expanded rectangle is located after the minimum enclosing rectangle is enlarged by a set multiple as the repair reference area, rapid extraction of the area adjacent to the area to be repaired is achieved, thereby improving the speed of selecting the repair reference area.

[0096] It should be noted that the embodiments of the present disclosure may also select the repair reference area of ​​the area to be repaired by other means. For example, the terminal may obtain the coordinates of each pixel on the outer contour of the reference area to be repaired. According to the relative position of each pixel and the center point of the reference area to be repaired, the target coordinate value in the coordinates of each pixel is expanded by a target multiple to obtain new coordinates corresponding to each pixel. The area with the outer contour of the position of the new coordinates corresponding to each pixel is the repair reference area. Of course, the terminal may also select the repair reference area of ​​the area to be repaired by other means, and the embodiments of the present disclosure are not limited to this.

[0097] In step 103, the pixel value of the area to be repaired is updated to 0.

[0098] In the disclosed embodiment, the terminal updates the pixel value of each pixel in the area to be repaired to 0, so as to avoid selecting pixels in the area to be repaired when subsequently selecting pixels from the repair reference area to update the area to be repaired, thereby ensuring the effectiveness of the selected pixels.

[0099] In step 104, the repair reference area is divided into a plurality of candidate areas of equal size to the area to be repaired, and a target candidate area is obtained by screening from the plurality of candidate areas.

[0100] The pixel average of the target candidate area is centered in the pixel averages of each candidate area arranged in the target order, and the target order includes a large to small order or a small to large order. In the disclosed embodiment, the terminal can calculate the pixel average of each candidate area. Arrange the pixel averages in the target order. The candidate area corresponding to the pixel average with the centered arrangement position in the arranged pixel averages is used as the target candidate area. Optionally, the terminal can record the target candidate area by recording the pixel points of each pixel on the outer contour of the target candidate area. Alternatively, if the area to be repaired is an irregular shape, the terminal can also divide the repair reference area into multiple candidate areas of the same size as the minimum circumscribed rectangle of the area to be repaired. The terminal can record the target candidate area by recording the coordinates, length and width of the upper left corner vertex of the target candidate area.

[0101] Wherein, the number of target candidate regions may be one or more. When the number of target candidate regions is one, the pixel average value of the target candidate region is located in the middle position within the pixel average values ​​of the candidate regions arranged in the target order. When the number of target candidate regions is multiple, the pixel average values ​​of the multiple target candidate regions may be arranged in connected middle positions within the pixel average values ​​of the candidate regions arranged in the target order, or the pixel average values ​​of the multiple target candidate regions may be arranged in non-adjacent center positions within the pixel average values ​​of the candidate regions arranged in the target order.

[0102] Optionally, the process of the terminal selecting the target candidate area from the multiple candidate areas may include: calculating the pixel average of each candidate area. Arranging the pixel averages in a target order. Among the arranged pixel averages, the candidate area corresponding to the target number of pixel averages located in the middle position is used as the target candidate area. The value of the target number can be at least one.

[0103] For example, the terminal can calculate the pixel average of each candidate area, and store the calculated pixel average of each candidate area in the target ordered list in the target order. Get the length N of the target ordered list, and take the candidate areas corresponding to the pixel averages of sequence numbers N / 2-1, N / 2, and N / 2+1 in the target list as the target candidate areas. Where N is the number of divided candidate areas.

[0104] In the implementation of the present disclosure, the pixel average of the target candidate area, which is arranged in the middle of the pixel averages of each candidate area arranged in the target order, may include: the pixel average of the red channel of each pixel in the target candidate area, which is arranged in the middle of the pixel average of the red channel of each candidate area arranged in the target order. Accordingly, the process of the terminal screening the target candidate area from multiple candidate areas may include: calculating the pixel average of the red channel of each pixel in each candidate area. Arranging the pixel averages in the target order. Among the arranged pixel averages, the candidate area corresponding to the target number of pixel averages located in the middle position is used as the target candidate area.

[0105] In the disclosed embodiment, the candidate area corresponding to the target number of pixel averages located in the middle position among the arranged pixel averages is used as the target candidate area. Since the average of multiple values ​​can represent the central tendency of the multiple values, the selected target number of pixel averages located in the middle position can represent that the colors reflected by each pixel in the corresponding candidate area are more concentrated on the color of the target object. Therefore, the selected target candidate area is more likely to not include the part of the target object that needs to be repaired, that is, it does not include the defective part of the target object. The effectiveness of selecting the target candidate area is guaranteed, and the efficiency of selecting the target candidate area is improved.

[0106] Furthermore, when the target object is skin, the pixel value of the red channel of the pixel can better represent the color of the skin than the pixel values ​​of other color channels. Therefore, the selected target candidate area is the candidate area corresponding to the average value of the pixels in the center. The average value of the pixels in the center can more effectively characterize that the colors reflected by each pixel in the corresponding candidate area are more concentrated on the color of normal skin. Therefore, the selected target candidate area can more likely not include the part of the skin that needs to be repaired, that is, does not include the defective part of the skin. The effectiveness of selecting the target candidate area is guaranteed, and the efficiency of selecting the target candidate area is improved.

[0107] In the disclosed embodiment, there may be multiple implementations of the terminal dividing the repair reference area into multiple candidate areas of equal size to the area to be repaired. Two optional implementations are described below as examples.

[0108] In a first optional implementation, the terminal may divide the repair reference area into multiple candidate areas of equal size to the area to be repaired. The terminal may directly divide the repair reference area into multiple candidate areas of equal size to the area to be repaired according to the size of the area to be repaired.

[0109] In a second optional implementation, the process in which the terminal divides the repair reference area into multiple candidate areas of the same size as the area to be repaired may include: using a sliding window of the same size as the area to be repaired to intercept multiple candidate areas from the repair reference area with a set step size. In the embodiment of the present disclosure, the sliding window can slide along two directions perpendicular to each other from the repair reference area with a set step size to intercept multiple candidate areas. The set step sizes step for sliding in different directions may be equal, or the set step sizes step for sliding in different directions may be different. The set step size may be a value set by the user through the terminal, or the set step size may be a fixed value stored in the terminal.

[0110] For example, assume that the step size step_x set in the horizontal direction is expand_wd / 2, and the step size step_y set in the vertical direction is expand_ht / 2. The terminal can create a sliding window equal to the size of the area to be repaired, and start from the upper left corner of the repair reference area, and slide horizontally with a step size of expand_wd / 2 to intercept multiple candidate areas. When one side of the sliding window overlaps with the right side of the repair reference area, that is, after the sliding window slides to the right side of the repair reference area, it can start from the upper left corner of the repair reference area and slide vertically with a step size of expand_ht / 2, and then slide horizontally with a step size of expand_wd / 2 to intercept multiple candidate areas again. Repeat the process of sliding from the starting point of the last horizontal sliding in the repair reference area, sliding vertically with a step size of expand_ht / 2, and then sliding horizontally with a step size of expand_wd / 2 to intercept multiple candidate areas again, until the sliding window slides to the lower right corner of the repair reference area, and finally intercepts multiple candidate areas.

[0111] In the disclosed embodiment, a method of intercepting a candidate area from the repair reference area using a sliding window of the same size as the area to be repaired utilizes the idea of ​​sliding window convolution, and each two adjacent intercepted candidate areas may have an overlapping portion, so that the number of intercepted candidate areas is large, and the number of obtained candidate areas is large compared to the first candidate area division method. And because there is less change between each two adjacent intercepted candidate areas, the division granularity of the candidate area is finer, which is conducive to selecting more suitable target candidate areas and improving the selection efficiency of effective target candidate areas.

[0112] Step 105: Use the pixel value of each pixel in the target candidate area to update the pixel value of the pixel at the corresponding position in the area to be repaired, so as to obtain the target face image.

[0113] Optionally, when the number of target candidate regions is one, the terminal may directly use the pixel value of each pixel in the target candidate region to update the pixel value of the pixel at the corresponding position in the region to be repaired, to obtain the target face image. The corresponding position means that the position of the pixel in the target candidate region is the same as the position of the pixel in the region to be repaired.

[0114] When there are multiple target candidate areas, the terminal can select a target pixel value of the target pixel from multiple pixels at positions corresponding to any pixel in the multiple target candidate areas according to set rules for any pixel in the area to be repaired, and update the pixel value of any pixel in the area to be repaired to the target pixel value to obtain the target face image.

[0115] In one example, there are multiple target candidate areas. The setting rule may be to select the average pixel value from multiple pixels as the target pixel value. The terminal uses the pixel value of each pixel in the target candidate area to update the pixel value of the pixel at the corresponding position in the area to be repaired, and the process of obtaining the target face image may include: calculating the pixel average value of the pixels at the corresponding position in each target candidate area as the target pixel value of the corresponding position in the area to be repaired. The pixel value of each pixel in the area to be repaired is updated to the corresponding target pixel value to obtain the target face image.

[0116] In another example, the number of target candidate regions is multiple, and the setting rule may be to select a pixel value corresponding to a maximum grayscale value among the multiple pixels as the target pixel value.

[0117] like Figure 3 As shown, the terminal uses the pixel value of each pixel in the target candidate area to update the pixel value of the pixel at the corresponding position in the area to be repaired, and the process of obtaining the target face image may include:

[0118] Step 301: Obtain the grayscale value of each pixel in each target candidate area.

[0119] Optionally, the terminal may capture each target candidate region from the face image to obtain a candidate image corresponding to each captured target candidate region, convert each candidate image into a grayscale image, and read the grayscale value of each pixel in each target candidate region.

[0120] Step 302: From the grayscale values ​​of pixels at corresponding positions in each target candidate region, select the pixel value corresponding to the maximum grayscale value as the target pixel value at the corresponding position in the region to be repaired.

[0121] In the disclosed embodiment, after obtaining the grayscale value of each pixel in each target candidate area, the terminal can compare the grayscale values ​​of each pixel at the same position in each target candidate area, select the maximum grayscale value as the target grayscale value, and use the pixel value corresponding to the target grayscale value as the target pixel value corresponding to the pixel at the position corresponding to the pixel in the area to be repaired.

[0122] Step 303: Update the pixel value of each pixel in the area to be repaired to the corresponding target pixel value to obtain the target face image.

[0123] In the disclosed embodiment, for a pixel at any position in the area to be repaired, the pixel value of the pixel is updated to the target pixel value corresponding to the pixel, so as to fill the position corresponding to the pixel in the area to be repaired with the target pixel value corresponding to the pixel. After the pixel values ​​of all pixels at all positions in the area to be repaired are updated to the corresponding target pixel values, the target face image is obtained.

[0124] In this way, since the grayscale value can reflect the brightness of the image, and the defective part of the target object is usually darker than the normal part, selecting the pixel value corresponding to the maximum grayscale value as the target pixel value for updating the area to be repaired can further avoid selecting pixels of the defective part of the target object, ensuring the validity of the selected target pixel value while retaining the target object's own texture.

[0125] In summary, the image processing method provided by the embodiment of the present disclosure obtains a face image including the area to be repaired of the target object, selects a repair reference area of ​​the area to be repaired, and the repair reference area includes the area to be repaired and the area adjacent to the area to be repaired. After the pixel value of the area to be repaired is updated to 0, the repair reference area is divided into a plurality of candidate areas of equal size to the area to be repaired, and the target candidate area is screened from the plurality of candidate areas. The pixel value of each pixel in the target candidate area is used to update the pixel value of the pixel at the corresponding position in the area to be repaired, so as to obtain the target face image. In this technical solution, by updating the pixel value of the area to be repaired to 0 and ensuring that the pixel average value of the selected target candidate area is centered in the pixel average values ​​of each candidate area arranged in a descending order or a descending order, it is ensured that the target candidate area screened for updating the area to be repaired comes from the area around the area to be repaired of the face image. And by directly using the pixel value of each pixel in the screened target candidate area to update the area to be repaired, the effect of repairing the area to be repaired is achieved. Compared with related technologies, the process of repairing the area to be repaired is simple, which improves the processing rate and thus improves the processing efficiency.

[0126] Please refer to Figure 4, which shows a flow chart of another image processing method provided by an embodiment of the present disclosure. The image processing method can be applied to an electronic device. The electronic device can be a terminal. Figure 4 As shown, the method includes:

[0127] In step 401, a face image is acquired, where the face image includes a to-be-repaired area of ​​a target object.

[0128] The explanation and implementation of this step can refer to the explanation and implementation of the aforementioned step 101, and will not be elaborated in detail in the embodiment of the present disclosure.

[0129] In step 402, a repair reference region of the area to be repaired is selected, where the repair reference region includes the area to be repaired and an area adjacent to the area to be repaired.

[0130] The explanation and implementation of this step can refer to the explanation and implementation of the aforementioned step 102, and will not be elaborated in detail in the embodiment of the present disclosure.

[0131] In step 403, the pixel value of the area to be repaired is updated to 0.

[0132] The explanation and implementation of this step can refer to the explanation and implementation of the aforementioned step 103, and will not be elaborated in detail in the embodiment of the present disclosure.

[0133] In step 404, the pixel value of each pixel in the area where the non-target object is located in the face image is updated to 0.

[0134] In the disclosed embodiment, the terminal may identify the area where the non-target object is located in the face image, and update the pixel value of each pixel in the area where the non-target object is located to 0.

[0135] Optionally, the terminal may input the face image into the target object detection model to obtain a target object mask image. The target object mask image identifies a target area where the target object is located and other areas except the target area. According to the target object mask image, the pixel value of each pixel in the target area where the non-target object is located in the face image is updated to 0.

[0136] For example, when the target object is the skin of a human face, the target object detection model may be a skin color detection model. The terminal inputs the face image into the skin color detection model to obtain a skin mask image (also called skin color mask). The pixel value of each pixel in the non-skin area of ​​the skin mask image is 0, and the pixel value of each pixel in the skin area is 1. The terminal may determine an area in the face image that overlaps with an area in the skin mask image where the pixel value is 1, and update the pixel value of each pixel in the area to 0.

[0137] In the disclosed embodiment, the pixels in the area where the target object is located are used to more effectively update and repair the pixels in the area to be repaired, thereby achieving the repair of the area to be repaired. Therefore, by updating the pixel values ​​of each pixel in the area where the non-target object is located in the face image to 0, the pixels in the face image that are not suitable for updating the area to be repaired can be effectively eliminated, and further ensure that when the pixels for updating the area to be repaired are selected from the repair reference area in the subsequent process, the pixels in the area where the non-target object is located are avoided, thereby avoiding updating the pixel values ​​of the pixels in the area where the non-target object is located to the area to be repaired, thereby improving the repair effect of the target object in the face image.

[0138] In step 405, the repair reference area is divided into a plurality of candidate areas of equal size to the area to be repaired, and a target candidate area is obtained by screening from the plurality of candidate areas. The pixel average value of the target candidate area is centered in the pixel average values ​​of the candidate areas arranged in a target order, and the target order includes an order from large to small or from small to large.

[0139] The explanation and implementation of this step can refer to the explanation and implementation of the aforementioned step 104, and will not be elaborated in detail in the embodiment of the present disclosure.

[0140] In step 406, the pixel values ​​of the pixels in the target candidate area are used to update the pixel values ​​of the pixels at the corresponding positions in the area to be repaired, so as to obtain the target face image.

[0141] The explanation and implementation of this step can refer to the explanation and implementation of the aforementioned step 105, and will not be elaborated in detail in the embodiment of the present disclosure.

[0142] In step 407, the target face image is fused with the face image to obtain a fused target image.

[0143] Optionally, the terminal may use a fusion algorithm to fuse the target face image with the face image to obtain a fused target image. The fusion algorithm may include a multi-segment fusion algorithm or a Boisson fusion algorithm. In this way, by fusing the target face image with the face image, the transition effect between the boundary between the area to be repaired and the non-repaired area other than the area to be repaired in the target face image is eliminated, so that the boundary between the area to be repaired and the non-repaired area is smoother, thereby making the final target image more harmonious and improving the restoration effect of the face image.

[0144] In summary, the image processing method provided by the embodiment of the present disclosure obtains a face image including the area to be repaired of the target object, selects a repair reference area of ​​the area to be repaired, and the repair reference area includes the area to be repaired and the area adjacent to the area to be repaired. After the pixel value of the area to be repaired is updated to 0, the repair reference area is divided into a plurality of candidate areas of equal size to the area to be repaired, and the target candidate area is screened from the plurality of candidate areas. The pixel value of each pixel in the target candidate area is used to update the pixel value of the pixel at the corresponding position in the area to be repaired, so as to obtain the target face image. In this technical solution, by updating the pixel value of the area to be repaired to 0 and ensuring that the pixel average value of the selected target candidate area is centered in the pixel average values ​​of each candidate area arranged in a descending order or a descending order, it is ensured that the target candidate area screened for updating the area to be repaired comes from the area around the area to be repaired of the face image. And by directly using the pixel value of each pixel in the screened target candidate area to update the area to be repaired, the effect of repairing the area to be repaired is achieved. Compared with related technologies, the process of repairing the area to be repaired is simple, which improves the processing rate and thus improves the processing efficiency.

[0145] Please refer to Figure 5 , which shows a block diagram of an image processing device provided by an embodiment of the present disclosure. Figure 5 As shown, the device includes: an acquisition module 501, a selection module 502, an update module 503 and a screening module 504.

[0146] An acquisition module 501 is used to acquire a face image, where the face image includes a region to be repaired of a target object;

[0147] A selection module 502 is used to select a repair reference area of ​​the area to be repaired, where the repair reference area includes the area to be repaired and an area adjacent to the area to be repaired;

[0148] An updating module 503 is used to update the pixel value of the area to be repaired to 0;

[0149] A screening module 504 is used to divide the repair reference area into multiple candidate areas of the same size as the area to be repaired, and screen the multiple candidate areas to obtain a target candidate area, wherein the pixel average value of the target candidate area is centered within the pixel average values ​​of the candidate areas arranged in a target order, and the target order includes an order from large to small or from small to large;

[0150] The updating module 503 is further used to use the pixel value of each pixel in the target candidate area to update the pixel value of the pixel at the corresponding position in the area to be repaired, so as to obtain the target face image.

[0151] In a possible implementation, the screening module 504 is further configured to:

[0152] Calculate the average pixel value of each candidate area;

[0153] Arrange the average values ​​of each pixel in target order;

[0154] The candidate area corresponding to the target number of pixel average values ​​located in the middle position among the arranged pixel average values ​​is taken as the target candidate area.

[0155] In a possible implementation, there are multiple target candidate regions, and the updating module 503 is further configured to:

[0156] Obtain the grayscale value of each pixel in each target candidate area;

[0157] From the grayscale values ​​of pixels at corresponding positions in each target candidate area, select the pixel value corresponding to the maximum grayscale value as the target pixel value at the corresponding position in the area to be repaired;

[0158] The pixel values ​​of each pixel in the area to be repaired are updated to the corresponding target pixel values ​​to obtain the target face image.

[0159] In a possible implementation, the screening module 504 is further configured to use a sliding window having a size equal to that of the region to be repaired to extract a plurality of candidate regions from the repair reference region with a set step size.

[0160] In a possible implementation, the updating module 503 is further configured to update the pixel value of each pixel in the area where the non-target object is located in the face image to 0.

[0161] In a possible implementation, the pixel average value of the target candidate region is centered in the pixel average values ​​of the candidate regions arranged in the target order, including:

[0162] The pixel average value of the red channel of each pixel in the target candidate region is centered within the pixel average values ​​of the red channel of each candidate region arranged in the target order.

[0163] In a possible implementation, the device further includes:

[0164] The fusion processing module is used to fuse the target face image with the face image to obtain the fused target image.

[0165] In a possible implementation, the selection module 502 is further configured to:

[0166] Determine the minimum bounding rectangle of the area to be repaired in the face image;

[0167] Enlarge the minimum bounding rectangle by a set multiple to obtain an expanded rectangle;

[0168] The area where the expanded rectangle is located is used as the repair reference area.

[0169] In summary, the image processing device provided by the embodiment of the present disclosure obtains a face image including the area to be repaired of the target object, selects a repair reference area of ​​the area to be repaired, and the repair reference area includes the area to be repaired and the area adjacent to the area to be repaired. After the pixel value of the area to be repaired is updated to 0, the repair reference area is divided into a plurality of candidate areas of equal size to the area to be repaired, and the target candidate area is screened from the plurality of candidate areas. The pixel value of each pixel in the target candidate area is used to update the pixel value of the pixel at the corresponding position in the area to be repaired, so as to obtain the target face image. In this technical solution, by updating the pixel value of the area to be repaired to 0 and ensuring that the pixel average value of the selected target candidate area is centered in the pixel average values ​​of each candidate area arranged in a descending order or a descending order, it is ensured that the target candidate area screened for updating the area to be repaired comes from the area around the area to be repaired of the face image. And by directly using the pixel value of each pixel in the screened target candidate area to update the area to be repaired, the effect of repairing the area to be repaired is achieved. Compared with related technologies, the process of repairing the area to be repaired is simple, which improves the processing rate and thus improves the processing efficiency.

[0170] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0171] Figure 6 600 is a block diagram of an electronic device according to an exemplary embodiment. The electronic device may be a terminal. The electronic device 600 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The electronic device 600 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0172] Typically, the electronic device 600 includes a processor 601 and a memory 602 .

[0173] The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 601 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0174] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 602 is used to store at least one instruction, which is used to be executed by the processor 601 to implement the information display method provided in the method embodiment of the present application.

[0175] In some embodiments, the electronic device 600 may further optionally include: a peripheral device interface 603 and at least one peripheral device. The processor 601, the memory 602 and the peripheral device interface 603 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 603 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio circuit 607, a positioning assembly 608 and a power supply 609.

[0176] The peripheral device interface 603 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 601 and the memory 602. In some embodiments, the processor 601, the memory 602, and the peripheral device interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 601, the memory 602, and the peripheral device interface 603 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0177] The radio frequency circuit 604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 604 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 604 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 604 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 604 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0178] The display screen 605 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen 605 is a touch display screen, the display screen 605 also has the ability to collect touch signals on the surface or above the surface of the display screen 605. The touch signal can be input to the processor 601 as a control signal for processing. At this time, the display screen 605 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 605 can be one, and the front panel of the electronic device 600 is set; in other embodiments, the display screen 605 can be at least two, which are respectively set on different surfaces of the electronic device 600 or are folded; in some other embodiments, the display screen 605 can be a flexible display screen, which is set on the curved surface or folded surface of the electronic device 600. Even, the display screen 605 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 605 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0179] The camera assembly 606 is used to capture images or videos. Optionally, the camera assembly 606 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 606 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0180] The audio circuit 607 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 601 for processing, or input them into the radio frequency circuit 604 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the electronic device 600. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 601 or the radio frequency circuit 604 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 607 may also include a headphone jack.

[0181] Positioning component 608 is used to locate the current geographic location of electronic device 600 to implement navigation or LBS (Location Based Service). Positioning component 608 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system or the European Union's Galileo system.

[0182] The power supply 609 is used to power various components in the electronic device 600. The power supply 609 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 609 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0183] In some embodiments, the electronic device 600 further includes one or more sensors 610 , including but not limited to: an acceleration sensor 611 , a gyroscope sensor 612 , a pressure sensor 613 , a fingerprint sensor 614 , an optical sensor 615 , and a proximity sensor 616 .

[0184] The acceleration sensor 611 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the electronic device 600. For example, the acceleration sensor 611 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 601 can control the display screen 605 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 611. The acceleration sensor 611 can also be used for collecting game or user motion data.

[0185] The gyro sensor 612 can detect the body direction and rotation angle of the electronic device 600, and the gyro sensor 612 can cooperate with the acceleration sensor 611 to collect the user's 3D actions on the electronic device 600. The processor 601 can implement the following functions based on the data collected by the gyro sensor 612: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0186] The pressure sensor 613 can be set on the side frame of the electronic device 600 and / or the lower layer of the display screen 605. When the pressure sensor 613 is set on the side frame of the electronic device 600, it can detect the user's holding signal of the electronic device 600, and the processor 601 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 613. When the pressure sensor 613 is set on the lower layer of the display screen 605, the processor 601 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 605. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0187] The fingerprint sensor 614 is used to collect the user's fingerprint, and the processor 601 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 614, or the fingerprint sensor 614 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 601 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 614 can be set on the front, back, or side of the electronic device 600. When a physical button or a manufacturer logo is set on the electronic device 600, the fingerprint sensor 614 can be integrated with the physical button or the manufacturer logo.

[0188] The optical sensor 615 is used to collect the ambient light intensity. In one embodiment, the processor 601 can control the display brightness of the display screen 605 according to the ambient light intensity collected by the optical sensor 615. Specifically, when the ambient light intensity is high, the display brightness of the display screen 605 is increased; when the ambient light intensity is low, the display brightness of the display screen 605 is reduced. In another embodiment, the processor 601 can also dynamically adjust the shooting parameters of the camera assembly 606 according to the ambient light intensity collected by the optical sensor 615.

[0189] The proximity sensor 616, also called a distance sensor, is usually disposed on the front panel of the electronic device 600. The proximity sensor 616 is used to collect the distance between the user and the front of the electronic device 600. In one embodiment, when the proximity sensor 616 detects that the distance between the user and the front of the electronic device 600 is gradually decreasing, the processor 601 controls the display screen 605 to switch from the screen-on state to the screen-off state; when the proximity sensor 616 detects that the distance between the user and the front of the electronic device 600 is gradually increasing, the processor 601 controls the display screen 605 to switch from the screen-off state to the screen-on state.

[0190] Those skilled in the art will understand that Figure 6 The structure shown in the figure does not constitute a limitation on the electronic device 600, and may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0191] In an exemplary embodiment, a non-computer readable storage medium is also provided, and when the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the image processing methods provided by the above-mentioned various method embodiments. For example, the computer readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0192] In an exemplary embodiment, a computer program product is also provided, including a computer program. When the computer program is executed by a processor, the image processing method provided by each of the above method embodiments can be executed.

[0193] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0194] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a face image, wherein the face image includes a to-be-repaired area of ​​the target object; Selecting a repair reference area of ​​the area to be repaired, wherein the repair reference area includes the area to be repaired and an area adjacent to the area to be repaired; Update the pixel value of the area to be repaired to 0; Divide the repair reference area into a plurality of candidate areas of equal size to the area to be repaired, and screen a target candidate area from the plurality of candidate areas, wherein the pixel average value of the target candidate area is centered within the pixel average values ​​of the candidate areas arranged in a target order, wherein the target order includes an order from large to small or from small to large; The pixel values ​​of the pixels in the target candidate area are used to update the pixel values ​​of the pixels in the corresponding positions in the area to be repaired, so as to obtain the target face image; The pixel average value of the target candidate area is centered in the pixel average values ​​of the candidate areas arranged in the target order, including: The pixel average value of the red channel of each pixel in the target candidate area is centered within the pixel average values ​​of the red channel of each candidate area arranged in the target order.

2. The method according to claim 1, characterized in that The step of screening a target candidate region from the plurality of candidate regions comprises: Calculating the average pixel value of each candidate area; Arrange the pixel averages according to the target order; The candidate area corresponding to the target number of pixel average values ​​located in the middle position among the arranged pixel average values ​​is taken as the target candidate area.

3. The method according to claim 1 or 2, characterized in that: The number of the target candidate regions is multiple, and the pixel value of each pixel in the target candidate region is used to update the pixel value of the pixel at the corresponding position in the to-be-repaired region to obtain the target face image, including: Obtaining the grayscale value of each pixel in each of the target candidate areas; From the grayscale values ​​of pixels at corresponding positions in each of the target candidate regions, select a pixel value corresponding to a maximum grayscale value as a target pixel value at the corresponding position in the region to be repaired; The pixel value of each pixel in the to-be-repaired area is updated to the corresponding target pixel value to obtain a target face image.

4. The method according to claim 1, characterized in that: The step of dividing the repair reference area into a plurality of candidate areas having the same size as the area to be repaired comprises: A sliding window having the same size as the area to be repaired is used to extract multiple candidate areas from the repair reference area with a set step size.

5. The method according to claim 1, characterized in that Before dividing the repair reference area into a plurality of candidate areas of equal size to the area to be repaired, and selecting a target candidate area from the plurality of candidate areas, the method further includes: The pixel value of each pixel in the area where the non-target object is located in the face image is updated to 0.

6. The method according to claim 1, characterized in that After the pixel values ​​of the pixels in the target candidate area are updated by using the pixel values ​​of the pixels in the corresponding positions in the area to be repaired to obtain the target face image, the method further includes: The target face image is fused with the face image to obtain a fused target image.

7. The method according to claim 1, characterized in that The selecting a repair reference area of ​​the area to be repaired comprises: Determine the minimum bounding rectangle of the area to be repaired in the face image; Enlarging the minimum circumscribed rectangle by a set multiple to obtain an expanded rectangle; The area where the expanded rectangle is located is used as the repair reference area.

8. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a face image, wherein the face image includes a to-be-repaired area of ​​a target object; A selection module, used for selecting a repair reference area of ​​the area to be repaired, wherein the repair reference area includes the area to be repaired and an area adjacent to the area to be repaired; An updating module, used for updating the pixel value of the to-be-repaired area to 0; A screening module, used for dividing the repair reference area into a plurality of candidate areas of equal size to the area to be repaired, and screening a target candidate area from the plurality of candidate areas, wherein the pixel average value of the target candidate area is centered within the pixel average values ​​of the candidate areas arranged in a target order, wherein the target order includes an order from large to small or an order from small to large; The updating module is further used to update the pixel values ​​of the pixels at the corresponding positions in the to-be-repaired area by using the pixel values ​​of the pixels in the target candidate area to obtain the target face image; The pixel average value of the target candidate area is centered in the pixel average values ​​of the candidate areas arranged in the target order, including: The pixel average value of the red channel of each pixel in the target candidate area is centered within the pixel average values ​​of the red channel of each candidate area arranged in the target order.

9. The device according to claim 8, characterized in that The screening module is further used for: Calculating the average pixel value of each candidate area; Arrange the pixel averages according to the target order; The candidate area corresponding to the target number of pixel average values ​​located in the middle position among the arranged pixel average values ​​is taken as the target candidate area.

10. The device according to claim 8 or 9, characterized in that The number of the target candidate regions is multiple, and the updating module is further used for: Obtaining the grayscale value of each pixel in each of the target candidate areas; From the grayscale values ​​of pixels at corresponding positions in each of the target candidate regions, select a pixel value corresponding to a maximum grayscale value as a target pixel value at the corresponding position in the region to be repaired; The pixel value of each pixel in the to-be-repaired area is updated to the corresponding target pixel value to obtain a target face image.

11. The device according to claim 8, characterized in that The screening module is further configured to use a sliding window having a size equal to that of the region to be repaired to extract a plurality of candidate regions from the repair reference region with a set step size.

12. The device according to claim 8, characterized in that The updating module is further used to update the pixel value of each pixel in the area where the non-target object is located in the face image to 0.

13. The device according to claim 8, characterized in that The device also includes: The fusion processing module is used to fuse the target face image with the face image to obtain a fused target image.

14. The device according to claim 8, characterized in that The selection module is further used for: Determine the minimum bounding rectangle of the area to be repaired in the face image; Enlarging the minimum circumscribed rectangle by a set multiple to obtain an expanded rectangle; The area where the expanded rectangle is located is used as the repair reference area.

15. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image processing method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image processing method as claimed in any one of claims 1 to 7.

17. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

Images