Image processing method and device, electronic equipment and computer readable storage medium

Abstract

The application provides an image processing method and device, electronic equipment and computer readable storage medium. The electronic equipment determines the arrangement of multiple portraits in a group image to be processed. If the arrangement of multiple portraits in the group image is neat, the correction strength of a correction model is reduced, and the multiple portraits are corrected according to the adjusted correction model. The correction model is used to adjust the projection mode of the image to reduce the perspective distortion of the image. The neat arrangement indicates that the distance between the arrangement position of the multiple portraits and the reference line is within a preset range. The above method can reduce the perspective distortion of the image to be processed, and can avoid the phenomenon that the portrait arrangement queue is obviously curved due to the excessive correction strength. By reducing the correction strength, the problem of image imbalance caused by the small size of the edge face and the magnification of the center face in the corrected image can be avoided, and the display effect of the image is improved.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to an image processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Wide-angle lenses offer a superior field of view, capturing memorable moments of more people and landscapes, and are widely used for large-scale group portraits, expansive natural scenery, and the photography of large buildings. In mobile devices, using wide-angle cameras for group photos has become a recent trend. However, the perspective projection method of wide-angle cameras can cause perspective distortion, resulting in distorted faces in the image.

[0003] In traditional methods, an algorithmic model can be used to convert the perspective projection in an image into a spherical projection to obtain a corrected image, thereby reducing facial distortion caused by perspective distortion.

[0004] However, straight lines are prone to appear curved in the corrected images, especially in group photos. The formation of the corrected group image will show obvious curvature, resulting in a decrease in image realism. Summary of the Invention

[0005] This application provides an image processing method, apparatus, electronic device, and computer-readable storage medium.

[0006] An image processing method, comprising:

[0007] Determine the arrangement of multiple portraits in the group photos to be processed;

[0008] If the arrangement of multiple portraits in a group photo is neat, the correction intensity of the correction model is reduced, and the multiple portraits are merged and corrected according to the adjusted correction model. The correction model is used to adjust the projection method of the image to reduce the perspective distortion of the image. The neat arrangement indicates that the distance between the arrangement position of the multiple portraits and the reference line is within a preset range.

[0009] In one embodiment, determining the arrangement of multiple portraits in the group image to be processed includes:

[0010] Multiple portrait regions detected from group photos are merged to obtain the merged processing area of ​​the group photos;

[0011] The arrangement of multiple portraits is determined based on the position of the merged processing area in the group image.

[0012] In one embodiment, determining the arrangement of multiple portraits based on the position of the merged processing region in the group image includes:

[0013] Extract the contour curve of the merged processing region;

[0014] If the contour curve is located within a preset area of ​​the image to be processed, the arrangement is determined to be neat; the preset area is located between two horizontal reference lines.

[0015] In one embodiment, after extracting the contour curve of the merged processing region, the method further includes:

[0016] If the contour curve has a portion that extends beyond the preset area, then the target face recognition box is determined in each face recognition box corresponding to the merged processing area; the distance between the bottom border of the target face recognition box and the bottom edge of the group image is minimized.

[0017] The area between the horizontal line containing the lower border and the lower edge of the contour curve within the merged processing area is defined as the target area.

[0018] Divide the target area into multiple sub-regions along the vertical direction;

[0019] The arrangement of multiple portraits in the group image is determined based on the area of ​​each sub-region.

[0020] In one embodiment, determining the arrangement of multiple portraits in a group image based on the area of ​​each sub-region includes:

[0021] Calculate the area ratio between any two sub-regions;

[0022] If all area ratios are within the preset threshold range, then the arrangement is determined to be neat.

[0023] If any area ratio exceeds the preset threshold range, the arrangement is determined to be disordered.

[0024] In one embodiment, before determining the arrangement of multiple portraits in the group image to be processed, the method further includes:

[0025] Perform face recognition on the image to be processed to obtain the face recognition bounding box corresponding to the image to be processed;

[0026] Count the number of face recognition bounding boxes corresponding to the image to be processed, and obtain the face recognition bounding box with the largest area in each face recognition bounding box;

[0027] If the number of face recognition bounding boxes is greater than a preset threshold, and the largest face recognition bounding box is smaller than a preset area threshold, then the image to be processed is determined to be a group photo image to be processed.

[0028] In one embodiment, after counting the number of face recognition boxes corresponding to the image to be processed and obtaining the face recognition box with the largest area among all face recognition boxes, the method further includes:

[0029] If the number of face recognition boxes is less than or equal to a preset number threshold, or the face recognition box with the largest area is greater than or equal to a preset area threshold, then a correction model is used to correct the portraits in the image to be processed.

[0030] In one embodiment, the above-mentioned correction model corrects the portraits in the image to be processed, including:

[0031] The system identifies low-confidence and high-confidence targets in the image to be processed. Low-confidence targets are human face regions that do not have corresponding face recognition bounding boxes, while high-confidence targets are human face regions that have corresponding face recognition bounding boxes.

[0032] The facial regions of high-confidence targets whose facial deformation exceeds the preset range are identified as high-confidence facial regions to be corrected, and a correction model is used to correct these high-confidence facial regions.

[0033] In one embodiment, after identifying low-confidence targets and high-confidence targets in the image to be processed, the method further includes:

[0034] Calculate the aspect ratio of low-confidence targets;

[0035] If the aspect ratio is greater than a preset aspect ratio threshold, determine whether the face region of the low-confidence target is adjacent to the high-confidence face region to be corrected.

[0036] If so, the grayscale value of the low-confidence target in the image to be processed is determined as the grayscale value of the low-confidence target in the corrected image.

[0037] In one embodiment, determining whether the face region of a low-confidence target is adjacent to a high-confidence face region to be corrected includes:

[0038] According to the preset expansion ratio, the facial recognition bounding box of the low-confidence target is expanded outward to obtain the target expansion bounding box;

[0039] Determine whether the target bounding box intersects with the high-confidence face region to be corrected;

[0040] If so, determine that the face region of the low-confidence target is adjacent to the high-confidence face region to be corrected.

[0041] An image processing apparatus, comprising:

[0042] The determination module is used to determine the arrangement of multiple portraits in the group image to be processed;

[0043] The correction module is used to reduce the correction intensity of the correction model when multiple portraits in a group photo are arranged in a neat manner, and to merge and correct multiple portraits according to the adjusted correction model. The correction model is used to adjust the projection method of the image to reduce the perspective distortion of the image. The neat arrangement means that the distance between the arrangement position of multiple portraits and the reference line is within a preset range.

[0044] An electronic device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the image processing method described above.

[0045] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the image processing method described above.

[0046] The aforementioned image processing method, apparatus, electronic device, and computer-readable storage medium involve the electronic device determining the arrangement of multiple portraits in a group image to be processed. If the arrangement of the multiple portraits in the group image is orderly, the correction intensity of the correction model is reduced, and the multiple portraits are merged and corrected according to the adjusted correction model. The correction model is used to adjust the projection method of the image to reduce perspective distortion. Orderly arrangement indicates that the distance between the arrangement position of the multiple portraits and the reference line is within a preset range. Because the electronic device determines the arrangement of multiple portraits in the group image, it can reduce the correction intensity of the correction model and merge the multiple portraits in the image to be processed when the portraits are orderly arranged. This reduces perspective distortion and avoids obvious bending of the portrait arrangement caused by excessive correction intensity. By reducing the correction intensity, it avoids image imbalance caused by excessively small edge face size and enlarged center face size in the corrected image, further improving the image display effect. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is an application environment diagram of an image processing method in one embodiment;

[0049] Figure 2 This is a flowchart of an image processing method in one embodiment;

[0050] Figure 3 This is a flowchart of an image processing method in one embodiment;

[0051] Figure 4 This is a schematic diagram of an image processing method in one embodiment;

[0052] Figure 5 This is a flowchart of an image processing method in one embodiment;

[0053] Figure 6 This is a schematic diagram of an image processing method in one embodiment;

[0054] Figure 7 This is a flowchart of an image processing method in one embodiment;

[0055] Figure 8 This is a flowchart of an image processing method in one embodiment;

[0056] Figure 9 This is a flowchart of an image processing method in one embodiment;

[0057] Figure 10 This is a flowchart of an image processing method in one embodiment;

[0058] Figure 11 This is a structural block diagram of an image processing device in one embodiment;

[0059] Figure 12 This is a structural block diagram of an image processing device in one embodiment;

[0060] Figure 13 This is a structural block diagram of an image processing device in one embodiment;

[0061] Figure 14 This is a structural block diagram of an image processing device in one embodiment;

[0062] Figure 15 This is a structural block diagram of an image processing device in one embodiment;

[0063] Figure 16 This is a structural block diagram of an electronic device in one embodiment. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] The image processing method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, the electronic device 100 can process the image 200 to be processed and correct perspective distortion in the image. The electronic device 100 may be, but is not limited to, a personal computer, a mobile terminal, a personal digital assistant, a wearable electronic device, etc.

[0066] Figure 2 This is a flowchart of an image processing method in one embodiment. The image processing method in this embodiment is designed to run on... Figure 1 Let's take electronic devices as an example for description. Figure 2 As shown, the image processing methods include:

[0067] S101. Determine the arrangement of multiple portraits in the group photo image to be processed.

[0068] The group photos to be processed can be taken with a wide-angle lens or a regular lens; there is no limitation on this. Wide-angle lenses offer a superior field of view and are widely used for capturing large groups of people. However, perspective projection during the imaging process causes perspective distortion, which is more pronounced with wide-angle lenses. For group photos containing multiple people, such as those taken in large group portraits, the electronic device can merge and correct the images. To avoid the distortion of straight lines after correction, the electronic device can first determine the arrangement of the people in the group photo and then apply appropriate strategies to correct the image based on the arrangement.

[0069] The aforementioned multiple portraits can be all portraits in the group photo image to be processed, multiple portraits with a large portrait area in the group photo image, or multiple portraits located in the middle of the group photo image; there is no limitation here.

[0070] The aforementioned arrangement can include whether the portraits in the group photos are arranged neatly or not, and can also include the degree of neatness of the portrait arrangement; in addition, the aforementioned arrangement can also include at least one area in the image to be processed where the portraits are arranged neatly, which is not limited here.

[0071] Specifically, when determining the arrangement of multiple portraits in a group image, the electronic device can determine whether the portraits are arranged neatly based on the position of the faces of each portrait in the group image, such as whether the face areas in the image to be processed are on the same straight line; it can also determine whether the portraits are arranged neatly based on the arrangement of the bottom edges of each portrait area in the group image, such as whether the feet of the portraits are on the same straight line; the method of determining the above arrangement is not limited here.

[0072] S102. If the arrangement of multiple portraits in the group photo is neat, reduce the correction intensity of the correction model and merge the multiple portraits according to the adjusted correction model; wherein, the correction model is used to adjust the projection method of the image to reduce the perspective distortion of the image; neat arrangement means that the distance between the arrangement position of multiple portraits and the reference line is within a preset range.

[0073] The aforementioned correction model can be used to adjust the projection method of an image to reduce perspective distortion. This model can be used to convert the perspective projection of an image into a spherical projection; alternatively, it can be used to convert the perspective projection of an image into a normal cylindrical projection (Mercator projection). The type of correction model is not limited here. Different correction intensities of the aforementioned correction models result in different correction effects; the greater the correction intensity, the smaller the perspective distortion of the corrected image. Taking the correction model for converting the perspective projection of the image to a spherical projection as an example, a greater correction intensity results in a corrected image that more closely approximates the ideal spherical projection.

[0074] The term "neat arrangement" indicates that the distance between the positions of multiple portraits in a group photo and a reference line is within a preset range. The reference line can be a horizontal line in the group photo used to measure the arrangement of the portraits; the group photo may include one or more reference lines. The position of the portraits can be the head, feet, or other reference positions within the portrait, and is not limited here. When the distance between the positions of each portrait in the group photo and the reference line is within the preset range, the multiple portraits can be considered to be neatly arranged. The smaller the distribution range of the aforementioned distance values, the higher the degree of neatness of the portrait arrangement; the larger the distribution range of the aforementioned distance values, the lower the degree of neatness of the portrait arrangement.

[0075] If the electronic device determines that the people in the image to be processed are arranged neatly, it can assume that the straight lines in the image are relatively prominent. After correction by the correction model, the user will easily perceive the curvature of the straight lines in the line of people. In this case, the electronic device can reduce the correction strength of the correction model and perform merging correction on the people in the image to be processed according to the adjusted correction model. When the electronic device performs merging correction on multiple people in a group photo, it can use the correction model to complete the correction on the area formed by merging multiple people in the group photo, thus obtaining the corrected group photo image.

[0076] When reducing the correction intensity, the electronic device can replace the parameters corresponding to the recommended correction intensity of the correction model with a set of preset parameters, where the correction intensity corresponding to the preset parameters is lower than the recommended correction intensity. Alternatively, the electronic device can determine the adjusted correction intensity based on the arrangement of people in the image to be processed, and then further determine the parameters of the correction model based on the adjusted correction intensity. For example, the electronic device can determine the descent gradient of the correction intensity based on the neatness of the people's arrangement.

[0077] The correction strength of the above correction model can be determined by the model parameters. The correction strength can be reduced by adjusting the values ​​of multiple model parameters, or by adjusting the value of a single model parameter; no limitation is made here. The values ​​of the above model parameters can be positively or negatively correlated with the correction strength.

[0078] When the electronic device determines that the arrangement of people in a group photo is not neat, the straight line features in the group photo are not obvious. After correction by the correction model, the user will not easily perceive the phenomenon that the straight line of the group of people has become curved. Therefore, the electronic device can set the correction intensity of the correction model to a preset recommended value and use the correction model to merge and correct multiple people in the group photo.

[0079] In the aforementioned image processing method, the electronic device determines the arrangement of multiple portraits in the group image to be processed. If the arrangement of multiple portraits in the group image is neat, the correction intensity of the correction model is reduced, and the multiple portraits are merged and corrected according to the adjusted correction model. The correction model is used to adjust the projection method of the image to reduce perspective distortion. Neat arrangement indicates that the distance between the arrangement position of the multiple portraits and the reference line is within a preset range. Because the electronic device determines the arrangement of multiple portraits in the group image, it can reduce the correction intensity of the correction model and merge the multiple portraits in the image to be processed when the portraits are neatly arranged. This reduces perspective distortion and avoids obvious bending of the portrait arrangement caused by excessive correction intensity. By reducing the correction intensity, it avoids image imbalance caused by excessively small edge faces and enlarged center faces in the corrected image, further improving the image display effect.

[0080] Figure 3 This is a flowchart illustrating an image processing method in one embodiment. This embodiment relates to an implementation method for an electronic device to determine the arrangement of multiple human images. Based on the above embodiment, as follows... Figure 3 As shown, S101 above includes:

[0081] S201. Merge multiple portrait regions detected from the group photo image to obtain the merged processing area of ​​the group photo image.

[0082] The aforementioned portrait area can be a rectangular area containing the portrait, or an irregular area bounded by the outline of the portrait; no limitation is made here.

[0083] The electronic device can perform facial recognition on the aforementioned group images to obtain the facial regions corresponding to each person in the group images. Specifically, the electronic device can use a neural network model to perform facial recognition on the group images. Based on obtaining the individual facial regions, the electronic device can merge the individual facial regions in the group images to obtain a merged processing area for merging processing of the individual facial regions. The facial regions in the merged processing area can have overlapping areas or can be independent facial regions; this is not limited here.

[0084] S202. Determine the arrangement of multiple portraits based on the position of the merged processing area in the group image.

[0085] Based on the obtained merged processing area, the electronic device can further determine the arrangement of multiple portraits in the group image according to the position of the merged processing area in the group image. The electronic device can determine whether the portraits in the group image are neatly arranged based on whether the merged processing area is within a preset area in the group image. The electronic device can determine whether each pixel in the merged processing area is located within the preset area, and can also determine whether the merged processing area is within the preset area by observing its outline.

[0086] Optionally, the electronic device can extract the contour curve of the merged processing area; if the contour curve is located in a preset area of ​​the image to be processed, the arrangement is determined to be neat; wherein, the preset area is located between two horizontal reference lines. The distance between the two horizontal reference lines is less than the width of the group image. The distance between the horizontal reference lines can be a preset fixed value, or it can be adjusted according to the shooting scene, which is not limited here. The position of the two horizontal reference lines can be a preset position, or it can be adjusted according to the recognition result of the group image. For example, the electronic device can set the bottom edge of the sky or buildings in the group image as the horizontal reference line.

[0087] When the contour curve of the merged processing area in a group photo image lies within the preset area, the electronic device can consider the portraits in the group photo image to be neatly arranged. When the contour curve of the merged processing area in a group photo image extends beyond the preset area, the electronic device can consider the portraits in the group photo image to be unevenly arranged, and can further judge the merged processing area to determine whether the multiple portraits are neatly arranged. Figure 4 Taking the group photo image shown as an example, the group photo image contains multiple portrait areas, which can be displayed as follows: Figure 4The rectangular frame in the image shows the preset area of ​​the group image, which is determined by two horizontal reference lines. If the outline curve of the merged processing area formed by the above multiple portrait areas is located within the preset area, the electronic device can consider the multiple portraits to be arranged neatly.

[0088] The image processing method described above involves merging multiple portrait regions in a group photo image using an electronic device. This allows for the rapid determination of the portrait arrangement based on the position of the merged processing region within the group photo image, thereby improving image processing efficiency.

[0089] Figure 5 This is a flowchart illustrating an image processing method in one embodiment. This embodiment relates to another way for an electronic device to determine the arrangement of multiple portraits, based on the above embodiment, such as... Figure 5 As shown, after the electronic device extracts the contour curve of the merged processing area, it also includes:

[0090] S301. If the contour curve has a portion of the curve that exceeds the preset area, then the target face recognition box is determined in each face recognition box corresponding to the merged processing area; the distance between the lower border of the target face recognition box and the bottom edge of the group image is minimized.

[0091] If the contour curve contains a portion that extends beyond the preset area, the electronic device can further assess the merged processing area to determine whether the multiple portraits in the group image are neatly arranged. The electronic device can acquire the face recognition results of the group image and determine the corresponding face recognition bounding boxes for each portrait in the merged processing area. Specifically, the electronic device can perform face recognition on the group image after portrait recognition; alternatively, it can acquire the face recognition bounding boxes before performing distortion correction processing on the group image. The specific stage for acquiring these face recognition bounding boxes is not limited here.

[0092] Electronic devices can calculate the distance between the bottom border of each face recognition box and the bottom edge of the image, and determine the face recognition box with the smallest distance as the target face recognition box.

[0093] S302. In the merged processing area, the area between the horizontal line where the lower border is located and the lower edge of the contour curve is determined as the target area.

[0094] Furthermore, the electronic device can determine the horizontal line containing the lower border of the target face recognition bounding box, as well as the lower edge of the merging processing area, and then define the merging processing area between the horizontal line and the lower edge as the target area. Based on the distribution of this target area, the electronic device can determine whether the faces in a group photo are neatly arranged.

[0095] S303. Divide the target area into multiple sub-regions according to the vertical direction.

[0096] Once the target area is determined, the electronic device can divide it into multiple sub-areas vertically. The electronic device can divide the target area into sub-areas according to a preset horizontal width, or according to a preset width ratio; there is no limitation on this. That is to say, the widths of the multiple sub-areas can be the same or different.

[0097] Electronic devices can divide a target area into multiple sub-regions. The number of sub-regions can be 3, 5, or other numbers, which is not limited here.

[0098] S304. Determine the arrangement of multiple portraits in the group image based on the area of ​​each sub-region.

[0099] After determining each sub-region, the electronic device can calculate the area of ​​each sub-region and then determine the arrangement of the people in the group image based on the area of ​​each sub-region. Specifically, the electronic device can determine whether the people are arranged neatly based on the area ratio of each sub-region, or based on the area difference between each sub-region; alternatively, the electronic device can also calculate the difference between the area ratios of each sub-region and determine whether the people are arranged neatly based on the difference in these ratios. The method for determining the arrangement is not limited here.

[0100] Optionally, when the electronic device divides the target area into multiple sub-regions according to a preset width, it can calculate the area ratio between any two sub-regions. If all area ratios are within a preset threshold range, the arrangement is determined to be neat; if any area ratio exceeds the preset threshold range, the arrangement is determined to be disordered. The smaller the area difference between the sub-regions, the more neatly the people in the group image are considered to be arranged. The preset threshold range can be 0.8 to 1.2, 0.75 to 1.25, or other ranges, which are not limited here.

[0101] by Figure 6 Taking the group photo image as an example, the target area is shown as the shaded area in the figure. The electronic device can divide the target area into three regions: left, middle, and right. Then, it calculates the area ratio between any two sub-regions, including the first ratio between the left and middle regions, the second ratio between the left and right regions, and the third ratio between the middle and right regions. If all three ratios are within the preset threshold range [0.8, 1.2], then it is determined that the multiple portraits in the group photo image are arranged neatly.

[0102] The above image processing method involves the electronic device determining the target area and dividing it into multiple sub-regions. The arrangement of these sub-regions is then determined based on their areas. This allows the electronic device to more accurately obtain the arrangement of the portraits even when the merged processing area is not entirely within the preset area. Consequently, it can more accurately correct distortion in group photos and improve the image display effect.

[0103] Figure 7 This is a flowchart illustrating an image processing method in one embodiment. This embodiment relates to a method by which an electronic device determines whether an image to be processed is a group photo image. Based on the above embodiment, such as... Figure 7 As shown, before S101 above, it also includes:

[0104] S401. Perform face recognition on the image to be processed to obtain the face recognition bounding box corresponding to the image to be processed.

[0105] After acquiring the image to be processed, the electronic device can perform face recognition on the image to obtain the corresponding face recognition bounding box. The image to be processed can correspond to one face recognition bounding box or multiple face recognition bounding boxes; this is not limited here. The electronic device can input the image to be processed into a face recognition model to obtain an image with marked face recognition bounding boxes output by the model.

[0106] S402. Count the number of face recognition boxes corresponding to the image to be processed, and obtain the face recognition box with the largest area in each face recognition box.

[0107] The electronic device can count the number of face recognition boxes corresponding to the image to be processed, calculate the size of each face recognition box, and determine the face recognition box with the largest area in the image to be processed.

[0108] S403. If the number of face recognition boxes is greater than a preset number threshold, and the face recognition box with the largest area is smaller than a preset area threshold, then the image to be processed is determined to be a group photo image to be processed.

[0109] The electronic device can store preset quantity thresholds and preset area thresholds. Based on the above steps, the electronic device can compare the number of face recognition boxes with the preset quantity thresholds and compare the maximum area value of the face recognition boxes with the preset area thresholds, and determine whether the image to be processed is a group photo based on the comparison results.

[0110] If the number of face recognition bounding boxes exceeds a preset threshold, the electronic device can assume that the image to be processed contains a large number of people, possibly indicating a group photo. If the largest face recognition bounding box is smaller than a preset area threshold, the electronic device can assume that the image is not focused on a single person, and the multiple people in the image are not part of the background. Therefore, if the number of face recognition bounding boxes exceeds the preset threshold, and the largest face recognition bounding box is smaller than the preset area threshold, the electronic device can assume that the image to be processed is a group photo.

[0111] If the number of face recognition boxes is less than or equal to a preset number threshold, or the face recognition box with the largest area is greater than or equal to a preset area threshold, the electronic device may consider the image to be processed not to be a group image and not to be subject to the image processing method described in the above embodiments for distortion correction. The electronic device may use a preset correction model to correct the images of people in the image to be processed individually.

[0112] The above image processing method allows the electronic device to determine whether the image to be processed is a group photo image by using the number of faces in the image and the largest face recognition box. This allows the device to determine a more suitable image processing method for the image to be processed, resulting in a better image display effect.

[0113] Figure 8 This is a flowchart illustrating an image processing method in one embodiment. This embodiment relates to a method by which an electronic device corrects individual portraits in an image to be processed. Based on the above embodiment, as... Figure 8 As shown, the above correction process includes:

[0114] S501. Identify low-confidence targets and high-confidence targets in the image to be processed; wherein, low-confidence targets are portrait regions that do not have corresponding face recognition boxes; and high-confidence targets are portrait regions that have corresponding face recognition boxes.

[0115] The electronic device can simultaneously perform face recognition and portrait recognition on the image to be processed, obtaining the face recognition bounding box and portrait region corresponding to the image. Furthermore, the electronic device can match the portrait region with the face recognition bounding box based on the position of each bounding box. If a portrait region has a corresponding face recognition bounding box, the electronic device can identify that portrait region as a high-confidence target; if a portrait region does not have a corresponding face recognition bounding box, the electronic device can identify that portrait region as a low-confidence target.

[0116] S502. The face regions of high-confidence targets whose face deformation exceeds the preset range are identified as high-confidence face regions to be corrected, and a correction model is used to correct the high-confidence face regions to be corrected.

[0117] For high-confidence targets, electronic devices can determine whether the facial region of the high-confidence target is deformed using a preset algorithm model. The electronic device can obtain the facial deformation parameters of the high-confidence target and then compare these parameters with a preset parameter range to determine whether the deformation parameters of the facial region exceed the preset range. Alternatively, the electronic device can input the facial region of the high-confidence target into the preset algorithm model, and the algorithm model can output the result of whether the facial deformation exceeds the preset range.

[0118] If the facial deformation of a high-confidence target exceeds a preset range, the electronic device can determine that its facial region is a high-confidence facial region to be corrected, and can use a correction model to correct the high-confidence facial region to be corrected. The correction strength of the above correction model can be a preset recommended correction strength, or it can be a correction strength determined by the electronic device based on the facial deformation, which is not limited here. If the facial deformation of a high-confidence target does not exceed the preset range, the electronic device can consider that no correction is needed for that facial region.

[0119] For low-confidence targets, since no face recognition bounding box corresponding to the portrait region is identified, the electronic device may not perform correction on the low-confidence target. When the electronic device corrects the face region of a high-confidence target in the image to be processed, it may stretch the low-confidence target, causing more severe distortion, or treat the low-confidence target as background. Therefore, the electronic device can further analyze the low-confidence target to determine whether its grayscale value needs to be protected to avoid distortion caused by other target correction processes.

[0120] The above image processing method allows electronic devices to identify high-confidence targets and low-confidence targets in the image to be processed, thereby enabling them to apply appropriate image processing methods to each portrait area. This can correct facial distortion of high-confidence targets and prevent low-confidence targets from being treated as background, resulting in a more realistic image display.

[0121] Figure 9 This is a flowchart illustrating an image processing method in one embodiment. This embodiment relates to a processing method for low-confidence targets by an electronic device. Based on the above embodiment, as follows... Figure 9 As shown, the above method also includes:

[0122] S601. Calculate the aspect ratio of the low-confidence target.

[0123] The aforementioned low-confidence target can be a rectangular human image area. When processing a low-confidence target, the electronic device can first calculate the aspect ratio of the low-confidence target and determine whether the grayscale value of the low-confidence target needs to be protected based on the aspect ratio.

[0124] S602. When the aspect ratio is greater than the preset aspect ratio threshold, determine whether the face region of the low-confidence target is adjacent to the high-confidence face region to be corrected.

[0125] Based on the above steps, the electronic device can compare the aspect ratio of the low-confidence target with a preset aspect ratio threshold. If the aspect ratio is less than or equal to the preset aspect ratio threshold, the electronic device can consider that the aspect ratio of the low-confidence target does not match that of a normal human image, and there is no need to protect the grayscale value of the low-confidence target.

[0126] If the aspect ratio is greater than a preset aspect ratio threshold, the electronic device can further determine whether there is a high-confidence face region to be corrected near the low-confidence target. If the face region of the low-confidence target is adjacent to the high-confidence face region to be corrected, the correction process for the high-confidence face region may cause the low-confidence target to be stretched and distorted; if the face region of the low-confidence target is not adjacent to the high-confidence face region to be corrected, the electronic device can assume that the correction of the high-confidence face region will not affect the grayscale value of the low-confidence target.

[0127] The electronic device can calculate the distance between the bounding box of the low-confidence target and the high-confidence face region to be corrected, and then determine whether the face region of the low-confidence target is adjacent to the high-confidence face region to be corrected based on the distance value; or, the electronic device can also calculate the correlation coefficient between the gray value of the low-confidence target and the gray value of the high-confidence face region to be corrected, and determine whether the low-confidence target intersects with the high-confidence face region to be corrected, without limitation.

[0128] Optionally, the electronic device can expand the face recognition bounding box of the low-confidence target outward according to a preset expansion ratio to obtain a target expansion bounding box; then, it determines whether the target expansion bounding box intersects with the high-confidence face region to be corrected; if so, it determines that the face region of the low-confidence target is adjacent to the high-confidence face region to be corrected. For example, if the resolution of the image to be processed is 640x480, the preset expansion ratio can be one-sixth, and the electronic device can expand the face recognition bounding box of the low-confidence target outward by one-sixth, and then determine whether the determined target expansion bounding box contains pixels of the high-confidence face region to be corrected.

[0129] S603. If so, the gray value of the low-confidence target in the image to be processed is determined as the gray value of the low-confidence target in the corrected image.

[0130] If the face region of a low-confidence target is adjacent to a high-confidence face region to be corrected, the electronic device can save the gray value of the low-confidence target in the image to be processed, and then determine it as the gray value of the low-confidence target in the corrected image, so that the gray value of the low-confidence target remains unchanged before and after image processing.

[0131] The above image processing method protects the grayscale value of low-confidence targets, keeping the grayscale value of the area unchanged before and after image processing. This avoids treating low-confidence targets as background during image processing and prevents face stretching of low-confidence targets, resulting in better image display.

[0132] In one embodiment, based on the above embodiments, an image processing method is provided, such as... Figure 10 As shown, it includes:

[0133] S701. Perform face recognition on the image to be processed to obtain the face recognition bounding box corresponding to the image to be processed.

[0134] S702. Count the number of face recognition boxes corresponding to the image to be processed, and obtain the face recognition box with the largest area in each face recognition box. If the number of face recognition boxes is greater than a preset number threshold, and the face recognition box with the largest area is less than a preset area threshold, then determine that the image to be processed is a group photo image to be processed, and proceed to step S703; if the number of face recognition boxes is less than or equal to the preset number threshold, or the face recognition box with the largest area is greater than or equal to the preset area threshold, then proceed to step S711.

[0135] S703: Merge multiple portrait regions detected from the group photo image to obtain the merged processing area of ​​the group photo image.

[0136] S704. Extract the contour curve of the merged processing area; if the contour curve has a part of the curve that exceeds the preset area, then execute S705; if the contour curve is located in the preset area of ​​the image to be processed, then execute S709.

[0137] S705. Determine the target face recognition frame in each face recognition frame corresponding to the merged processing area.

[0138] S706. In the merged processing area, the area between the horizontal line where the lower border is located and the lower edge of the contour curve is determined as the target area.

[0139] S707. Divide the target area into multiple sub-regions according to the vertical direction.

[0140] S708. Calculate the area ratio between any two sub-regions respectively; if all area ratios are within the preset threshold range, then execute S709; if any area ratio exceeds the preset threshold range, then execute S710.

[0141] S709. Reduce the correction intensity of the correction model and merge and correct multiple portraits according to the adjusted correction model.

[0142] S710. Set the correction intensity of the correction model to the preset recommended value, and use the correction model to merge and correct multiple portraits in the group photo.

[0143] S711. Identify low-confidence targets and high-confidence targets in the image to be processed; execute S712 for high-confidence targets; execute S713 for low-confidence targets.

[0144] S712. The face regions of high-confidence targets whose face deformation exceeds the preset range are identified as high-confidence face regions to be corrected, and a correction model is used to correct the high-confidence face regions to be corrected.

[0145] S713. Calculate the aspect ratio of low-confidence targets.

[0146] S714. If the aspect ratio is greater than the preset aspect ratio threshold, determine whether the face region of the low confidence target is adjacent to the high confidence face region to be corrected; if so, execute S715.

[0147] S715. The grayscale value of the low-confidence target in the image to be processed is determined as the grayscale value of the low-confidence target in the corrected image.

[0148] The image processing method described above is similar in principle and technical effect to the methods described in the above embodiments, and will not be repeated here.

[0149] It should be understood that, although Figure 2-10 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2-10 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0150] Figure 11This is a structural block diagram of an image processing apparatus according to one embodiment. Figure 11 As shown, the above-mentioned device includes:

[0151] The determination module 10 is used to determine the arrangement of multiple portraits in the group image to be processed;

[0152] The correction module 20 is used to reduce the correction intensity of the correction model when the arrangement of multiple portraits in a group photo is neat, and to merge and correct the multiple portraits according to the adjusted correction model; wherein, the correction model is used to adjust the projection method of the image to reduce the perspective distortion of the image; neat arrangement means that the distance between the arrangement position of the multiple portraits and the reference line is within a preset range.

[0153] In one embodiment, based on the above embodiments, such as Figure 12 As shown, the determining module 10 includes:

[0154] The merging unit 101 is used to merge multiple portrait regions detected from the group photo image to obtain the merged processing area of ​​the group photo image.

[0155] The first determining unit 102 is used to determine the arrangement of multiple portraits based on the position of the merged processing area in the group image.

[0156] In one embodiment, based on the above embodiment, the first determining unit 102 is specifically used to: extract the contour curve of the merged processing area; if the contour curve is located in a preset area of ​​the image to be processed, then determine that the arrangement is neat; the preset area is located between two horizontal reference lines.

[0157] In one embodiment, based on the above embodiments, such as Figure 13 As shown, the aforementioned determining module 10 further includes a second determining unit 103, used for: determining a target face recognition box in each face recognition box corresponding to the merged processing area when there is a part of the contour curve that exceeds the preset area; minimizing the distance between the lower border of the target face recognition box and the bottom edge of the group image; determining the area between the horizontal line where the lower border is located and the lower edge of the contour curve in the merged processing area as the target area; dividing the target area into multiple sub-regions according to the vertical direction; and determining the arrangement of multiple portraits in the group image based on the area of ​​each sub-region.

[0158] In one embodiment, based on the above embodiment, the second determining unit 103 is specifically used to: calculate the area ratio between any two sub-regions respectively; if each area ratio is within a preset threshold range, the arrangement is determined to be neat; if any area ratio exceeds the preset threshold range, the arrangement is determined to be disordered.

[0159] In one embodiment, based on the above embodiments, such as Figure 14 As shown, the above-mentioned device also includes a recognition module 30, used for: performing face recognition on the image to be processed to obtain the face recognition box corresponding to the image to be processed; counting the number of face recognition boxes corresponding to the image to be processed, and obtaining the face recognition box with the largest area in each face recognition box; if the number of face recognition boxes is greater than a preset number threshold, and the face recognition box with the largest area is less than a preset area threshold, then the image to be processed is determined to be a group photo image to be processed.

[0160] In one embodiment, based on the above embodiments, such as Figure 15 As shown, the above-mentioned device also includes a separate correction module 40, which is used to: correct the portraits in the image to be processed separately using a correction model when the number of face recognition boxes is less than or equal to a preset number threshold, or the face recognition box with the largest area is greater than or equal to a preset area threshold.

[0161] In one embodiment, based on the above embodiments, the aforementioned correction module 40 is specifically used to: identify low-confidence targets and high-confidence targets in the image to be processed; wherein, a low-confidence target is a portrait region without a corresponding face recognition box; a high-confidence target is a portrait region with a corresponding face recognition box; the face region of a high-confidence target whose face deformation exceeds a preset range is determined as a high-confidence face region to be corrected, and a correction model is used to correct the high-confidence face region to be corrected.

[0162] In one embodiment, based on the above embodiment, the device further includes a protection module, configured to: calculate the aspect ratio of the low-confidence target; if the aspect ratio is greater than a preset aspect ratio threshold, determine whether the face region of the low-confidence target is adjacent to the high-confidence face region to be corrected; if so, determine the gray value of the low-confidence target in the image to be processed as the gray value of the low-confidence target in the corrected image.

[0163] In one embodiment, based on the above embodiments, the protection module is specifically used to: expand the face recognition box of the low-confidence target outward according to a preset expansion ratio to obtain the target expansion box; determine whether the target expansion box intersects with the high-confidence face region to be corrected; if so, determine that the face region of the low-confidence target is adjacent to the high-confidence face region to be corrected.

[0164] The division of the various modules in the above-described image processing device is only for illustrative purposes. In other embodiments, the image processing device may be divided into different modules as needed to complete all or part of the functions of the above-described image processing device.

[0165] For specific limitations regarding the image processing apparatus, please refer to the limitations on the image processing method above, which will not be repeated here. Each module in the aforementioned image processing apparatus can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the operations corresponding to each module.

[0166] Figure 16 This is a schematic diagram of the internal structure of an electronic device in one embodiment. For example... Figure 16 As shown, the electronic device includes a processor and a memory connected via a system bus. The processor provides computing and control capabilities to support the operation of the entire electronic device. The memory may include non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The computer programs can be executed by the processor to implement an image processing method provided in the following embodiments. The internal memory provides a cached runtime environment for the operating system computer programs in the non-volatile storage media. The electronic device can be any terminal device such as a mobile phone, tablet computer, PDA (Personal Digital Assistant), POS (Point of Sales), in-vehicle computer, wearable device, etc.

[0167] The various modules in the image processing apparatus provided in this application embodiment can be implemented in the form of a computer program. This computer program can run on a terminal or server. The program modules constituted by this computer program can be stored in the memory of an electronic device. When the computer program is executed by a processor, it implements the steps of the method described in the embodiments of this application.

[0168] This application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of an image processing method.

[0169] A computer program product containing instructions that, when run on a computer, causes the computer to perform an image processing method.

[0170] Any references to memory, storage, databases, or other media used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0171] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An image processing method, characterized in that, include: Determine the arrangement of multiple portraits in the group photos to be processed; The arrangement includes at least one of the following: the people in the group photo are arranged neatly; the people in the group photo are not arranged neatly; the degree of neatness of the people in the group photo; and at least one region in the group photo where the people are arranged neatly. If the arrangement of multiple portraits in the group photo is neat, the correction intensity of the correction model is reduced, and the multiple portraits are merged and corrected according to the adjusted correction model; wherein, the correction model is used to adjust the projection method of the image to reduce the perspective distortion of the image; the neat arrangement indicates that the distance between the arrangement position of the multiple portraits and the reference line is within a preset range; the reference line is a horizontal line in the group photo used to measure the arrangement of the portraits.

2. The method according to claim 1, characterized in that, Determining the arrangement of multiple portraits in the group image to be processed includes: Multiple portrait regions detected from the group photos are merged to obtain the merged processing area of ​​the group photos; The arrangement of the multiple portraits is determined based on the position of the merged processing area in the group image.

3. The method according to claim 2, characterized in that, Determining the arrangement of the multiple portraits based on the position of the merged processing region in the group image includes: Extract the contour curve of the merged processing area; If the contour curve is located in a preset area of ​​the image to be processed, then the arrangement is determined to be neat; the preset area is located between two horizontal reference lines.

4. The method according to claim 3, characterized in that, After extracting the contour curve of the merged processing region, the method further includes: If the contour curve has a portion that extends beyond the preset area, then a target face recognition frame is determined in each face recognition frame corresponding to the merged processing area; the distance between the lower border of the target face recognition frame and the bottom edge of the group image is minimized. The area between the horizontal line containing the lower border and the lower edge of the contour curve within the merged processing area is defined as the target area. The target area is divided into multiple sub-regions along the vertical direction; The arrangement of multiple portraits in the group image is determined based on the area of ​​each sub-region.

5. The method according to claim 4, characterized in that, Determining the arrangement of multiple portraits in the group image based on the area of ​​each sub-region includes: Calculate the area ratio between any two sub-regions; If all the area ratios are within a preset threshold range, then the arrangement is determined to be neat. If any area ratio exceeds the preset threshold range, the arrangement is determined to be irregular.

6. The method according to any one of claims 1-5, characterized in that, Before determining the arrangement of multiple portraits in the group photo to be processed, the process further includes: Perform face recognition on the image to be processed to obtain the face recognition box corresponding to the image to be processed; The number of face recognition boxes corresponding to the image to be processed is counted, and the face recognition box with the largest area in each face recognition box is obtained; If the number of face recognition boxes is greater than a preset number threshold, and the face recognition box with the largest area is smaller than a preset area threshold, then the image to be processed is determined to be the group photo image to be processed.

7. The method according to claim 6, characterized in that, After counting the number of face recognition bounding boxes corresponding to the image to be processed and obtaining the face recognition bounding box with the largest area among all face recognition bounding boxes, the method further includes: If the number of face recognition boxes is less than or equal to a preset number threshold, or the face recognition box with the largest area is greater than or equal to a preset area threshold, then the correction model is used to correct the portraits in the image to be processed.

8. The method according to claim 7, characterized in that, The step of using the correction model to correct the portraits in the image to be processed includes: Identify low-confidence targets and high-confidence targets in the image to be processed; wherein, the low-confidence targets are portrait regions that do not have corresponding face recognition bounding boxes; and the high-confidence targets are portrait regions that have corresponding face recognition bounding boxes. The facial regions of high-confidence targets whose facial deformation exceeds the preset range are identified as high-confidence facial regions to be corrected, and the correction model is used to correct the high-confidence facial regions to be corrected.

9. The method according to claim 8, characterized in that, After identifying low-confidence targets and high-confidence targets in the image to be processed, the process further includes: Calculate the aspect ratio of the low-confidence target; If the aspect ratio is greater than a preset aspect ratio threshold, determine whether the face region of the low-confidence target is adjacent to the high-confidence face region to be corrected; If so, the grayscale value of the low-confidence target in the image to be processed is determined as the grayscale value of the low-confidence target in the corrected image.

10. The method according to claim 9, characterized in that, Determining whether the face region of the low-confidence target is adjacent to the high-confidence face region to be corrected includes: According to a preset expansion ratio, the facial recognition bounding box of the low-confidence target is expanded outward to obtain the target expansion bounding box; Determine whether the target bounding box intersects with the high-confidence face region to be corrected; If so, determine that the face region of the low-confidence target is adjacent to the high-confidence face region to be corrected.

11. An image processing apparatus, characterized in that, include: The determination module is used to determine the arrangement of multiple portraits in the group image to be processed; The arrangement includes at least one of the following: the people in the group photo are arranged neatly; the people in the group photo are not arranged neatly; the degree of neatness of the people in the group photo; and at least one region in the group photo where the people are arranged neatly. The correction module is used to reduce the correction intensity of the correction model when the arrangement of multiple portraits in the group photo is neat, and to merge and correct the multiple portraits according to the adjusted correction model; wherein, the correction model is used to adjust the projection mode of the image to reduce the perspective distortion of the image; the neat arrangement indicates that the distance between the arrangement position of the multiple portraits and the reference line is within a preset range; the reference line is a horizontal line in the group photo used to measure the arrangement of the portraits.

12. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, the processor performs the steps of the image processing method as described in any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 10.

Images