Figure image cutting adaptation method based on face detection

Abstract

The invention discloses a human face detection-based figure image cutting adaptation method. The method comprises the following steps of: performing human face detection on an image to obtain image human face information; when the human face is not detected, cutting a target image in the middle area of the original image; when a human face is detected, selecting one piece of human face information in a human face detection result as a human face information reference of the current image, and cutting and adapting the image based on the human face information reference; when the cut image has transverse and vertical lines penetrating through the image, removing the transverse and vertical lines, and adjusting the face information reference to carry out adaptation again; and cutting the adaptive result to obtain a target image conforming to the target size proportion. According to the method, different cutting adaptation methods are adopted according to the ratio of the face area to the image area, the image aspect ratio serves as the cutting adaptation target, different cutting methods are selected according to the face information and the difference between the face width and the target width, and good cutting adaptation results can be obtained for various image types.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and is a method for cropping and adapting human images based on face detection. Background Technology

[0002] With the development of smart devices, the resolution of images captured by users is increasing, leading to a corresponding increase in storage costs. Therefore, in network communication, it is necessary to compress or crop various images. This is primarily for user-uploaded images, which need to be cropped to save storage and transmission costs. Another application is cropping out key information for pre-display; users can then choose to view the original image when needed, saving resources compared to displaying all images at once. Furthermore, existing image display devices have fixed aspect ratios, while images have varying aspect ratios. This results in a mismatch between the original image and the display device, requiring manual adjustments to fit the image content and fill the display screen for easy viewing. This leads to significant adjustment work, especially for scenarios requiring large-scale image browsing. Therefore, to preserve the main information of the original image without loss of accuracy (resolution), images need to be cropped to retain as much of the subject information as possible and adapt to the display device. However, manual cropping is labor-intensive and lacks precise control over the cropped size.

[0003] User-uploaded images serve a wide range of purposes for showcasing personal profiles, especially for professionals like actors and models. They typically need to upload a headshot as part of their professional documentation, usually a single image submitted when applying for modeling jobs. These headshots often feature the same person taken from different angles and against various backgrounds. When cropping and adapting these user-uploaded images, it's crucial to preserve the subject's information; otherwise, cropping out key features may result in a poor display.

[0004] Portrait photos typically vary in size, shooting angle, distance, and pose, requiring appropriate cropping to highlight key information. Displaying these images directly without cropping and adaptation can result in a cluttered and unbalanced appearance, severely impacting the reading experience. Additionally, users sometimes upload model cards as profile photos. A model card (MOKA) serves as a model's instruction manual, typically displaying multiple representative photos of the model on a single image interface to showcase their body measurements, appearance, etc., with each photo separated by a border. If a user uploads a template image as their profile picture, using common image scaling methods for size adaptation will result in a smaller proportion of the person in the adapted image. If the profile picture is cropped and adapted directly based on the target image size and the person's main information, the special nature of template images may easily result in horizontal and vertical dividing lines in the cropped and adapted image, affecting the image display effect. Therefore, in order to obtain a better visual effect, the profile picture of the person needs to be cropped appropriately and in a targeted manner, taking into account the adaptation issues of special images, and resolving the horizontal and vertical dividing line issues that may occur during the adaptation and cropping process.

[0005] Currently, cropping portrait photos is usually done manually or by simply cropping the height or width of the image. These methods either require a high level of manual labor or result in partial truncation of the subject, leading to poor cropping results. For example, manual cropping may not be precise enough to accurately center the face in the thumbnail, impacting the user experience. Existing automatic scaling solutions result in the subject's information occupying too small a proportion, failing to showcase the subject's highlights. Furthermore, existing automatic cropping and adaptation solutions, when directly applied to portrait photo adaptation scenarios, do not achieve ideal cropping results, such as loss of subject information or the inclusion of horizontal and vertical dividing lines from the subject's profile in the cropped image. Therefore, this invention proposes a face detection-based cropping and adaptation method for portrait photos, especially for profile images of actors and models.

[0006] Chinese patent 201510745433 proposes an image cropping method and system for image capture. The solution in this patent determines the area of ​​the target cropping region based on the face area. However, the method of determining the cropping based on the area still has the problem of inconsistent image sizes, and the display adaptability on fixed display devices cannot be guaranteed. Summary of the Invention

[0007] The problems to be solved by the present invention are as follows: For the need to automatically adapt and crop an uploaded image according to its content, especially the human figure, into an image with the required size ratio for display, the existing technologies of directly scaling or directly detecting the target for fixed-size cropping have poor effects. Especially for a modular card image with regional attributes, how to avoid being affected by the horizontal and vertical frame lines in the modular card when adapting to crop the human figure is a problem that needs to be solved in automatic adaptation and cropping.

[0008] The technical solution of the present invention is: A method for cropping and adapting a human image based on face detection, which crops and adapts an image according to the target size ratio and the human figure to obtain a target image that meets the target size ratio, including:

[0009] 1) Perform face detection on the image, and perform step 2) or 3) according to the detection result;

[0010] 2) If no face information is detected, directly crop the image according to the target size ratio, and output the adapted cropping result;

[0011] 3) If face information is detected, weight the face information detection result. The weighting method is to sort in descending order according to the face area, select a suitable result as the face information benchmark, and crop and adapt the image based on this face information benchmark, including:

[0012] 3.1) Obtain the face information, sort according to the face area, and select the Kth face detection result as the face information benchmark of the current image, N is the number of detected faces, and K is a set value;

[0013] 3.2) Calculate the ratio p of the face area of the face information benchmark to the image area, set a threshold P, and the range of P is 0 to 1. When p >= P, crop the target size ratio image from the face information benchmark according to the method of shrinking from the outside to the inside; when p < P, crop the target size ratio image from the face information benchmark according to the method of expanding from the inside to the outside;

[0014] 3.3) Detect whether there are horizontal and vertical lines passing through the cropped image. If so, take the largest rectangular area obtained by the intersection of the horizontal and vertical lines as the new image to be cropped, calculate the ratio q of the face area of the face information benchmark at this time to the area of the largest rectangular area, set a threshold Q, and the range of Q is 0 to 1. When q >= Q, adjust the face information benchmark to the intersection with the largest rectangular area; when q < Q, regard the entire largest rectangular area at this time as the face, update the face information benchmark to the largest rectangular area, and then re-crop according to step 3.2) using the new face information benchmark and the new image area to be cropped; if not, output the cropped image as the adapted cropping result.

[0015] Furthermore, in section 3.2), the method of cropping the target image from the outside in using the facial information reference includes:

[0016] First, calculate the target height based on the ratio of the original image width to the target size.

[0017] If the image height is greater than the target height, compare the target height with the face height. If the face height is greater than the target height, it means that the target height cannot contain the entire face. Vertically shrink the upper and lower boundaries of the face information reference to the target height. If the face height is less than the target height, it means that the target height can contain the entire face. Vertically expand the upper and lower boundaries of the face information reference to the target height.

[0018] If the image height is less than the target height, the target width is calculated based on the image width. If the face width is greater than the target width, it means that the target width cannot contain the entire face, and the left and right boundaries of the face information reference are horizontally shrunk to the target width. If the face width is less than the target width, it means that the target width can contain the entire face, and the left and right boundaries of the face information reference are horizontally expanded to the target width.

[0019] Cropping and fitting of the image based on the shrunk or expanded boundaries.

[0020] Furthermore, in section 3.2), the target image is cropped using an inside-out expansion method based on facial information, including:

[0021] Calculate the head-to-shoulder ratio (head_to_shoulder) and the face height to calculate the target width (target_width), and then calculate the target height (target_height) based on the aspect ratio (aspect_ratio) of the target size.

[0022] When target_height<=height, the original image is a super-height image relative to the target image. Based on the face information benchmark, the left and right cropping boundary values ​​are calculated based on the vertical bisector and half of the target width. Based on the horizontal bisector and the face height, the top and bottom cropping boundary values ​​are calculated.

[0023] When target_height>height, the original image is an ultra-wide image relative to the target image. The upper and lower boundaries are the upper and lower boundaries of the original image. Based on the face information benchmark, the left and right cropping boundary values ​​are calculated based on the vertical bisector and 1 / 2 of the target width.

[0024] Furthermore, when no face information is detected, center cropping is performed: using the shorter side length as a reference, the aspect ratio of the original image is compared with the aspect ratio of the target image. When ratio <= aspect ratio, the original image is a super-height image relative to the target image; when ratio > aspect ratio, the original image is a super-wide image relative to the target image. If the original image is a super-height image relative to the target image, the top and bottom boundaries are shrunk based on the horizontal bisector of the original image, and the image is cropped vertically. If the original image is a super-wide image relative to the target image, the left and right boundaries are shrunk based on the vertical bisector of the original image, and the image is cropped horizontally.

[0025] Furthermore, before performing face detection, images of different formats are converted into a unified format for processing; input images whose short side length exceeds a set threshold are scaled proportionally so that their short side length equals the set threshold, thus ensuring that the short side length of all input images is less than or equal to the preset length threshold.

[0026] Furthermore, after the target image is adapted and cropped, the target image is scaled proportionally so that its short side length is equal to the set threshold. In this way, the short side length of all output images should be greater than or equal to the preset length threshold.

[0027] Furthermore, the detection of horizontal and vertical lines running through the image is as follows: obtain the background color pixel values ​​of the image, perform binarization processing on the image based on the background color pixel values, perform an opening operation on the binarized image to extract the horizontal and vertical lines, and calculate the bounding rectangles of each as the detection results.

[0028] This invention proposes an automatic cropping method for portrait photos based on face detection. This method improves the automatic cropping effect of portrait photos, ensuring that the main body of the subject (below the head, above the chest, and including the shoulders) is completely cropped. It overcomes the shortcomings of simple automatic cropping techniques that suffer from poor cropping results due to neglecting the main subject, while also reducing manpower requirements and improving display quality. Compared to existing image adaptation methods, this invention adaptively selects different cropping methods based on aspect ratio and face area information, thereby achieving better adaptation and cropping results.

[0029] Compared to existing image cropping and adaptation methods that often rely solely on face position and area information to directly obtain the area surrounding the face as the cropping result, this invention uses the image aspect ratio as the cropping and adaptation target. It selects different cropping methods not only based on face information but also on the difference between the face's width and height and the target's width and height. This invention proposes first using different cropping and adaptation methods based on the ratio of the face area to the image area. These different cropping methods also consider the proportions of various parts of the human body to preserve the main subject information as much as possible. Furthermore, existing image cropping and adaptation methods often fail to consider situations where users upload group photos or template images as profile pictures, resulting in cropped results with template image dividing lines that affect the visual appeal. This invention provides additional processing for these special cases, achieving good cropping and adaptation results for various image types. Attached Figure Description

[0030] The present invention is illustrated by way of example through one or more embodiments and the corresponding pictures in the accompanying drawings. It should be noted that these illustrative descriptions do not constitute a limitation on the embodiments.

[0031] Figure 1 This is the main flowchart of the method for adapting human image photos in this invention.

[0032] Figure 2 This is a flowchart of the image adaptation method for facesless images used in the human image photo adaptation method of this invention.

[0033] Figure 3 This is a flowchart of the image adaptation method based on facial information used in the portrait photo adaptation method of this invention.

[0034] Figure 4 This is a schematic diagram of a method for adapting and cropping a portrait photo of a person whose facial information has not been detected, provided by an embodiment of the present invention.

[0035] Figure 5 This is a schematic diagram of a method for adapting and cropping a portrait photo that has detected individual facial information, provided by an embodiment of the present invention.

[0036] Figure 6 This is a schematic diagram of a method for adapting and cropping a portrait photo of a person with multiple detected facial information and no horizontal or vertical lines in the preliminary adaptation results, provided by an embodiment of the present invention.

[0037] Figure 7 This is a schematic diagram of a method for adapting and cropping a portrait photo with multiple detected facial information and horizontal and vertical lines in the preliminary adaptation result, provided by an embodiment of the present invention.

[0038] Figure 8This is a schematic diagram of a method for cropping and adapting human images using an inside-out expansion method, provided by an embodiment of the present invention.

[0039] Figure 9 This is a schematic diagram of a method for cropping and adapting a portrait image using an outside-to-inside shrinkage method, provided by an embodiment of the present invention. Detailed Implementation

[0040] This invention provides an automatic image cropping and adaptation method based on face detection. It involves performing face detection on an image to obtain facial information, and then cropping the image accordingly. This method is particularly suitable for cropping portraits, especially model images. The cropping objective of this invention is to obtain an image that conforms to a target aspect ratio for display at the target size, without considering the image size itself. The image size can be directly enlarged or reduced proportionally without affecting the proportions of the people in the image. Different image display devices generally have fixed aspect ratios, but the aspect ratio of an image may not be suitable for the display device. Therefore, this invention proposes an image cropping and adaptation method to obtain a cropped image that conforms to the target aspect ratio without affecting the proportions of the people in the image.

[0041] Optionally, before performing face detection on the images to be adapted, the process further includes: converting images of different formats into a unified format for processing; and scaling input images whose shorter side length exceeds a set threshold proportionally so that their shorter side length equals the set threshold. Thus, the shorter side length of all input images should be less than or equal to a preset length threshold. Further, the set length threshold is preferably 1280. This step is to unify the format and size of different original images uploaded by different users to facilitate subsequent length and width related calculations.

[0042] Optionally, after adapting the original image to obtain the target image, the method further includes: scaling the target image proportionally so that its shorter side length is greater than or equal to a set threshold. Further, the set length threshold is preferably 500. This step is to scale the cropped and adapted image according to the device size for easier display on the device.

[0043] When the number of faces N in the face detection result is 0, it means that no face information was detected in the image. The original image is then directly cropped to fit the target size ratio.

[0044] The preferred cropping method is as follows: First, calculate the horizontal and vertical bisectors of the original image. Since the main information of most images is located in the center of the image, cropping and adaptation are performed based on the center point.

[0045] Calculate the aspect ratio of the original image and set the target aspect ratio (aspect_ratio), which ranges from 0 to ∞. When ratio <= aspect_ratio, the original image is significantly taller than the target image; the left and right boundaries are the original image's left and right boundaries. Using the horizontal bisector as a reference, calculate the top and bottom cropping boundaries based on the target height and crop the original image to obtain the cropped result. When ratio > aspect_ratio, the original image is significantly wider than the target image; the top and bottom boundaries are the original image's top and bottom boundaries. Using the vertical bisector as a reference, calculate the left and right cropping boundaries based on the target width and crop the original image to obtain the cropped result.

[0046] This step aims to preserve as much information as possible from the center of the image. Compared to existing image cropping and adaptation methods that crop both horizontally and vertically simultaneously, this invention only crops one of the horizontal or vertical directions, thus achieving better information preservation.

[0047] When the number of faces (N) detected is greater than 0, it indicates that facial information has been detected in the image. The facial information detection results are weighted, sorted in descending order by face area, and a suitable result is selected as the facial information baseline. Based on this baseline, the user-uploaded image is cropped and adapted. Specifically, based on face area, the Kth face detection result is selected as the facial information baseline for the current image, where K ranges from 0 to N, and N is the number of detected faces. This facial information baseline only includes the human face detection results. This invention needs to consider the integrity of the cropped image; therefore, subsequent cropping will be performed based on the baseline using an outside-to-inward contraction method and an inside-to-outward expansion method, calculating the specific cropping range. For example, the head-to-shoulder ratio is used to calculate the expansion to retain more subject information. In this invention, unless otherwise specified, the face region or facial information baseline refers to the result directly detected by the face detector, because facial information must be preserved first, and then the shoulder and neck information of the person will be considered.

[0048] The ideal adaptation result is a portrait area extending from the top of the head to above the chest line and including the shoulders. Selecting a face that is too large may result in the loss of other facial information, especially important for casting actors and models who need to consider not only physical features but also makeup and appearance to avoid poor adaptation results such as large headshots. Conversely, selecting a face that is too small may lead to false positives from the face detector, detecting non-face areas. Furthermore, adaptation results based on smaller face areas may exhibit more severe image blurring when scaled to the target display resolution. Therefore, this invention preferably sets the K value as... That is, facial information centered in area. This indicates rounding up to the nearest integer.

[0049] After obtaining the face information reference, the image is cropped and adapted, specifically including:

[0050] Calculate the ratio p of the face area of the face information reference to the image area. Here, the image area refers to the area of the original image. Set a threshold P, and the range of P is 0 to 1. When p >= P, crop the target size ratio image from the outside to the inside according to the face information reference; when p < P, crop the target image from the inside to the outside according to the face information reference. This step is to select different cropping methods according to the ratio of the face area to the image area.

[0051] Furthermore, considering that the ideal adaptation result is the main portrait area of the person. If the face area is too large, direct cropping may lose other person information except the face; if the face area is too small, direct cropping may cause more serious image blurring problems. Therefore, the threshold P value is preferably set to 0.5.

[0052] Crop the target image from the outside to the inside according to the face information reference, specifically including:

[0053] The cropping of the face information reference from the outside to the inside to obtain the target image in 3.2) includes:

[0054] First, calculate the target height based on the original image width and the target size ratio,

[0055] If the image height > the target height, judge the target height and the face height. When the face height > the target height, it means that the target height cannot contain the entire face, and vertically contract the upper and lower boundaries of the face information reference to the target height; when the face height < the target height, it means that the target height can contain the entire face, and vertically expand the upper and lower boundaries of the face information reference to the target height;

[0056] If the image height < the target height, then calculate the target width based on the image width. When the face width > the target width, it means that the target width cannot contain the entire face, and horizontally contract the left and right boundaries of the face information reference to the target width; when the face width < the target width, it means that the target width can contain the entire face, and horizontally expand the left and right boundaries of the face information reference to the target width;

[0057] Crop and adapt the image according to the contracted or expanded boundaries.

[0058] The aforementioned expansion refers to additionally cropping the area around the face from the original image to fill areas where facial information is insufficient, without changing the image size. Similarly, contraction refers to shrinking the area around the facial information reference vertically or horizontally, without changing the image size. Specifically, when calculating the required expansion height based on the face height, to preserve as much of the person's hair as possible, the upper boundary value is expanded proportionally based on the face height, preferably at a ratio of 0.6. The upper boundary is extended upwards by 0.6 * the facial information reference height per unit pixel, and then the position of the lower boundary is calculated based on the target height.

[0059] In the outside-in shrinkage cropping method, let the aspect ratio of the original image be 'ratio' and the aspect ratio of the target image be 'aspect_ratio'. When 'ratio' <= 'aspect_ratio', it means the original image is much taller than the target image (image height > target height), and cropping is performed to adapt the height. When 'ratio' > 'aspect_ratio', it means the original image is much wider than the target image (image height < target height), and cropping is performed to adapt the width. As a preferred solution, the target aspect ratio 'aspect_ratio' is actually set to 0.65.

[0060] The target image is cropped using an inside-out expansion method based on facial information benchmarks, specifically including:

[0061] The target width, target_width, is calculated based on the head-to-shoulder ratio and the face height.

[0062] The target height (target_height) is calculated based on the target aspect ratio (aspect_ratio, which ranges from 0 to ∞) and the target width.

[0063] When `target_height <= height`, it indicates that the original image is hyper-height relative to the target image. Based on the facial information baseline and the vertical bisector, the left and right cropping boundary values ​​are calculated to be half the target width. Based on the horizontal bisector, the top and bottom cropping boundary values ​​are calculated according to the face height.

[0064] When target_height > height, it means that the original image is an ultra-wide image relative to the target image. The upper and lower boundaries are the upper and lower boundaries of the original image. Based on the face information benchmark and the vertical bisector, the left and right cropping boundary values ​​are calculated to be 1 / 2 of the target width.

[0065] The calculation of the upper and lower cropping boundary values ​​based on the face height includes: in order to preserve the hair of the person as much as possible, the upper boundary value is calculated proportionally based on the face height, with the preferred ratio being 1.1; and the lower boundary is calculated based on the target height.

[0066] As a preferred option, the target aspect ratio is set to 0.65, and the head-to-shoulder ratio is set to 2.2.

[0067] The above-described method of cropping the target image from the outside in based on facial information aims to preserve as much information as possible about the area around the face when the ratio of face area to image area is large. The above-described method of cropping the target image from the inside out based on facial information aims to preserve as much information about the face and hair area as possible when the ratio of face area to image area is small. Compared to existing image cropping and adaptation methods, which often directly crop the information around the face proportionally, this invention selects different cropping methods based on the ratio of face area to image area to achieve greater preservation of facial information. Compared to manual cropping, which easily overlooks fine hairline information, this invention preserves as much hair information as possible.

[0068] For the detection of modular images, this invention performs horizontal and vertical dividing line detection on the initially cropped thumbnail image, specifically including:

[0069] Obtain the background color pixel values ​​of the user-uploaded image, perform binarization processing on the image based on the background color pixel values, perform opening operation on the binarized image to extract horizontal and vertical lines, and calculate the bounding rectangles of each as the detection results.

[0070] When the initially cropped image contains horizontal or vertical lines that run through the image (i.e., region dividing lines in the template photo), remove these lines from the cropped result, adjust the facial information baseline, and perform a second adaptation to achieve more refined image adaptation and cropping. Specifically, this includes:

[0071] Remove the horizontal and vertical lines from the initial cropping area to obtain a rectangular region containing only the person's information. Calculate the proportion of the face within this rectangular region and set a threshold Q, where Q ranges from 0 to 1.

[0072] When the proportion of the face region within the rectangular region is greater than a threshold, it indicates a large amount of facial information in the region. The intersection of the face region and the rectangular region is used as the facial information reference, and further adaptation and cropping are performed on the rectangular region, using the same cropping method as described above. When the proportion of the face region within the rectangular region is less than a threshold, it indicates a small amount of facial information in the region. The rectangular region is used as the facial information reference, and further adaptation and cropping are performed on the rectangular region. This step addresses the situation where users directly upload template images or group photos as profile pictures. Compared to existing image cropping and adaptation methods that do not consider template image cropping and adaptation scenarios, this invention additionally processes this type of multi-person scenario to achieve better cropping results in various situations.

[0073] To clearly and completely describe the purpose, technical solution, and advantages of the embodiments of the present invention, the technical solution of the present invention will be fully and thoroughly described below with reference to the accompanying drawings of the embodiments of the present invention. It should be noted that the described embodiments are only a part of the present invention, and not all of the embodiments. The accompanying drawings are only used to illustrate preferred embodiments and are not intended to limit the present invention.

[0074] Example 1 (Main Process)

[0075] This invention provides a method for cropping and adapting portrait photos, the main process of which is as follows: Figure 1 As shown in the figure, this embodiment details the main process of cropping and adapting human image photos according to the present invention, including the following operations:

[0076] Preprocessing of portrait photos.

[0077] Optionally, the format of user-uploaded images in different formats is determined, and all are converted to a unified image format for processing; the length and width of the input images are determined, and input images with a shorter side exceeding a set threshold are scaled proportionally to make their shorter side equal to the set threshold. Thus, the shorter side length of all input images should be less than or equal to the preset set threshold. Further, the set threshold is preferably 1280.

[0078] Perform face detection on portrait photos.

[0079] In this embodiment, the portrait photo can be the original input image or a scaled-down input image. Face detection algorithms include, but are not limited to, template matching-based methods and deep learning-based methods. Different processing methods are selected based on the detected faces.

[0080] If the number of detected faces is 0, it means that no face information was detected in the image, and the image adaptation method for images without detected faces is used.

[0081] If no facial information is detected in the user's uploaded profile photo, it means that the person's face occupies too small a proportion in the image or has low clarity.

[0082] If the number of detected faces is greater than 0, it means that facial information has been detected in the image. The facial information detection results are weighted, and the appropriate result is selected as the facial information benchmark. The specific selection logic is the same as in Example 4. The uploaded image is cropped and adapted based on the facial information benchmark. The specific cropping logic is the same as in Example 3.

[0083] The initial cropping results are subjected to horizontal and vertical line detection. If horizontal and vertical lines that run through the image are detected in the initial cropped image, i.e., the area dividing lines in the model card photo, the lines that run through the image are removed, i.e., the area dividing lines in the model card. The facial information baseline is adjusted and the image is adapted again to achieve more refined image adaptation and cropping. The specific cropping logic is the same as in Example 5.

[0084] Post-processing of portrait photos.

[0085] Optionally, after cropping and adapting the user-uploaded original image to obtain the target image, the width and height of the output image are determined. Input images with a longer side shorter than a set length threshold are scaled proportionally to make their longer side equal to the set length threshold. Thus, the longer side length of all output images should be greater than or equal to the preset length threshold. Further, the length threshold is preferably 500.

[0086] Example 2 (No Face)

[0087] This embodiment describes an image adaptation method for images where no faces are detected. For most photos without people, the main image information is located in the center of the image. Therefore, the central portion of the original image is directly cropped and adapted based on the original image size and the target image size. The main process is as follows: Figure 2 As shown in the diagram, the cropping and fitting are illustrated below. Figure 4 As shown in (a) and (b), the following operations are included:

[0088] Calculate the horizontal and vertical bisectors and aspect ratio of the original image, and set the target aspect ratio (aspect_ratio), which ranges from 0 to ∞.

[0089] When ratio <= aspect_ratio, it means that the original image is super-high relative to the target image. In this case, the left and right boundaries of the target cropping region are set to the left and right boundaries of the original image. Based on the horizontal bisector, the upper and lower cropping boundaries are calculated to be 1 / 2 of the target height. The original image is then cropped to obtain the cropping result.

[0090] When ratio > aspect_ratio, it means that the original image is an ultra-wide image relative to the target image. In this case, the upper and lower boundaries of the target cropping region are the upper and lower boundaries of the original image. Based on the vertical bisector, the left and right cropping boundaries are calculated according to the target width, each being 1 / 2 of the target width. The original image is then cropped to obtain the cropping result.

[0091] Furthermore, the target aspect ratio is actually set to 0.65. It should be noted that the target aspect ratio is not limited to this value, and the equality of size relationships is not limited to a portion. This is only an illustration of one case in this embodiment.

[0092] Example 3 (Single face without horizontal and vertical lines)

[0093] For the portrait photo with detected face information in this embodiment, select the appropriate result as the face information benchmark. Considering that the thumbnail of the portrait photo needs to highlight the person information as much as possible, the original image is cropped and adapted according to the face information benchmark and the target image size. The main process is as Figure 3 shown, and the cropping and adaptation schematic diagram is as Figure 5 (a), (b), (c) of Figure 6 (a), (b) of

[0094] First, take the only face information detection result as the face information benchmark.

[0095] Calculate the ratio p of the face area to the image area, and set the threshold P. The range of P is 0 to 1. When p >= P, crop the target image by the inward contraction method according to the face information, and the specific adaptation logic is the same as that of Example 6; when p < P, crop the target image by the outward expansion method according to the face information, and the specific adaptation logic is the same as that of Example 7.

[0096] Furthermore, considering that the ideal adaptation result is the main portrait area of the person. If the face area is too large, direct cropping may lose other person information except the face; if the face area is too small, direct cropping may cause more serious image blurring problems. Therefore, the threshold P value is preferably set to 0.5.

[0097] Example 4 (Multiple faces without horizontal and vertical lines)

[0098] For the portrait photo with detected multiple face information in this embodiment, select the appropriate result as the face information benchmark. Considering that the thumbnail of the portrait photo needs to highlight the person information as much as possible, the original image is cropped and adapted according to the face information benchmark and the target image size. The main process is as Figure 3 shown, and the cropping and adaptation schematic diagram is as Figure 7 (a), (b) of

[0099] First, weight the face information detection results. Sort the faces according to the face area, and select the Kth face detection result as the face information benchmark of the current image. The range of K is 0 to the number of faces.

[0100] Further, considering that the ideal adaptation result is a portrait area below the head and above the bust line and including both shoulders. If a face with too large an area is selected, other character information except the face may be lost. Especially for the recruitment of occupations such as actors and models, in addition to the appearance of the person, attention should also be paid to makeup and grooming. As a result, adaptation results such as headshots with poor effects may occur. If a face with too small an area is selected, on the one hand, it may be a false positive generated by the face detector, that is, an image area that is actually not a face is detected. On the other hand, for the adaptation result obtained based on a smaller face area, when scaled to the target display resolution, more serious image blurring problems may occur. Therefore, the K value is preferably set to 1 / 2 * the number of faces, that is, the face information with a medium area.

[0101] Calculate the ratio p of the face area to the image area, and set a threshold P. The range of P is 0 to 1. When p >= P, the target image is cropped by the inward contraction method according to the face information, and the specific adaptation logic is the same as that of Embodiment Six. When p < P, the target image is cropped by the outward expansion method according to the face information, and the specific adaptation logic is the same as that of Embodiment Seven.

[0102] Further, considering that the ideal adaptation result is the main portrait area of the person. If the face area is too large, direct cropping may lose other character information except the face. If the face area is too small, direct cropping may cause more serious image blurring problems. Therefore, the threshold P value is preferably set to 0.5.

[0103] Embodiment Five (with horizontal and vertical lines)

[0104] Based on Embodiment Four, after obtaining the preliminary cropping result, horizontal and vertical dividing lines of the preliminarily cropped thumbnail image are detected. When there are horizontal and vertical lines that penetrate the image, that is, the area dividing lines in the model card photo, the penetrating lines in the cropping result, that is, the area dividing lines in the model card, are removed, and the face information benchmark is adjusted and adapted again to achieve more refined picture adaptation and cropping, including the following operations:

[0105] The image is binarized, and open operations are performed on the binarized image to extract horizontal and vertical lines, and the circumscribed rectangles are calculated respectively as the horizontal and vertical line detection results.

[0106] Remove the horizontal and vertical line parts in the initial cropping result to obtain a rectangular area with only character information; calculate the proportion of the face area in the rectangular area, and set a threshold Q. The range of Q is 0 to 1.

[0107] Further, the threshold Q value is preferably 0.5.

[0108] When the proportion of the face region in the rectangular region is greater than the threshold, it indicates that there is a lot of face information in the region. The part where the face region and the rectangular region intersect is used as the face information reference, and the rectangular region is adapted and cropped again. The specific cropping method is the same as in Example 4.

[0109] When the proportion of the face region in the rectangular region is less than the threshold, it indicates that there is less face information in the region. The rectangular region is used as the face information reference, and the adaptation and cropping are performed again on the rectangular region. The specific cropping method is the same as in Example 4.

[0110] Example 6 (Outside-in contraction method)

[0111] This embodiment illustrates how to crop a target image from the outside in using a shrinking method based on facial information. Figure 9 As shown, it specifically includes:

[0112] Calculate the aspect ratio of the original image, and set the target aspect ratio (aspect_ratio), which ranges from 0 to ∞. Specifically, the target aspect ratio is set to 0.65.

[0113] When ratio <= aspect_ratio, it means that the original image is super-high relative to the target image. In this case, cropping and adaptation are performed based on the face height and the target height, and the left and right boundaries of the target cropping area are set to the left and right boundaries of the original image.

[0114] When the face height is greater than or equal to the target height, it indicates that the target height cannot encompass the entire face. In this case, the face region is shrunk vertically based on the image information, with each vertical shrinkage value being half the difference between the face height and the target height. The original image is then cropped to obtain the cropping result.

[0115] When the face height is less than the target height, it indicates that the target height can encompass the entire face. In this case, the face region is expanded vertically based on the image information. To preserve as much of the hair as possible, the upper boundary value is expanded by a certain proportion of the face height. The lower boundary is calculated based on the target height, and the original image is cropped to obtain the cropped result. Furthermore, the preferred ratio is 0.6.

[0116] When ratio > aspect_ratio, it means that the original image is an ultra-wide image relative to the target image. In this case, cropping and adaptation are performed based on the face width and the target width, and the upper and lower boundaries of the target cropping area are set to the upper and lower boundaries of the original image.

[0117] When the face width is greater than or equal to the target width, it indicates that the target width cannot encompass the entire face. In this case, the face region is shrunk to the left and right based on the image information, with each shrunk value being half the difference between the face width and the target width. The original image is then cropped to obtain the cropped result.

[0118] When the face width is less than the target width, it means that the target width can include the entire face. In this case, the face area is expanded left and right based on the image information, and the expansion value is 1 / 2 of the difference between the face width and the target width. The original image is then cropped to obtain the cropping result.

[0119] It should be noted that the equality of size is not limited to a few cases; this is only one example from this embodiment.

[0120] Example 7 (Inside-Out Expansion Method)

[0121] This embodiment illustrates the method of cropping the target image from the inside out based on facial information, such as... Figure 8 As shown, it specifically includes:

[0122] The target width (target_width) is calculated based on the head-to-shoulder ratio (head_to_shoulder) and the face height. The target height (target_height) is calculated based on the target aspect ratio (aspect_ratio, ranging from 0 to ∞) and the target width. Furthermore, the actual target aspect ratio (aspect_ratio) is set to 0.65, and the preferred head-to-shoulder ratio (head_to_shoulder) is 2.2.

[0123] When `target_height <= height`, it indicates that the original image is hyper-height relative to the target image. Based on the facial information baseline and the vertical bisector, the left and right cropping boundary values ​​are calculated to be half the target width. To preserve as much of the person's hair as possible, the upper boundary value is calculated based on the horizontal bisector at a certain proportion of the face height. The lower boundary is calculated based on the target height, and the original image is cropped to obtain the cropped result. Furthermore, the preferred ratio is 1.1.

[0124] When target_height > height, it means that the original image is an ultra-wide image relative to the target image. In this case, the upper and lower boundaries of the target cropping region are set to the upper and lower boundaries of the original image. Based on the face information reference and the vertical bisector, the left and right cropping boundary values ​​are calculated to be 1 / 2 of the target width. The original image is then cropped to obtain the cropping result.

[0125] It should be noted that the equality of size is not limited to a few cases; this is only one example from this embodiment.

Claims

1. A face-detection-based character image cropping adaptation method, characterized in that Adapting and cropping the image according to the target size ratio and the figure image to obtain a target image conforming to the target size ratio, comprising: 1) performing face detection on the image, and performing step 2) or 3) according to the detection result; 2) if no face information is detected, directly cropping the image according to the target size ratio, and outputting the adaptation and cropping result; 3) if face information is detected, weighting the face information detection result, selecting a suitable result as a face information reference according to the descending order of face area, and cropping and adapting the image based on the face information reference, comprising: 3.1) Obtain face information, sort according to face area, select the Kth face detection result as the face information reference of the current image, N is the number of detected faces, and K is a set value. 3.2) calculating the ratio p of the area of the face information reference to the area of the image, setting a threshold P, P ranging from 0 to 1, when p >= P, cropping the face information reference according to the outside-in contraction method to obtain a target size ratio image; when p < P, cropping the face information reference according to the inside-out expansion method to obtain a target size ratio image; 3.3) detecting whether there are horizontal and vertical lines that pass through the image in the cropped image, if there are, taking the largest rectangular region obtained by the intersection of the horizontal and vertical lines as a new image to be cropped, calculating the ratio q of the face area of the face information reference to the area of the largest rectangular region, setting a threshold Q, Q ranging from 0 to 1, when q >= Q, adjusting the face information reference to the intersection with the largest rectangular region; when q < Q, regarding the entire largest rectangular region as a face at this time, updating the face information reference to the largest rectangular region, and then using the new face information reference and the new image region to be cropped to re-crop according to step 3.2); if not, outputting the cropped image as the adaptation and cropping result.

2. The method according to claim 1, characterized in that The outside-in contraction method for cropping the target image from the face information reference in 3.2) comprises: firstly calculating the target height based on the original image width and the target size ratio, if the image height > the target height, judging the target height and the face height, when the face height > the target height, it is indicated that the target height cannot contain the entire face, and the upper and lower boundaries of the face information reference are vertically contracted to the target height; when the face height < the target height, it is indicated that the target height can contain the entire face, and the upper and lower boundaries of the face information reference are vertically expanded to the target height; if the image height < the target height, then calculating the target width based on the image width, when the face width > the target width, it is indicated that the target width cannot contain the entire face, and the left and right boundaries of the face information reference are horizontally contracted to the target width; when the face width < the target width, it is indicated that the target width can contain the entire face, and the left and right boundaries of the face information reference are horizontally expanded to the target width; cropping and adapting the image according to the contracted or expanded boundaries.

3. The method for cropping and adapting person images based on face detection according to claim 1, characterized in that: The inside-out expansion method for cropping the target image from the face information reference in 3.2) comprises: calculating the head-to-shoulder ratio head_to_shoulder, and the face height to calculate the target width target_width, and the target height target_height according to the aspect ratio aspect_ratio of the target size ratio; When target_height<=height, the original image is a super-high image relative to the target image, and according to the face information benchmark, the left and right cutting boundary values are calculated based on the vertical bisector with 1 / 2 of the target width, and the upper and lower cutting boundary values are calculated based on the horizontal bisector according to the face height. When target_height>height, the original image is a super-wide image relative to the target image, and the upper and lower boundaries are the original image upper and lower boundaries, and according to the face information benchmark, the left and right cutting boundary values are calculated based on the vertical bisector with 1 / 2 of the target width.

4. The method for cropping and adapting person images based on face detection according to claim 1, characterized in that: When no face information is detected, center cutting is performed: taking the short side length as the benchmark, the ratio of the original image width-height ratio ratio and the aspect ratio of the target size ratio are compared, when ratio<=aspect_ratio, the original image is a super-high image relative to the target image, when ratio>aspect_ratio, the original image is a super-wide image relative to the target image, and if the original image is a super-high image relative to the target image, the upper and lower boundaries are contracted based on the horizontal bisector of the original image, and the upper and lower cutting images are cut; when the original image is a super-wide image relative to the target image, the left and right boundaries are contracted based on the vertical bisector of the original image, and the left and right cutting images are cut.

5. The method for cropping and adapting person images based on face detection according to claim 1, characterized in that: Before face detection, pictures of different formats are converted into a unified format for processing; for input images with a short side length exceeding a set threshold, the input images are scaled proportionally so that the short side length is equal to the set threshold, so that the short side length of all input images is less than or equal to the preset length threshold.

6. The method of claim 5, wherein the method further comprises: determining a face region of the image; and determining a face region of the image. The short side length threshold is set to 1280.

7. A method for cropping and adapting person images based on face detection according to claim 1, characterized in that: After the target image is obtained by adaptive cutting, the target image is scaled proportionally so that the short side length is equal to the set threshold, so that the short side length of all output images is greater than or equal to the preset length threshold.

8. The method according to claim 7, characterized in that The short side length threshold is set to 500.

9. The method of claim 1, wherein the method further comprises: determining a face region of the image; and determining a face region of the image. The detection of horizontal and vertical lines throughout the image is: obtaining the background color pixel value of the image, performing binaryzation processing on the image based on the background color pixel value, performing open operation on the binaryzation image to extract horizontal and vertical lines, and calculating the outer rectangle as the detection result, respectively.

Images