A method for automatically processing portrait photos into standard ID photos

Abstract

The present invention provides a method for automatically processing a portrait photo into a standard ID photo, which belongs to the field of computer vision and image processing technology, and specifically is a technology for automatically processing a portrait photo into an ID photo, which can automatically adjust the posture of the person so that the eyes of the person are level with the shoulders. The technology involves the fields of face detection, key point positioning, image rotation, background synthesis, image segmentation, shoulder key point detection and position adjustment, and is specifically applied to the automatic generation of ID photos. The present invention provides a method for automatically processing a portrait photo into a standard ID photo, which is different from traditional manual processing, can automatically process a portrait photo into an ID photo that meets standard requirements, and can automatically adjust the posture of the person so that the eyes of the person are level with the shoulders, eliminating the tediousness of traditional manual processing, improving processing efficiency, saving labor, and achieving higher processing accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to a method for processing a portrait photo into a standard ID photo. Background Art

[0002] With the rapid development of information technology, especially advancements in computer vision and artificial intelligence, automated ID photo generation is becoming increasingly necessary. Traditional ID photo capture typically requires manual operation, requiring the subject to maintain strict posture and standards. However, in practice, users may fail to achieve perfect posture for various reasons, resulting in photos that fail to meet standard ID photo requirements. For example, the eyes may not align with the shoulders, or the background may not meet regulatory requirements. Therefore, automatically generating ID photos that meet these standards has significant practical value.

[0003] Currently, some image processing technologies attempt to generate ID photos through face detection and posture correction. However, these methods often rely on manual adjustments or can only correct photos through simple static rotations. Furthermore, most technologies are unable to automatically adjust the alignment of shoulders and eyes or adequately account for differences in user posture. Existing technologies still have significant room for improvement in processing accuracy, efficiency, and the breadth of application scenarios.

[0004] Therefore, developing a more automated and precise image processing technology that can transform portraits into ID photos that meet standard requirements and automatically adjust the subject's posture so that their eyes are level with their shoulders has become a pressing technical challenge. This paper addresses this need and proposes a technical solution that uses an intelligent algorithm to process images, automatically correct a subject's posture, and generate ID photos that meet standard requirements. Summary of the Invention

[0005] The present invention is proposed to alleviate or solve at least one aspect or at least one point of the above problems. The present invention provides a method for automatically processing a portrait photo into a standard ID photo, comprising the following steps:

[0006] Step S1: Input a portrait photo and use a pre-trained facial key point detection model to detect facial key points and locate the coordinates of the eyes;

[0007] Step S2: Calculate the required rotation direction and angle based on the coordinates of the eyes to ensure horizontal alignment of the eyes; Step S3: Use the cutout model to perform precision cutout to generate a transparent background portrait image, rotate the transparent background portrait image to obtain a transparent background portrait image with the eyes aligned;

[0008] Step S4: cropping the transparent background portrait image according to the rotation result, and synthesizing the transparent background with the standard background to produce a standard ID photo;

[0009] Step S5: Using a pre-trained human shoulder key point detection model, detect the key points of the left and right shoulders of the standard ID photo;

[0010] Step S6: Calculate the difference between the average shoulder height and the preset reference height based on the coordinates of the left and right shoulder key points to determine the shoulder that needs to be adjusted;

[0011] Step S7: Using a local deformation algorithm, adjust the shoulder key points that need to be adjusted so that their heights are aligned with the reference values, and obtain a result image in which the eyes and shoulders of the original image are flush.

[0012] Preferably, the facial key point detection model is a model in computer vision that can identify the structural features of the face in the image and accurately mark multiple key point features, wherein the key point features include eyes, nose, ears, corners of the mouth, jaw, shoulder and neck area below the ears, and key shoulder areas;

[0013] Preferably, the face landmark detection model uses hierarchical feature extraction and feature pyramid structure to ensure that objects of various sizes are detected, and non-maximum suppression is used to refine these predictions to filter out duplicate or low-confidence boxes, thereby achieving more accurate object detection.

[0014] Preferably, the facial key point detection model adopts an improved backbone network and neck architecture, the backbone network adopts a faster and more efficient variant of the cross-stage partial bottleneck structure, and the improved backbone network adds a layer of cross-stage local spatial attention module after the spatial pyramid pooling fast module.

[0015] Preferably, the facial key point detection includes the following steps:

[0016] Collect and preprocess portrait data;

[0017] The facial landmark detection model passes the input image into a convolutional neural network to extract features and perform object detection;

[0018] Self-training of facial key point detection model;

[0019] The facial landmark detection model automatically identifies faces in images and labels their landmarks.

[0020] Preferably, the step of calculating the required rotation direction and angle according to the coordinates of the eyes includes the following steps:

[0021] Analyze the coordinates of both eyes in the input portrait photo and calculate the horizontal axis distance and rotation angle;

[0022] Based on the relative relationship between the distance difference between the two eyes on the horizontal axis and the image width, the difference is used to determine the degree of horizontal offset of the two eyes and further calculate the rotation angle.

[0023] Preferably, the cutout model is used to accurately separate the foreground from the background to generate a high-quality transparent background portrait image. The cutout model includes a shared encoder module, a pyramid pooling module, a semantic context branch module, and a high-resolution detail branch module. The backbone network of the cutout model adopts a high-resolution network, which is also used as a shared encoder. The output of the last stage of the backbone network is sent to the pyramid pooling module to obtain richer semantic context information.

[0024] Preferably, the semantic context branch module consists of five blocks, each block includes convolution, batch normalization, activation function and a bilinear upsampling module. The output of the semantic context branch is also used to supervise the semantic segmentation task, which includes three categories: foreground, background and transition area.

[0025] Preferably, the input of the high-resolution detail branch is formed by concatenating the intermediate features of the shared encoder after an upsampling operation.

[0026] Preferably, the intermediate features extracted by the shared encoder at different levels are first restored to the same resolution through an upsampling operation, and then these feature maps are spliced ​​in the channel dimension to form the initial input of the high-resolution detail branch.

[0027] Preferably, the semantic context feature map of the semantic context branch module and the detail feature map of the high-resolution detail branch module are subjected to guided flow processing, wherein the guided flow processing is to splice the semantic context feature map from the semantic context branch and the detail feature map of the high-resolution detail branch, and send them into a process including convolution, batch normalization, rectified linear unit activation function and convolution, batch normalization, S-shaped activation function to generate a guided map, and then perform dot multiplication and addition operations on the guided map and the detail feature map of the high-resolution detail branch to generate a feature map of the next stage of high-resolution detail.

[0028] The present invention provides a method for automatically processing a portrait photo into a standard ID photo, which is an image processing technology with a higher degree of automation and better precision. It can automatically process a portrait photo into an ID photo that meets standard requirements and can automatically adjust the person's posture so that the person's eyes are level with the shoulders, eliminating the tedious traditional manual processing, improving processing efficiency, saving labor, and achieving higher processing precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flowchart for ID photos with portrait photo processing as the standard.

[0030] Figure 2 This is the backbone network structure diagram of the face key point detection model.

[0031] Figure 3 This is a diagram of the fast module structure of spatial pyramid pooling.

[0032] Figure 4 Figure 2 is the structural diagram of the cross-stage local spatial attention module.

[0033] Figure 5 This is the network architecture of the cutout model.

[0034] Figure 6 To guide the flow structure. DETAILED DESCRIPTION

[0035] The following description of the embodiments of the present invention with reference to the accompanying drawings is intended to explain the overall inventive concept of the present invention and should not be understood as limiting the present invention. In the present invention, the same reference numerals represent the same or similar components.

[0036] The features described herein can be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided to illustrate only some of the many possible ways to implement the methods, devices, and / or systems described herein, which will become clear after understanding the disclosure of the present invention.

[0037] The terms used herein are only used to describe various examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular is intended to include the plural. The terms "comprise," "include," and "have" indicate the presence of the recited features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0038] In order to enable those skilled in the art to use the contents of the present invention, the following exemplary embodiments may be provided in conjunction with specific application scenarios, specific system, device and component parameters, and specific connection methods. However, for those skilled in the art, these embodiments are only examples, and the general principles defined herein can be applied to other embodiments and application scenarios without departing from the spirit and scope of the present invention.

[0039] According to an exemplary embodiment of the present invention: This example provides a method for automatically processing a portrait photo into a standard ID photo, such as Figure 1 As shown, the following steps are included:

[0040] Step S1: Input a portrait photo and use a pre-trained facial landmark detection model to detect facial landmarks and locate the coordinates of the eyes. A facial landmark detection model is a computer vision model that identifies the structural features of a face in an image and accurately labels the coordinates of multiple landmarks. These landmarks are typically fixed and include the eyes, nose, ears, corners of the mouth, jaw, the neck below the ears, and key areas of the shoulders.

[0041] Step S11: Before training the facial landmark detection model, portrait photos must be collected and preprocessed. First, use annotation tools to annotate key features in the portrait image obtained from the photograph. These key features include the eyes, nose, ears, corners of the mouth, jaw, the neck area below the ears, and shoulders. Each annotated point must be assigned a unique label to ensure the accuracy and standardization of the labeled information. The label for each landmark not only helps distinguish different features but also ensures that the model accurately learns the spatial relationships and feature information of these features during training. The annotated image data, along with the corresponding label information, must be converted into a text format suitable for model training. This allows the model to use this labeled information to automatically identify and infer the locations of key features in the image during the learning process. After organizing the annotated data, all portrait images and their corresponding label information must be compiled into a complete training dataset. During dataset construction, ensuring dataset diversity is crucial to prevent model overfitting and improve generalization. To comprehensively evaluate model performance, datasets are typically divided into training and validation sets. The training set contains most of the sample data and is used for the model training process, while the validation set contains unseen samples and is used to regularly evaluate the performance of the model on new data to ensure that the model does not simply memorize the training data during the training process and lose its predictive ability for new samples.

[0042] Step S12: The face keypoint detection model performs object detection by passing the input image into a convolutional neural network to extract features. Specifically, the network predicts the bounding boxes and class probabilities of objects within these grids. In order to handle multi-scale detection, Figure 2As shown, the facial landmark detection model employs hierarchical feature extraction and a feature pyramid structure to ensure detection of objects of various sizes. Non-maximum suppression is then used to refine these predictions to filter out duplicate or low-confidence boxes, resulting in more accurate object detection. The facial landmark detection model utilizes an improved backbone and neck architecture, significantly enhancing feature extraction capabilities, thereby improving object detection accuracy and multi-task performance. The backbone network utilizes a faster and more efficient variant of the cross-stage partial bottleneck architecture. This architecture utilizes two convolutional layers with smaller convolution kernels to accelerate computation while maintaining performance. To further improve efficiency, the backbone network is improved by adding a cross-stage local spatial attention module after the spatial pyramid pooling fast module. This reduces redundant computation through a depthwise separable approach, improving efficiency. The neck is responsible for aggregating features at different resolutions and passing them to the head for prediction. This section typically involves upsampling and concatenating feature maps at different levels to ensure the model learns both global and local details from information at different scales. The feature pyramid structure upsamples the deep high semantic features to the same resolution as the shallow features and fuses the upsampled deep features with the shallow features.

[0043] The cross-stage partial bottleneck structure is a network optimization method that aims to improve the computational efficiency of neural networks. By splitting the network's computational process into multiple stages and partially connecting these stages, this structure can more efficiently transmit information, maintain high performance while reducing unnecessary calculations. The spatial pyramid pooling fast module is used to perform maximum pooling operations at different scales and connect the pooling results to capture features at multiple spatial scales, enhancing the model's adaptability to features of various scales. Its structure is shown in the figure below. Figure 3 The cross-stage local spatial attention module enhances the spatial attention in the feature map, helping the model to pay more attention to important areas in the image. This module enables the model to effectively focus on specific areas of interest by spatially pooling features, thereby improving the accuracy of key point detection. Its structure is shown in the figure below. Figure 4 shown.

[0044] Step S13: Improving performance and efficiency during facial landmark detection model training. First, by performing various data augmentation operations on the input images, such as rotation, scaling, cropping, and flipping, the diversity of the training data is increased. This not only expands the coverage of the dataset but also effectively improves the model's generalization capability, reduces the risk of overfitting, and enables the trained model to better adapt to different input images. Second, the present invention employs a multi-scale training strategy, using images of varying scales for training. This strategy enables the model to learn features from small to large objects, thereby enhancing the model's multi-scale detection capabilities in practical applications. This strategy is particularly effective when processing facial images of varying sizes and resolutions, maintaining good detection accuracy. Furthermore, to enhance the model's ability to learn difficult samples, the present invention introduces a hard negative mining strategy. This hard negative mining strategy is a commonly used optimization method in object detection or landmark detection tasks. It aims to dynamically adjust the weights of training samples to enable the model to better learn samples that are difficult to classify or detect. Dynamically adjusting sample weights during training allows the model to pay more attention to difficult samples such as complex backgrounds, occluded objects, blurred or small objects. Through this strategy, the model is able to perform more stably and accurately when dealing with challenges in real-world scenarios, further improving the accuracy of facial landmark detection in various environments.

[0045] Step S14: Input the portrait photo into a trained facial key point detection model. After training, the facial key point detection model can automatically identify faces in images and annotate their key points. After processing the input image, the model returns a result containing the coordinates of multiple facial key points. These coordinates include the eyes, nose, ears, corners of the mouth, jaw, the neck area below the ears, and key areas of the shoulders. From all annotated key point coordinates, the coordinates of key points related to both eyes are further filtered out. By extracting the coordinate information of both eyes from the set of detected key points, accurate data support is provided for subsequent image processing and optimization steps.

[0046] Step S2: Calculate the required rotation direction and angle based on the coordinates of the two eyes to ensure that the two eyes are aligned in the horizontal direction. Step S21: The present invention implements the calculation of the horizontal axis distance and the calculation of the rotation angle by analyzing the coordinates of the two eyes in the input portrait photo. The specific steps are as follows: First, based on the face key point detection model, obtain the coordinates of the two eyes in the input portrait photo, set the left eye coordinates to (x1, y1), and the right eye coordinates to (x2, y2). Calculate the distance difference between the two eyes on the horizontal axis to obtain the position difference of the two eyes in the horizontal direction. Formula (1) represents its calculation process: Δx=|x2-x1| (1)

[0047] Δx represents the distance difference; x2 represents the horizontal coordinate of the right eye; x1 represents the horizontal coordinate of the left eye;

[0048] Step S22: Based on the relative relationship between the horizontal distance difference between the two eyes and the image width, the horizontal offset between the two eyes is determined using this difference, and the rotation angle is further calculated. The ratio of the vertical deviation to the horizontal deviation between the two eye coordinates is calculated using the inverse tangent function to obtain the rotation angle, which determines the required alignment angle for the two eyes. The direction of rotation is then determined based on the sign of the rotation angle. If the angle is positive, the right eye is tilted upward, and the rotation direction is clockwise; if the angle is negative, the right eye is tilted downward, and the rotation direction is counterclockwise.

[0049] Step S3: Use the cutout model to perform precise cutout to generate a transparent background portrait image, rotate the transparent background portrait image to obtain a transparent background portrait image aligned with the human eye. The cutout model is mainly used to accurately separate the foreground from the background to generate a high-quality transparent background portrait image. Figure 5As shown, the cutout model of the present invention includes a shared encoder module, a pyramid pooling module, a semantic context branch module, and a high-resolution detail branch module. The backbone network of the cutout model of the present invention adopts a high-resolution network, which is also used as a shared encoder. For example, it is composed of five blocks, which are 1 / 2 times downsampling, 1 / 4 times downsampling, 1 / 8 times downsampling, 1 / 16 times downsampling, and 1 / 32 times downsampling. At the output of the last stage of the backbone network, the feature map after 1 / 32 times downsampling is sent to the pyramid pooling module to obtain richer semantic context information; the feature map after 1 / 4 times downsampling is sent to the upsampling module to restore to the same resolution, and then these feature maps are spliced ​​in the channel dimension to form the initial input of the high-resolution detail branch. A branch design is adopted to clarify the semantic prediction and detail prediction tasks. In the semantic context branch, the feature map obtained by the pyramid pooling module is used as the input of the semantic context. Exemplarily, the semantic context branch consists of five blocks, each of which includes convolution, batch normalization, activation function and a bilinear upsampling module, which realizes 1 / 16 resolution, 1 / 8 resolution, 1 / 4 resolution, 1 / 2 resolution and 1 / 1 resolution respectively. In addition, the output of the semantic context branch is also used to supervise the semantic segmentation task, which includes three categories: foreground, background and transition area. The input of the high-resolution detail branch is spliced ​​by the intermediate features of the shared encoder after upsampling operation. Specifically, the intermediate features extracted by the shared encoder at different levels are first restored to the same resolution through upsampling operation, and then these feature maps are spliced ​​in the channel dimension to form the initial input of the high-resolution detail branch. This design can retain more detail information, thereby improving the ability of the cutout model to accurately separate foreground edges and complex areas. At the same time, a guidance flow is introduced from the intermediate features of 1 / 16 and 1 / 4 resolution of the semantic context branch to inject semantic information to help high-resolution details obtain more detailed image details. The guidance flow, whose structure is as follows Figure 6 As shown. After the semantic context feature map from the semantic context branch and the detail feature map of the high-resolution detail branch are spliced, they are sent to a convolution, batch normalization, rectified linear unit activation function and convolution, batch normalization, S-shaped activation function to generate a guidance map. Then, the guidance map is multiplied and added with the detail feature map of the high-resolution detail branch to generate the feature map of the next stage of high-resolution detail. The total cross entropy loss function is shown in formula (2), the total loss function is shown in formula (3), and the overall loss function is shown in formula (4):

[0050] Ω=Ω f ∪Ω b ∪Ω t

[0051] Among them L srepresents the total cross entropy loss; c represents the category index; i represents the index of the pixel; Ω represents the location set of all pixels in the image; Represents the true label value of the i-th pixel in category c; represents the predicted probability of the i-th pixel in category c; express The natural logarithm of Ω f Indicates belonging to the foreground category; Ω b Indicates belonging to the background category; Ω t represents the set of pixel locations belonging to the third category;

[0052]

[0053] L d represents the total loss function; Ω t Represents the pixel set of the target area; i pixel index; Indicates the transparency loss of the i-th pixel; represents the gradient loss of the i-th pixel; represents the predicted transparency value of the i-th pixel, generated by the model; The true transparency value of the i-th pixel; ε represents a small positive value to avoid the square difference being zero; represents the gradient of the predicted transparency value of the i-th pixel; Represents the gradient of the true transparency value of the i-th pixel;

[0054]

[0055] L f represents the overall loss function; i represents the pixel index; Ω represents the image pixel set; Indicates the transparency loss of the i-th pixel; represents the gradient loss of the i-th pixel; represents the composite loss of the i-th pixel; represents the predicted composite image value of the i-th pixel; represents the true composite image value of the i-th pixel; ε represents a small positive value to avoid the square difference being zero; represents the predicted transparency value of the i-th pixel; represents the true foreground pixel value of the i-th pixel; Represents the true background pixel value of the i-th pixel; Represents the probability that a pixel belongs to the background.

[0056] The high-resolution network structure consists of multiple stages, each of which is processed by network branches of different resolutions. At each stage, the network gradually optimizes and fuses features of different scales through cross-branch information exchange, thereby enhancing the expressive power of features. As the network depth increases, it is able to better capture detailed information and global semantics in the image. The high-resolution network maintains high-resolution features at each stage, while also generating and fusing low-resolution feature maps to obtain richer semantic information. An effective multi-scale feature fusion strategy is adopted to interactively fuse feature maps of different resolutions. Information is exchanged between low-resolution and high-resolution feature maps, so that each branch can not only learn lower-level detailed information, but also incorporate global semantic information from other scales.

[0057] A shared encoder refers to a design in which multiple tasks or multiple modules share the same part of the encoding network structure in a high-precision natural image matting model. When upstream and downstream tasks or multiple branches share the same encoder, the encoder's network layers are shared by multiple tasks, thereby reducing the use of computing resources and parameters, and helping to share information between tasks, improving learning efficiency and generalization ability.

[0058] The pyramid pooling block module uses a pyramid pooling operation to pool feature maps at different scales. The pooled features are then concatenated or fused to obtain multi-scale semantic information. This helps the model better understand and segment objects of different scales in the image. The pyramid pooling operation effectively expands the receptive field of the neural network, enabling the model to better capture global and local information in the image. The pyramid pooling block module can be applied to the decoder's output feature map, providing more context for the segmentation task. This helps the model more accurately classify pixels into different semantic categories, thereby improving segmentation accuracy and generalization.

[0059] Step S31: The present invention provides a technology based on a deep learning cutout model for generating a portrait image with a transparent background.

[0060] A portrait image with a transparent background is a portrait image with a transparent background that includes a transparency channel. This method performs precise image segmentation on the input portrait photo, extracts the subject area, and generates an image with a transparency channel, where the transparency channel is used to represent the area separating the subject from the background, thereby obtaining a portrait image with a transparent background. Specifically, the cutout model analyzes the subject's features in the input image, accurately separates the subject's outline, and effectively removes the complex background. The resulting transparent background portrait image not only clearly displays the subject's image but also preserves details such as hair and clothing edges, ensuring high-quality visual effects and enabling easy synthesis with other backgrounds.

[0061] Step S4: Crop the transparent background portrait image according to the rotation result, and synthesize the transparent background with the standard background to process it into a standard ID photo.

[0062] Step S41: Based on the calculated eye coordinates and rotation angle, the generated transparent background portrait image is first rotated to ensure horizontal alignment of the eyes within the image. This rotation process, based on the coordinate difference between the eyes and the pre-calculated angle, accurately rotates the image to the ideal position, achieving the subject's posture required for a standard ID photo. Next, the transparent background portrait image is appropriately cropped based on the rotation angle and the image's aspect ratio. This cropping process ensures that the image maintains proper proportions, centered around the subject, and conforms to the size and composition requirements of an ID photo. After cropping, the transparent background portrait image is scaled and composited with a preset standard background. This background can be customized to meet specific requirements, including common standard ID photo backgrounds such as blue and white. Finally, through background scaling and composition, a complete image conforming to the standard ID photo format is generated. This step not only ensures the uniformity and standardization of the image background, but also optimizes the presentation of the subject's posture, ensuring that the generated ID photo meets all specifications and meets practical application needs.

[0063] Step S5: Use a pre-trained human shoulder keypoint detection model to detect the keypoints of the left and right shoulders of the standard ID photo. This model shares the same architecture as the facial keypoint detection model used in step S1, but uses different training data, specifically annotated and trained for the shoulders. Based on computer vision technology, the shoulder keypoint detection model extracts structural features from the input image and accurately locates keypoints of the shoulders. These keypoints, including specific areas of the left and right shoulders, can be used to further correct image pose or calculate shoulder symmetry, providing a foundation for the standardized processing of standard ID photos.

[0064] Step S51: training the human shoulder key point detection model. First, the key parts of the portrait image are annotated by using the annotation tool. There are 12 key points on the left and right sides of the face. Each annotated point must be assigned a unique label to ensure the accuracy and standardization of the labeled information. The annotated image data needs to be converted into a text format that can be used for model training together with the corresponding label information. After the annotated data is sorted, all portraits and their corresponding label information need to be organized into a complete training data set. In the process of constructing the data set, in order to avoid overfitting of the model and improve its generalization ability, the diversity of the data set must be ensured. In order to comprehensively evaluate the performance of the model, the data set is usually divided into a training set and a validation set. The training set contains most of the sample data and is used for the training process of the model, while the validation set contains unseen samples and is used to regularly evaluate the performance of the model on new data to ensure that the model does not only remember the training data during the training process and lose its predictive ability for new samples.

[0065] Step S52: Input the generated ID photo into the human shoulder key point detection model. After sufficient training, the human shoulder key point detection model can automatically identify the human body in the image and mark its key points. By processing the input image, the model outputs a result containing the coordinates of multiple human key points, with 12 key points marked on each side of the face. Among all the marked key points, the key point coordinates of the left and right shoulders are further extracted and filtered as basic data for subsequent processing. Step S6: Based on the coordinates of the left and right shoulder key points, the difference between the average shoulder height and the preset reference height is calculated to determine the shoulder that needs adjustment.

[0066] Step S61: Based on the coordinates of the acquired left and right shoulder key points, the average heights of the left and right shoulders are first calculated. Next, a preset value is set as a reference height for the shoulders. The average heights of the left and right shoulders are compared with the reference height, and the height difference between each shoulder and the reference height is calculated and the absolute value is taken. By comparing the height differences between the two sides, the side with the larger difference is determined as the "adjustment side" requiring adjustment, and the series of key points on that side are defined as the "pre-adjustment key points."

[0067] Step S62: Based on the relative positional relationships between the pre-adjustment keypoints and the keypoints on the other shoulder, the coordinates of the adjusted keypoints are calculated one by one. Specifically, the horizontal coordinates of the adjusted keypoints remain consistent with those of the pre-adjustment keypoints to ensure lateral positional accuracy; while the vertical coordinates of the adjusted keypoints are adjusted to align with those of the corresponding keypoints on the other shoulder to achieve vertical alignment. This adjustment process aims to maintain symmetry between the shoulders and their associated keypoints, while optimizing the accuracy and balance of the overall structure, ensuring the accuracy and consistency of the model in detecting and labeling keypoints on the human body.

[0068] Step S7: Using a local deformation algorithm, the shoulder keypoints that require adjustment are adjusted so that their heights align with the reference value, resulting in an image with both eyes and shoulders aligned in the original image. Local deformation algorithms are a technique commonly used in image processing and computer graphics. By deforming and adjusting specific areas within an image, they can achieve various functions, such as geometric deformation, detail enhancement, and defect repair. In the present invention, the local deformation algorithm is used to adjust the position of the shoulder keypoints to align them with a set reference height, thereby achieving image standardization.

[0069] Step S71: First, based on the coordinates of the shoulder key points before adjustment and the target key points after adjustment, the shoulder area in the image is selected as the area to be adjusted, and the shoulder key points are defined as control points. Control points are used to determine the benchmark and range of image deformation and are crucial parameters in the deformation process. Then, based on the position and distribution of the control points, a deformation grid covering the entire image is constructed. Each unit of this grid is determined by the control points, and its structure and distribution directly affect the smoothness and accuracy of the deformation effect. By analyzing the relationship between the control points and the grid nodes, the offset between the control points and the target adjusted position is calculated, and the deformation parameters that need to be applied during the deformation process are determined.

[0070] Step S72: Based on the above deformation parameters, the position of each pixel in the image after deformation is calculated one by one using the grid interpolation method. In order to ensure the visual continuity and accuracy of the image after deformation, it is necessary to perform color interpolation calculation on the pixels in the new position. The image that has completed the deformation calculation is reconstructed as a whole to generate an adjusted image. In the adjusted image, the relative positions of the eyes and shoulders are horizontally aligned, ensuring the symmetry of the image and the standardization of the human posture, meeting the production requirements of standard ID photos. The local deformation algorithm can not only accurately adjust the position of the key points of the shoulders, but also ensure the overall quality and detail consistency of the image, and realize efficient adjustment of the shoulder area and optimization of the visual effect. Interpolation calculation is a technology widely used in mathematics, image processing, data analysis and other fields. Its core purpose is to estimate the value of unknown points between known data points, so as to construct a more continuous and smooth result.

[0071] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes may be made to the embodiments and combinations of elements may be made without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for automatically processing a portrait photo into a standard ID photo, characterized by: The following steps are included: Step S1: Input a portrait photo and use a pre-trained facial key point detection model to detect facial key points and locate the coordinates of the eyes; Step S2: Calculate the required rotation direction and angle based on the coordinates of both eyes to ensure that both eyes are aligned in the horizontal direction; Step S3: Use the cutout model to perform precision cutout to generate a transparent background portrait image, rotate the transparent background portrait image to obtain a transparent background portrait image aligned with the human eye, The cutout model is used to accurately separate the foreground from the background and generate high-quality transparent background portraits. The cutout model includes a shared encoder module, a pyramid pooling module, a semantic context branch module, and a high-resolution detail branch module. The backbone network of the cutout model uses a high-resolution network, which also serves as a shared encoder. The output of the last stage of the backbone network is fed into the pyramid pooling module to obtain richer semantic context information. The semantic context branch module consists of five blocks, each of which includes convolution, batch normalization, activation function and a bilinear upsampling module. The output of the semantic context branch is also used to supervise the semantic segmentation task, which includes three categories: foreground, background and transition area. The input of the high-resolution detail branch is formed by concatenating the intermediate features of the shared encoder after upsampling. The intermediate features extracted by the shared encoder at different levels are first restored to the same resolution through upsampling operations, and then these feature maps are spliced ​​in the channel dimension to form the initial input of the high-resolution detail branch. The semantic context feature map of the semantic context branch module and the detail feature map of the high-resolution detail branch module are subjected to guided flow processing. The guided flow processing is to splice the semantic context feature map from the semantic context branch and the detail feature map of the high-resolution detail branch, and then send them into a convolution, batch normalization, rectified linear unit activation function and convolution, batch normalization, S-shaped activation function to generate a guided map. Then, the guided map is multiplied and added with the detail feature map of the high-resolution detail branch to generate a feature map of the next stage of high-resolution detail. The guidance flow is introduced into the intermediate features of the 1 / 16 and 1 / 4 resolutions of the semantic context branch. The guidance flow concatenates the semantic context feature map from the semantic context branch and the detail feature map of the high-resolution detail branch, and then feeds it into a convolution, batch normalization, rectified linear unit activation function and convolution, batch normalization, S-shaped activation function to generate a guidance map. Perform dot multiplication and addition operations on the guide map and the detail feature map of the high-resolution detail branch to generate the feature map of the next stage of high-resolution detail. The total cross entropy loss function is shown in formula (2): (2) in represents the total cross entropy loss; c represents the category index; i represents the pixel index; Represents the set of locations of all pixels in the image; Represents the true label value of the i-th pixel in category c; represents the predicted probability of the i-th pixel in category c; express The natural logarithm of Indicates belonging to the foreground category; Indicates belonging to the background category; represents the set of pixel locations belonging to the third category; Step S4: cropping the transparent background portrait image according to the rotation result, and synthesizing the transparent background with the standard background to produce a standard ID photo; Step S5: Use the pre-trained human shoulder key point detection model to detect the key points of the left and right shoulders of the standard ID photo. Use annotation tools to mark key areas in the portrait image. There are 12 key points on each side of the face. Each annotation point must be assigned a unique label to ensure the accuracy and standardization of the annotation information. The annotated image data and the corresponding label information must be converted into a text format that can be used for model training. After the annotated data is organized, all portrait images and their corresponding label information need to be organized into a complete training dataset. Step S6: Calculate the difference between the average shoulder height and the preset reference height based on the coordinates of the left and right shoulder key points to determine the shoulder that needs to be adjusted. Based on the coordinates of the key points of the left and right shoulders, we first calculate the average height of the left and right shoulders respectively. Then, we set a preset value as the reference height of the shoulders and compare the average height of the left and right shoulders with the reference height respectively. We calculate the height difference between each shoulder and the reference height and take its absolute value. By comparing the height difference between the two sides, we determine the side with the larger difference as the "adjustment side" that needs to be adjusted, and define the series of key points on this side as the "key points before adjustment". Based on the relative positional relationship between the key points before adjustment and the key points on the other shoulder, the coordinates of the adjusted key points are calculated one by one. Specifically, the horizontal coordinates of the adjusted key points remain consistent with those of the key points before adjustment to ensure the accuracy of the horizontal position; while the vertical coordinates of the adjusted key points are adjusted to be consistent with the vertical coordinates of the corresponding key points on the other side to achieve vertical alignment. This adjustment process aims to ensure the symmetry of the two shoulders and their related key points, while optimizing the accuracy and balance of the overall structure, ensuring the accuracy and consistency of the model in the detection and labeling tasks of human body key points; Step S7: Using the local deformation algorithm, adjust the shoulder key points that need to be adjusted so that their heights are aligned with the reference values, and obtain the result image of the original image where the eyes and shoulders are flush. According to the coordinates of the shoulder key points before adjustment and the target key points after adjustment, the shoulder area in the image is selected as the area to be adjusted, and the shoulder key points are defined as control points. According to the position and distribution of the control points, a deformation grid covering the entire image is constructed. By analyzing the relationship between the control points and the grid nodes, the offset between the control points and the target adjusted position is calculated, and the deformation parameters that need to be applied during the deformation process are determined. Based on the above deformation parameters, the grid interpolation method is used to calculate the position of each pixel in the image after deformation. The image after the deformation calculation is reconstructed as a whole to generate an adjusted image. In the adjusted image, the relative positions of the eyes and shoulders are horizontally aligned.

2. The method for automatically processing a portrait photo into a standard ID photo according to claim 1, characterized in that: The facial key point detection model is a model in computer vision that can identify the structural features of the face in an image and accurately mark multiple key point features, including the eyes, nose, ears, corners of the mouth, jaw, shoulder and neck areas below the ears, and key shoulder areas; The facial landmark detection model uses hierarchical feature extraction and a feature pyramid structure to ensure that objects of various sizes are detected. Non-maximum suppression is used to refine these predictions to filter out duplicate or low-confidence boxes, thereby achieving more accurate object detection.

3. The method for automatically processing a portrait photo into a standard ID photo according to claim 1, characterized in that: The face key point detection model adopts an improved backbone network and neck architecture. The backbone network adopts a faster and more efficient variant of the cross-stage partial bottleneck structure. The improved backbone network adds a cross-stage local spatial attention module after the spatial pyramid pooling fast module.

4. The method for automatically processing a portrait photo into a standard ID photo according to claim 1, characterized in that: The facial key point detection comprises the following steps: Collect and preprocess portrait data; The facial landmark detection model passes the input image into a convolutional neural network to extract features and perform object detection; Self-training of facial key point detection model; The facial landmark detection model automatically identifies faces in images and labels their landmarks.

5. The method for automatically processing a portrait photo into a standard ID photo according to claim 1, characterized in that: The step of calculating the required rotation direction and angle according to the coordinates of the eyes includes the following steps: Analyze the coordinates of both eyes in the input portrait photo and calculate the horizontal axis distance and rotation angle; Based on the relative relationship between the distance difference between the two eyes on the horizontal axis and the image width, the difference is used to determine the degree of horizontal offset of the two eyes and further calculate the rotation angle.

Images