Image processing system and method for processing an image
By employing a lightweight skip fusion connection network structure and hierarchical hybrid loss, combined with a data augmentation module, the challenge of human portrait segmentation on low-cost devices is addressed, achieving efficient and robust image segmentation and alignment while reducing computational complexity and cost.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BLACK SESAME TECH CO LTD
- Filing Date
- 2022-07-19
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies struggle to efficiently and accurately segment and align human portrait regions in images on low-cost devices, especially when dealing with challenging human body components. Furthermore, traditional methods are unreliable to variations in lighting, body shape distortion, and motion blur.
A lightweight skip fusion connection network structure is adopted, which combines hierarchical hybrid loss and human-centered data augmentation module. Features are extracted through encoder-decoder unit and refined using binary cross-entropy, gradient loss and local binary cross-entropy loss. Pre-alignment is performed in combination with data augmentation module.
It enables efficient and robust segmentation and alignment of human portrait regions on low-cost devices, improves stability to lighting changes and body shape deformation, and reduces computational complexity and cost.
Smart Images
Figure CN115375703B_ABST
Abstract
Description
Technical Field
[0001] This invention generally relates to systems and methods for processing portrait regions of images. More specifically, this invention aims to segment human portrait regions of images. Background Technology
[0002] Human portrait segmentation technology is applied in fields such as film, online education, and entertainment. Traditionally, this task was accomplished using green screen technology, which required expensive equipment and specific environments. Most segmentation methods rely on the color difference between the foreground and background to achieve background removal. When the color difference is not significant, the results of these methods are less consistent.
[0003] A granted U.S. patent, 9,807,316, assigned to HTC, discloses a technique for extracting objects from an image captured by a first camera based on an adaptive model mask. Furthermore, the patent discloses an adaptive model mask based on pose data (POS) and the relative motion between the camera and the foreground object. While this process provides three layers of masks for segmentation, it still fails to achieve accurate segmentation of the entire portrait region and center alignment of the portrait region within the image.
[0004] Another patent application, WO2009109127, assigned to the Chinese University of Hong Kong, discloses a technology relating to a human feature recognition system. This system includes a face detection module, a human body segmentation module, and a boundary matting module, wherein the face detection module uses an active lifting procedure and a lazy lifting procedure in a hybrid cascaded structure. Although this invention is an improvement on a prior patent because it includes a semantic object extraction technique applicable to digital video processing, pattern recognition, and computer vision, it fails to achieve cost-effective and accurate portrait segmentation from images.
[0005] Another patent application, CN110689542, transferred to Tsinghua University, provides a human image segmentation processing technology based on a multi-level convolutional neural network. Although this invention is an improvement on previous inventions to accurately and efficiently segment portraits in real time, it still lacks the ability to segment portraits using simple tools, instead relying on high-tech and expensive tools.
[0006] Therefore, to overcome the shortcomings of existing technologies, such as when dealing with challenging human body components (e.g., hair and hands), a hierarchical hybrid loss is needed instead of the traditional segmentation loss. Finally, to customize human portrait segmentation applications and reduce the dimensionality of the learning space, a unique data augmentation strategy is innovatively developed, resulting in a more uniform distribution of training data, more stable performance, and faster convergence. In view of the foregoing invention, there is a need in the art for a system to overcome or mitigate the shortcomings of the aforementioned existing technologies.
[0007] It is evident that numerous methods and systems suitable for various purposes have been developed in the prior art. Furthermore, even if these inventions may therefore be suitable for their specific purposes, they are not suitable for the purposes of the present invention as described above. Therefore, there is a need for an advanced image processing system to identify and segment human portraits in images, and then output aligned portraits from the images. Summary of the Invention
[0008] This invention utilizes low-cost neural networks to predict object segmentation. It includes a skip-fusion connection network structure that uses fewer hidden layer parameters and significantly reduces computational costs.
[0009] The use of emerging and rapidly developing smart mobile devices has enabled people to experience a more convenient and intelligent lifestyle within the reach of a single click. Unlike traditional bulky PCs or even laptops, smartphones, with their superior portability and computing power, have become indispensable assistants in our daily lives. People use smartphones for business meetings, ordering food, and buying movie tickets.
[0010] Beyond basic and passive interactions, people now expect deeper communication with devices, necessitating more intelligent equipment. Thanks to powerful deep learning technology, intelligent devices are better able to understand the world. Since such devices will frequently interact with human users, understanding human attributes or behaviors is crucial.
[0011] Semantic segmentation, as an essential basic function, can provide the location of objects of interest in an image with pixel-level precision through machine semantic-level intelligence. It paves the way for further processing and improvements in commercial applications such as photographic enhancement, AR, and VR. However, providing high-quality and robust semantic segmentation masks on edge computing chips while reducing computational complexity is extremely challenging.
[0012] Before the rise of deep neural networks, researchers aiming to segment human portraits had to manually extract features such as skin color, body shape, and facial features, and then feed these into pre-designed classifiers (like Adaboost). These methods typically suffer from poor generalization. They are vulnerable to external noise, such as changes in lighting, body shape distortion, and even motion blur.
[0013] The emergence of deep learning networks has alleviated these problems because they learn from a large number of examples through their powerful regression capabilities. Robustness has been greatly improved, but it is still insufficient for industrial applications, especially in embedded or mobile devices.
[0014] To ensure accuracy in this pixel-level classification problem, most researchers use very heavy networks such as FCNs, which imposes a significant computational cost on devices. Few innovative designs have yielded efficient yet hardware-friendly networks that balance the trade-off between performance and complexity.
[0015] Segmenting details requires additional effort, which often leads to more complex networks, such as head matting. Applying a loss penalty during training is one way to address this problem.
[0016] Finally, for the training set, data augmentation is widely used to improve sample diversity. However, as the amount of data increases, the distribution of human positions becomes less uniform. Few works address this issue because high-quality human segmentation datasets are scarce, and they tend to use highly complex networks to accommodate low-quality annotations. If the goal is to reduce the number of neural layers in the network, the dimensionality of the target problem to be learned should also be reduced accordingly.
[0017] The primary objective of this invention is to provide a novel human portrait segmentation system that offers deployable complexity and superior performance. This system is an image processing system for processing images, comprising a classification module consisting of an encoder-decoder unit and skip fusion connection logic.
[0018] The encoder-decoder unit comprises an encoder and a decoder. The encoder extracts and classifies multiple distinguishing features and multiple similarity features from the portrait region. The multiple similarity features are classified based on one or more predefined levels of semantic categories. The decoder refines the one or more similar features to generate multiple refined features. Skip-fusion connection logic concatenates these multiple refined features to generate a refined portrait region of the image.
[0019] The hierarchical hybrid loss module includes a binary cross-entropy (BCE) loss unit, a gradient loss unit, and a local binary cross-entropy (BCE) loss unit. The BCE loss unit is used to adjust the first mask applied to the first portion of the refined portrait region. The gradient loss unit is used to apply a second mask to the refined portrait region, and the local binary cross-entropy loss unit is used to adjust the third mask applied to the second portion of the refined portrait region to form the masked portrait region.
[0020] The human-centric data augmentation module is used to pre-align the masked portrait region with the central region and generate the output image.
[0021] Another object of the present invention is to provide an encoder-decoder architecture with light-skip connection fusion to generate refined portrait regions of an image.
[0022] Another objective of this invention is to provide a layered blending loss for enriching detail segmentation and forming masked portrait regions.
[0023] Another objective of this invention is to provide a human-centered data augmentation scheme that enables uniform distribution of training data, pre-aligns the masked portrait region with the central region, and generates an output image.
[0024] Other objects and aspects of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, which illustrate by way of example features of embodiments according to the invention.
[0025] In order to achieve the above and related objectives, the present invention may be implemented in the form shown in the accompanying drawings, but it should be noted that the drawings are merely illustrative and may be modified in the specific structure illustrated and described within the scope of the appended claims.
[0026] Although the invention has been described above with reference to various exemplary embodiments and implementations, it should be understood that the various features, aspects, and functions described in one or more individual embodiments are not limited to their applicability to particular embodiments. Rather, they may be applied individually or in various combinations to one or more other embodiments of the invention, whether or not such embodiments are described, and whether such features are presented as part of the described embodiments. Therefore, the breadth and scope of the invention should not be limited by any of the exemplary embodiments described above.
[0027] In some cases, the presence of extended words and phrases such as “one or more,” “at least,” “but not limited to,” or other similar phrases should not be interpreted as implying or requiring a narrower case in the absence of such extended phrases. Attached Figure Description
[0028] The objects and features of the invention will become more apparent from the following description taken in conjunction with the accompanying drawings and the appended claims. It should be understood that these drawings depict only exemplary embodiments of the invention and are therefore not intended to limit its scope. The invention will be described and explained with additional features and details using the drawings, wherein:
[0029] Figure 1 An image processing system according to the present invention is shown;
[0030] Figure 2 The classification module of the image processing system is shown;
[0031] Figure 3A The hierarchical blending loss module of the image processing system is shown;
[0032] Figure 3B A gradient loss unit for hierarchical hybrid loss according to the present invention is shown;
[0033] Figure 3C A hierarchical mixing loss module in hierarchical mixing loss according to the present invention is shown;
[0034] Figure 4A This illustrates a human-centered data augmentation module of an image processing system;
[0035] Figure 4B A human-centered data augmentation module according to the present invention is shown; and
[0036] Figure 5 The diagram illustrates a method for processing images. Detailed Implementation
[0037] With the emergence and rapid development of smart mobile devices, people can experience a more convenient and intelligent lifestyle within a single click. Unlike traditional bulky PCs or even laptops, smartphones, with their superior portability and computing power, have become indispensable assistants in our daily lives. People use smartphones for business meetings, ordering food, and buying movie tickets.
[0038] Beyond basic and passive interaction, people now expect deeper communication with devices, forcing devices to become more intelligent. Thanks to powerful deep learning technology, intelligent devices are able to better understand the world. Since such devices will frequently interact with human users, understanding human attributes or behaviors is crucial.
[0039] Semantic segmentation, as an essential basic function, can provide the location of objects of interest in an image with pixel-level precision through machine semantic-level intelligence. It paves the way for further processing and improvements in commercial applications such as photographic enhancement, AR, and VR. However, providing high-quality and robust semantic segmentation masks on edge computing chips while reducing computational complexity is extremely challenging.
[0040] This invention proposes a human body segmentation method with the following main innovations: (1) a lightweight and efficient encoder-decoder structure with a skip fusion connection structure; (2) a hybrid loss with overall segmentation supervision and detail characterization capabilities; and (3) a new hierarchical data augmentation scheme to enable the network to perform better in real-world use cases.
[0041] This invention seeks to provide improvements for portrait segmentation. More specifically, this invention relates to improvements for human portrait segmentation. Furthermore, this invention proposes a low-cost neural network for predicting object segmentation. This invention employs a skip-fusion connection network structure, which uses fewer hidden layer parameters and significantly reduces computational cost. To handle challenging human body components, such as hair and hands, a hierarchical hybrid loss is proposed instead of the traditional segmentation loss with designed weights.
[0042] Figure 1 An image processing system according to the present invention is illustrated. This system is an image processing system 100 for processing images. Specifically, system 100 processes images by segmenting portrait regions from them. Furthermore, the image processing system includes a classification module 200, a hierarchical hybrid loss module 300, and a human-centered data augmentation module 400.
[0043] The classification module 200 includes an encoder-decoder unit and skip fusion connection logic. The encoder-decoder unit comprises an encoder and a decoder. The encoder extracts and classifies multiple distinguishing features and multiple similar features from the portrait region. Furthermore, the multiple similar features are classified based on one or more predefined levels of semantic categories. The encoder and decoder form multiple layers. The decoder refines one or more similar features to generate multiple refined features.
[0044] The skip fusion connection logic concatenates multiple refinement features to generate refined portrait regions of an image. It reuses multiple refinement features from each of one or more predefined levels to cascade one or more predefined levels in the encoder and decoder layers. Synthesis is performed by adding the skip fusion connection logic to fusion neurons. The fusion neurons are trained to learn the fusion ratio between multiple similar features. The skip fusion connection logic cascades multiple refinement features based on size similarity and channel number to add connections.
[0045] The hierarchical blending loss module 300 includes a binary cross-entropy (BCE) loss unit, a gradient loss unit, and a local binary cross-entropy (BCE) loss unit. The BCE loss unit modulates a first mask applied to a first portion of the refined portrait region. The gradient loss unit applies a second mask to the refined portrait region, and the local binary cross-entropy loss unit modulates a third mask applied to a second portion of the refined portrait region to form a masked portrait region. The first portion includes the human body, while the second portion includes the human body's hands or hair.
[0046] Furthermore, the gradient loss unit is computed using the Sobel operator. The gradient loss unit penalizes one or more bumps in the original image's portrait region. These bumps may include hair or beard. The hierarchical blending loss unit is formulated by combining each of the BCE loss unit, the gradient loss unit, and the local BCE loss unit. The hierarchical blending loss unit is adjusted using one of several weight parameters.
[0047] The human-centered data augmentation module 400 is used to pre-align the masked portrait region and the central region, and generate the output image.
[0048] Figure 2 The diagram illustrates a classification module 200 of an image processing system. The classification module 200 is used to classify portrait regions of an image. The classification module includes an encoder-decoder unit 202 and skip fusion connection logic 208.
[0049] The encoder-decoder unit 202 includes an encoder 204 and a decoder 206. The encoder 204 extracts and classifies multiple distinguishing features and multiple similar features from the portrait region. The multiple similar features are then classified based on one or more predefined levels of semantic categories. The decoder 206 refines one or more similar features to generate multiple refined features. A skip fusion connection logic 208 is used to cascade the multiple refined features to generate a refined portrait region of the image.
[0050] Image segmentation, which is essentially a pixel-level classification problem, usually requires a more complex structure to represent the relationship between region pixels and the overall semantic classification. The encoder-decoder structure is suitable for this problem because the encoder can extract discriminative and semantic features at different levels, while the decoder helps to refine semantic details along its upsampling path.
[0051] However, the pure encoder-decoder pipeline lacks communication between corresponding layers, resulting in a lack of sufficient feature details. To improve feature utilization, a skip fusion connection scheme is proposed.
[0052] Skip connections reuse features from each level of encoder 204 and improve convergence speed during training by connecting corresponding levels of features in the encoder and decoder layers.
[0053] The most straightforward skip connection scheme is to concatenate features of the same size in each corresponding layer to preserve the original description within the network. However, this inevitably increases the complexity of the framework, sometimes by at least twice the computational intensity of the decoder. To address this issue, a skip connection and fusion neuron addition operation is proposed, which requires that the features in corresponding layers not only have the same size but also the same number of channels, so that the corresponding features can be added together.
[0054] Fusion neurons can be trained to learn the fusion ratio between the feature encoder and decoder. This reduces the complexity of the resulting network. During training, backpropagation forces the network to learn the benefits of this simplified skip-connection fusion, resulting in good performance.
[0055] Figure 3A The diagram illustrates a hierarchical blending loss module 300a within an image processing system. The hierarchical blending loss module 300a includes a Binary Cross-Entropy (BCE) loss unit 302, a gradient loss unit 304, and a local BCE loss unit 306. The Binary Cross-Entropy (BCE) loss unit 302 adjusts a first mask applied to a first portion of the refined portrait region. The gradient loss unit 304 applies a second mask to the refined portrait region. The local binary cross-entropy (BCE) loss unit 306 adjusts a third mask applied to a second portion of the refined portrait region to form a masked portrait region.
[0056] The loss used in this proposal has several components. The first is the traditional BCE (Binary Cross-Entropy) loss, which normalizes the segmentation of the entire mask. In addition, gradient loss is applied to the original image and the predicted mask to preserve rich details, and the Sobel operator is used to compute the gradient.
[0057] The differences between mask images are penalized. Due to their highly nonlinear local patterns, certain body components (such as the human hand) are often difficult to handle. To address this, an additional hand loss is proposed. This is a local BCE loss that only considers the region within the hand's bounding box. To use this loss, we also annotate the hand region in the segmented ground truth data. In this way, the local loss is presented on top of the conventional loss to refine the results, thus presenting the mask results in a hierarchical manner. Together with the adjusted weight parameters, all losses are combined to obtain the following formula.
[0058] Figure 3B The illustration shows a gradient loss unit 300b according to the present invention. The loss used in this proposal has several parts. The first is a conventional BCE (Binary Cross Entropy) loss to adjust the segmentation of the entire mask. In addition, gradient loss is applied to the original image and the predicted mask to preserve rich details, and the gradient is calculated using the Sobel operator in conjunction with the following formula.
[0059]
[0060] Then, the differences between the mask images are penalized.
[0061] The first figure illustrates the gradient loss, which helps to enrich details, especially in the hair region. The gradient loss uses image 308 to mask all bumps 310 and places a mask overlay layer 312 on the image. The gradient loss penalizes any bumps at the image level, while the BCE loss constrains the human semantic mask. Their combination enables the network to recognize so-called semantic bumps, such as hair.
[0062] Figure 3C The illustration shows a hierarchical blending loss module 300c according to the present invention. The hierarchical blending loss used in this proposal has several parts. The first is a conventional BCE (Binary Cross Entropy) loss, which modulates the segmentation of the entire mask. In addition, gradient loss is applied to the original image and the predicted mask to preserve rich details, and the gradient is calculated using the Sobel operator in conjunction with the following formula.
[0063]
[0064] Then, the difference between the image and layer 2 (318) is penalized. As observed in 314, certain body components (such as the human hand) are often difficult to handle due to their highly nonlinear local patterns. To address this, an additional hand loss is observed in 316. This is a local BCE loss that only considers the region within the hand's bounding box. To use this loss, we also label the hand region in the segmented ground truth data. In this way, a local loss is presented on top of the conventional loss to refine the results, thus presenting the masked results in a hierarchical manner. Together with the adjusted weight parameters, all losses are combined to obtain the following formula.
[0065] L_total=L_BCE(image)*W_image+L_BCE(hand)*W_hand+GRD(image)*W_image
[0066] The result 318 has the applied, suggested loss mask. Figure 3C The values are displayed as 314 and 316. Figure 3B The gradient loss, which helps to enrich details, especially in the hair region, is used. The gradient loss penalizes any bumps at the image level, while the BCE loss constrains the semantic mask to human level. Their combination enables the network to recognize so-called semantic bumps, such as hair.
[0067] Furthermore, the figure shows a comparison between the hierarchical blending loss and the traditional BCE loss. First, for direct visual performance, the scheme with blending loss achieves better and more complete mask segmentation of the hand region. Moreover, due to the involvement of the hierarchical loss, the resulting mask becomes a continuous distribution, with the background in the range of 0–0.3, the foreground human body in the range of 0.6–0.8, and the hand region above 0.8, as it is severely penalized by the additional hand loss.
[0068] Furthermore, segmentation is no longer a binary classification problem, but rather a regression problem that benefits network convergence. The network can then identify which part it should focus on and put in more effort to learn and achieve better results.
[0069] Figure 4A The illustration shows a human-centered data augmentation module 400a in an image processing system. This module pre-aligns the masked portrait region with the central region and generates an output image.
[0070] The pre-alignment of training data 402 is performed by pre-alignment unit 404. The workflow is depicted in the figure below. The center and size of the human body can be found in the mask within the image and mask. The corresponding affine transformation matrix is calculated such that, when applied, the original human portrait moves to the central region of the image while undergoing relatively consistent scaling.
[0071] Figure 4B A human-centered data augmentation module 400b according to the present invention is illustrated. When preparing training data, the data distribution should be consistent with actual use cases so that the network can perform well in similar scenarios. Therefore, we propose an additional augmentation process on top of conventional data augmentation schemes during training. It can be seen that the main human figure in frame 406 has a high probability of being located in the center.
[0072] Therefore, the training data should be pre-aligned. The workflow is shown in the figure below. Given the mask center and size of the portrait image, find the center and size 408 of the human body in the mask. Calculate the corresponding affine transformation matrix so that, after its application, the original human portrait will be moved to the central region 410 of the image and scaled relatively uniformly 412. This can cover most user cases.
[0073] Furthermore, this process has the advantage of effectively suppressing background people, which facilitates operation when employing photographic effects such as defocus. After the human-centered alignment 410, augmentation 414 is performed, executing conventional data augmentation strategies such as rotation, translation, and scaling in a controlled manner to obtain pre-aligned training data 416.
[0074] Figure 5 The illustration shows a method 500 for processing images. This method includes: 502: extracting multiple discriminative features and multiple similarity features using an encoder. The multiple discriminative features and multiple similarity features are derived from one or more predefined levels of the image. 504: classifying the multiple similarity features based on semantic classification.
[0075] 506: Refine one or more similar features using a decoder to generate multiple refined features. 508: Concatenate one or more refined features of the same size. 510: Generate refined portrait regions using skip fusion connection logic.
[0076] 512: Apply a first mask to the first part of the refined portrait region based on binary cross-entropy (BCE) loss. 514: Apply a second mask to the refined portrait region based on gradient loss. 516: Apply a third mask to the second part of the refined portrait region based on local binary cross-entropy (BCE) loss.
[0077] 518: Adjust the hierarchical blending loss to generate the masked portrait region by associating the weights with each of the binary cross-entropy (BCE) loss, gradient loss, and local binary cross-entropy (BCE) loss. 520: Pre-align the masked portrait region with the central region using a human-centric data augmentation module to generate the output image.
[0078] While various embodiments of the invention have been described above, it should be understood that they are presented only as examples and not as limiting. Similarly, the figures may depict exemplary architectures or other configurations of the invention, done to aid in understanding the features and functions that may be included in the invention. The invention is not limited to the illustrated exemplary architectures or configurations, but various alternative architectures and configurations can be used to achieve the desired features.
[0079] Although the invention has been described above with reference to various exemplary embodiments and implementations, it should be understood that the various features, aspects, and functions described in one or more individual embodiments are not limited to their applicability to particular embodiments. The embodiments described may be applied individually or in various combinations to one or more other embodiments of the invention, whether or not those embodiments are described, and whether or not these features are presented as part of the described embodiments. Therefore, the breadth and scope of the invention should not be limited by any of the exemplary embodiments described above.
[0080] In some cases, the presence of expansive words and phrases such as “one or more,” “at least,” “but not limited to,” or other similar phrases should not be interpreted as indicating or requiring a narrower situation where such expansive phrases may not exist.
Claims
1. An image processing system for processing images, characterized in that, include: A classification module is used to classify the portrait regions of the image, wherein the classification module includes: An encoder-decoder unit, wherein the encoder-decoder unit comprises: An encoder is used to extract and classify multiple similar features from the portrait region, wherein the multiple similar features are classified based on one or more predefined levels of semantic categories; and A decoder is used to refine the one or more similar features to generate multiple refined features; A skip fusion connection logic is used to cascade the multiple refinement features to generate a refined portrait region of the image; A hierarchical hybrid loss module, wherein the hierarchical hybrid loss module includes: Binary cross-entropy (BCE) loss unit is used to adjust a first mask applied to a first portion of the refined portrait region, wherein the first portion includes a human portrait. Gradient loss unit, used to apply the second mask to the refined portrait region; and A local binary cross-entropy (BCE) loss unit is used to adjust a third mask applied to a second portion of the refined portrait region to form a masked portrait region, wherein the second portion includes the hands or feet of the human portrait. A human-centered data augmentation module is used to pre-align the mask portrait area with the central area and generate an output image.
2. The image processing system according to claim 1, wherein The encoder and the decoder form multiple layers.
3. The image processing system according to claim 1, wherein The skip fusion connection logic reuses the multiple refinement features for each of the one or more predefined levels to generate the refined portrait region.
4. The image processing system according to claim 3, wherein The cascading is performed by the skip fusion connection logic based on fusion neurons.
5. The image processing system as described in claim 4, characterized in that, The fusion neurons are trained to learn the fusion ratio between the multiple refinement features.
6. The image processing system according to claim 1, characterized in that, The jump fusion connection logic cascades the multiple refined features based on the similarity of the channel size and number.
7. The image processing system according to claim 1, characterized in that, The gradient loss unit applies the second mask by analyzing one or more bumps in the portrait region.
8. The image processing system according to claim 7, characterized in that, The one or more bulges are based on hair or beard.
9. The image processing system according to claim 1, characterized in that, The hierarchical hybrid loss unit calculates the hybrid loss based on the BCE loss unit, the gradient loss unit, and the local BCE loss unit.
10. The image processing system according to claim 9, characterized in that, The hybrid loss is obtained by applying weights to each of the BCE loss, gradient loss, and local BCE loss to perform a weighted calculation.
11. A method for processing images, characterized in that, include: Extract multiple similar features from the image at one or more predefined levels; The multiple similar features are classified based on semantic categories; Refine the one or more similar features to generate multiple refined features; The one or more refinement features of the same size are concatenated to generate a refined portrait region; A first mask is applied to a first portion of the refined portrait region based on the binary cross-entropy (BCE) loss, wherein the first portion includes a human portrait. A second mask is applied to the refined portrait region based on gradient loss; A third mask is applied to the second part of the refined portrait region based on the local binary cross-entropy (BCE) loss, wherein the second part includes the hands or feet of the human portrait. The blending loss is adjusted by associating weights with each of the binary cross-entropy (BCE) loss, the gradient loss, and the local binary cross-entropy (BCE) loss to generate the masked portrait region; and The masked portrait area is pre-aligned with the central area to generate the output image.
12. An electronic device, characterized in that, Includes a processor and a computer-usable medium having computer program logic for enabling at least one processor in a computer system to process images via a software platform, the computer program logic comprising: Extract multiple similar features from the image at one or more predefined levels; The multiple similar features are classified based on semantic categories; Refine the one or more similar features to generate multiple refined features; The one or more refinement features of the same size are concatenated to generate a refined portrait region; A first mask is applied to a first portion of the refined portrait region based on the binary cross-entropy (BCE) loss, wherein the first portion includes a human portrait. A second mask is applied to the refined portrait region based on gradient loss; A third mask is applied to the second part of the refined portrait region based on the local binary cross-entropy (BCE) loss, wherein the second part includes the hands or feet of the human portrait. The blending loss is adjusted by associating weights with each of the binary cross-entropy (BCE) loss, the gradient loss, and the local binary cross-entropy (BCE) loss to generate the masked portrait region; and The mask portrait area is pre-aligned with the center area to generate the output image.