Comparison learning guided cheilogramma trimming method
By optimizing feature extractor and Generative Adversarial Network (GAN) for trimming, the problem of difficulty in capturing lip texture features and lack of personalized trimming capabilities in the prior art is solved, and a more natural and personalized lip texture is achieved. Trim effect.
Patent Information
- Application Number
- CN202411479380.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-23
- Publication Date
- 2025-02-07
AI Technical Summary
The existing deep learning-based lip texture trimming technology is difficult to accurately capture and represent various features of lip texture, and lacks personalized recognition and trimming capabilities, which cannot meet users' personalized needs.
The feature extractor is optimized by the contrast learning mechanism. The feature extractor optimized by the contrast learning mechanism extractor extracts the texture features of the lips area and inputs them to generate an adversarial network (GAN) for trimming, generating natural and consistent with user expectations.
The effect and personalization of lip texture trimming are improved, and the resulting trimming results are more natural and realistic, which can better adapt to different lip characteristics and user needs.
Smart Images
Figure CN119399494A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of local image processing, and in particular to a lip wrinkle trimming method guided by contrast learning. Background Art
[0002] With the continuous development of digital image processing technology, image retouching technology based on deep learning has become increasingly mature. Among them, research on lip texture retouching has also attracted widespread attention. Traditional lip texture retouching methods often rely on manual operations or rule-based algorithms. Their effects are limited by the operator's subjective judgment and experience, and they cannot adapt well to different lip features and personalized needs. In response to the limitations of traditional methods, lip texture retouching technology based on deep learning came into being.
[0003] The deep learning-based lip texture retouching technology uses a deep neural network to learn and represent the features of lip images, thereby realizing the automatic retouching of lip texture. This technology usually includes two main steps: feature learning and retouching generation. In the feature learning stage, the deep neural network is trained with a large amount of lip image data to learn the high-level representation of lip texture. In the retouching generation stage, based on the learned lip features, the retouched lip image is generated using methods such as the generative adversarial network (GAN).
[0004] However, existing deep learning-based lip texture retouching technologies have some limitations. First, due to the complexity and diversity of lip texture, existing models often have difficulty accurately capturing and representing the various features of lip texture. Second, since the training of deep learning models requires a large amount of annotated data and computing resources, existing lip texture retouching models are often limited by the amount of data and computing power. In addition, existing models often lack the ability to personalized recognize and retouch different lip texture features, and cannot meet the requirements of personalized user needs. Summary of the invention
[0005] In view of this, the purpose of the present invention is to propose a lip texture modification method guided by comparative learning, which can improve the effect and personalization of lip texture modification.
[0006] According to one aspect of the present invention, a comparative learning-guided lip wrinkle modification method is provided, the method comprising:
[0007] Obtain the image to be processed and segment it to obtain the lip area;
[0008] Based on the lip region, a feature extractor optimized by contrastive learning mechanism is used;
[0009] Extract the texture features of the lip area through the optimized feature extractor;
[0010] The texture feature is input into a trimmer to modify the lip area.
[0011] In the above technical solution, the feature extractor optimized by the contrast learning mechanism can more accurately capture and distinguish the subtle texture features of the lip area, thereby improving the effect and accuracy of lip line correction.
[0012] The processing logic of the overall step is obtained through the following key steps:
[0013] 1. The system obtains the image to be processed and uses a deep learning model to accurately segment the lip area, ensuring that subsequent processing is focused only on the lip area and avoiding the influence of other irrelevant areas.
[0014] 2. Optimize the feature extractor based on the contrastive learning mechanism. By introducing the contrastive learning mechanism, the system is able to optimize the feature extractor so that it can better learn the unique features of the lip area. Contrastive learning enhances the discriminative ability of the feature extractor by maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs.
[0015] 3. Using the optimized feature extractor, the system can more accurately extract the texture features of the lip area. These features include not only macroscopic shape information but also microscopic texture details, which is crucial for subsequent lip line retouching.
[0016] 4. The extracted texture features are input into a specially designed trimmer, which can make fine corrections to the lip area based on the features. The trimmer may use a generative adversarial network (GAN) or other generative models to ensure that the trimmed lip area is both natural and in line with the user's expectations.
[0017] Compared with the traditional method, the specific advantages of each step are:
[0018] 1. When acquiring the image to be processed and segmenting the lip area, using a deep learning model for segmentation can better handle complex background and lighting conditions and improve the accuracy and robustness of segmentation;
[0019] 2. When optimizing the feature extractor based on the contrastive learning mechanism, the feature extractor can be optimized by the contrastive learning mechanism, which can more accurately capture and distinguish the subtle texture features of the lip area, and improve the accuracy and discrimination ability of feature extraction;
[0020] 3. When extracting the texture features of the lip area, the optimized feature extractor can capture both macro and micro texture features, providing richer information for subsequent trimming. When inputting the texture features into the trimmer for correction, the use of a generative adversarial network (GAN) or other generative models can generate more natural and user-friendly trimming results, thereby improving the realism of the trimming effect and user satisfaction.
[0021] The outstanding advantages of the overall solution are: the feature extractor optimized by the contrastive learning mechanism can more accurately capture the subtle texture features of the lip area, providing high-quality input for subsequent trimming; the use of generative adversarial networks (GAN) or other generative models for trimming can generate natural and realistic trimming results, avoiding the distortion and unnatural phenomena common in traditional methods; image segmentation and feature extraction through deep learning models can better handle complex backgrounds and lighting conditions, improving the robustness and adaptability of the system; because the trimming effect is natural and meets user expectations, user satisfaction is significantly improved, especially in application scenarios that require fine trimming (such as beauty makeup, virtual makeup trials, etc.). In summary, this solution optimizes the feature extractor through the contrastive learning mechanism and combines it with the generative model for trimming, which significantly improves the effect and accuracy of lip wrinkle trimming, and has obvious advantages over traditional methods.
[0022] In some embodiments, based on the lip region, a feature extractor optimized by contrastive learning mechanism is used, specifically:
[0023] Data enhancement is performed based on the lip region to obtain positive and negative samples; wherein the positive samples are different enhanced versions based on the lip region, and the negative samples are enhanced versions of different lip regions;
[0024] The feature extractor is used to extract lip texture features of positive and negative samples, and mapped to a low-dimensional embedding space, and a contrast loss function is defined based on the similarities and differences between the lip texture features;
[0025] The feature extractor is optimized based on the contrastive loss function.
[0026] In the above technical solutions, traditional feature extraction methods usually rely on pre-trained convolutional neural networks (CNNs), which may perform poorly when processing subtle texture features in the lip area. This solution introduces a contrastive learning mechanism, which significantly improves the discriminative ability of the feature extractor by maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs, enabling it to more accurately capture the subtle texture features of the lip area.
[0027] In some embodiments, the contrast loss function adopts the InfoNCE loss function.
[0028] In the above technical solution, InfoNCE (Info Noise Contrastive Estimation) is chosen as the loss function in contrastive learning, mainly because it performs well in maintaining feature discrimination and enhancing sample similarity. InfoNCE loss effectively learns more accurate feature expressions by maximizing the similarity between positive samples and target features while minimizing the similarity between negative samples and target features. This design enables the model to better distinguish semantically similar and dissimilar samples during contrastive learning, which is very suitable for tasks such as lip wrinkle retouching that require clear feature differences. In actual scenarios, the images generated by the generator need to be consistent with the input images at the feature level. By using InfoNCE loss, the generated model can be guided to be as close to positive samples as possible in the feature space and away from negative samples during training. This feature enables the generator to generate images with desired features, making them more in line with the requirements for specific attributes.
[0029] In some embodiments, based on the similarities and differences between the lip texture features, specifically:
[0030] For each pair of positive samples, the distance between their feature vectors in the embedding space is minimized, the cosine similarity is used to measure the similarity of the feature vectors of the positive sample pairs, and the similarity probability of the positive sample pairs is calculated through the Softmax function;
[0031] For each pair of negative samples, the distance between their feature vectors in the embedding space is maximized, the cosine similarity is used to measure the similarity of the feature vectors of the negative sample pairs, and the similarity probability of the negative sample pairs is calculated by the Softmax function.
[0032] In some embodiments, the trimmer adopts a generative adversarial network structure.
[0033] In the above technical solution, a generative adversarial network (GAN) structure is used to generate natural and realistic trimming results.
[0034] The advantages of excellent treatment of lips include:
[0035] 1High-precision texture generation, which can capture and generate subtle textures in the lip area, such as lip lines, gloss, etc. These details are crucial for natural and realistic retouching effects;
[0036] 2. Natural transition: the transition between the generated lip area and the surrounding skin is more natural, avoiding the abrupt boundary phenomenon commonly seen in traditional methods;
[0037] 3 Robustness and adaptability, which can better handle complex background and lighting conditions, generate natural and realistic lip effects in different environments, and adapt to various shooting scenes.
[0038] According to another aspect of the present invention, a neural network for comparative learning-guided lip wrinkle retouching is provided, comprising:
[0039] A segmentation subnetwork is configured to segment the acquired image to be processed to obtain a lip region;
[0040] A contrastive learning subnetwork, which is configured to be a feature extractor optimized by a contrastive learning mechanism based on the lip region;
[0041] A feature extraction subnetwork, configured to extract texture features of the lip region through an optimized feature extractor;
[0042] A trimming subnetwork is configured to modify the lip region based on the texture feature.
[0043] In the above technical scheme, in order to better use the above method, the present application proposes a neural network for lip line retouching guided by comparative learning, and each module corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.
[0044] According to another aspect of the present invention, there is provided an apparatus for lip line shaping guided by comparative learning, comprising the following modules connected in sequence:
[0045] Segmentation module: used to segment the acquired image to be processed to obtain the lip area;
[0046] A contrastive learning module, for using a feature extractor optimized by a contrastive learning mechanism based on the lip region;
[0047] Feature extraction module: used to extract the texture features of the lip area through the optimized feature extractor;
[0048] Trimming module: used to input the texture feature into a trimmer to modify the lip area.
[0049] In the above technical scheme, in order to better use the above method, the present application proposes a device for lip line correction guided by comparative learning, and each module corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.
[0050] According to another aspect of the present invention, there is provided an apparatus for comparative learning-guided lip wrinkle modification, comprising:
[0051] at least one processor and a memory communicatively coupled to the at least one processor;
[0052] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above method.
[0053] In the above technical solution, in order to better run and process the method, the above method is stored in a memory, and the stored method is executed by a processor. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.
[0054] According to another aspect of the present invention, a computer-readable storage medium is provided, storing a computer program, wherein the computer program implements the above method when executed by a processor.
[0055] In the above technical solution, in order to better run and use the method, the above method is stored in a computer-readable storage medium, and the above method is implemented by a processor. It should be noted that the principle and effect of each step have been described above and will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0057] Figure 1 It is a flow chart of an embodiment of a comparative learning guided lip wrinkle modification method of the present invention;
[0058] Figure 2 It is a logic diagram of the overall process of an embodiment of a lip wrinkle modification method guided by comparative learning of the present invention;
[0059] Figure 3 The present invention is a schematic diagram of the overall process logic of an embodiment of a device for lip line modification guided by comparative learning. DETAILED DESCRIPTION
[0060] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.
[0061] The present invention provides a lip texture trimming method guided by comparative learning, which can improve the effect and personalization of lip texture trimming.
[0062] Embodiment 1
[0063] See also Figure 1 , a comparative learning guided lip wrinkle modification method, the method comprising:
[0064] S1, obtaining the image to be processed and segmenting it to obtain the lip area;
[0065] S2, based on the lip region, a feature extractor optimized by contrastive learning mechanism;
[0066] In this embodiment, S2, based on the lip region, a feature extractor optimized by contrastive learning mechanism is used, specifically:
[0067] S211, performing data enhancement based on the lip region to obtain positive and negative samples; wherein the positive samples are different enhanced versions based on the lip region, and the negative samples are enhanced versions of different lip regions;
[0068] S212, using the feature extractor to extract lip texture features of positive and negative samples, and mapping them to a low-dimensional embedding space, and defining a contrast loss function based on the similarities and differences between the lip texture features;
[0069] S213: Optimizing the feature extractor based on the contrast loss function.
[0070] S3, extracting texture features of the lip area through the optimized feature extractor;
[0071] In this embodiment, the contrast loss function adopts the InfoNCE loss function.
[0072] In this embodiment, based on the similarities and differences between the lip texture features, specifically:
[0073] S221. For each pair of positive samples, minimize the distance between their feature vectors in the embedding space, use cosine similarity to measure the similarity of the feature vectors of the positive sample pair, and calculate the similarity probability of the positive sample pair through the Softmax function;
[0074] S222. For each pair of negative samples, the distance between their feature vectors in the embedding space is maximized, the cosine similarity is used to measure the similarity of the feature vectors of the negative sample pair, and the similarity probability of the negative sample pair is calculated by the Softmax function.
[0075] S4, inputting the texture feature into a trimmer to modify the lip area.
[0076] In this embodiment, the trimmer adopts a generative adversarial network structure. A natural and realistic trimming result is generated by a generative adversarial network (GAN) structure, so that the trimming result is more natural and realistic, and more in line with the needs of users.
[0077] In this embodiment, the present invention adopts a technical solution combining deep learning and contrastive learning, making full use of the advantages of deep learning in feature extraction, and at the same time learning the similarities and differences of lip textures through a contrastive learning mechanism, thereby improving the accuracy and robustness of feature representation. The trimmer module realizes personalized trimming of lip textures based on the feature representation obtained by contrastive learning.
[0078] A specific operation example is: suppose there is a selfie uploaded by a user, and the lip area needs to be retouched to make it look more natural and beautiful. First, the system uses a deep learning model to segment the image and accurately segment the lip area. Then, the system uses a feature extractor optimized by a contrastive learning mechanism to extract features from the segmented lip area, which significantly improves the discriminant ability of the feature extractor, enabling it to more accurately capture the subtle texture features of the lip area. Then, using the optimized feature extractor, the system can more accurately extract the texture features of the lip area. Finally, the extracted texture features are input into a specially designed generative adversarial network (GAN). The generator generates the retouched lip area based on the features, while the discriminator determines whether the generated result is natural. Through continuous iterative optimization, the generator can generate a retouched result that meets the user's expectations. Finally, the system merges the retouched lip area with the original image and outputs the final retouched image. Through the above steps, the system can efficiently and accurately retouch the lip area and generate natural and realistic retouching results. GAN has superior advantages in processing subtle textures and personalized customization of the lip area that traditional image processing methods cannot match.
[0079] Embodiment 2
[0080] A neural network for contrastive learning-guided lip wrinkle retouching, comprising:
[0081] The segmentation subnetwork is configured to segment the acquired image to be processed to obtain the lip area; in this embodiment, the subnetwork uses the segmentation network to segment the lip part of the image, so that the subsequent modules can accurately process the target area, while saving the computing power of the entire model.
[0082] The contrastive learning subnetwork is configured as a feature extractor optimized by the contrastive learning mechanism based on the lip region; in this embodiment, the subnetwork introduces the contrastive learning mechanism, performs multiple data enhancements (such as rotation, flipping, color jitter, etc.) on the segmented lip image, and generates positive and negative sample pairs (positive sample pairs: different enhanced versions of the same image, negative sample pairs: enhanced versions of different images). The feature extractor is used to extract the feature vectors of the positive and negative sample pairs, and the feature vectors are mapped to a low-dimensional embedding space to learn the similarities and differences between the lip textures. The contrastive loss function (InfoNCE loss) is defined. For each pair of positive sample pairs, the distance between their feature vectors in the embedding space is minimized. Specifically, the cosine similarity is used to measure the similarity of the feature vectors of the positive sample pairs, and the similarity probability of the positive sample pairs is calculated by the Softmax function. For each pair of negative sample pairs, the distance between their feature vectors in the embedding space is maximized. By making the similarity probability of the negative sample pairs as close to 0 as possible, the positive sample pairs are encouraged to be closer in the embedding space and the negative sample pairs to be farther away. Improve the accuracy and robustness of feature representation.
[0083] A feature extraction subnetwork is configured to extract texture features of the lip area through an optimized feature extractor; in this embodiment, the subnetwork uses a deep convolutional neural network (CNN) as a feature extractor to extract high-level abstract features of the lip texture from the input image.
[0084] A trimming subnetwork is configured to modify the lip region based on the texture feature. In this embodiment, a trimmer module is designed to achieve personalized trimming of the lip texture based on the feature representation obtained by contrastive learning. The trimmer module adopts a generative adversarial network (GAN) structure to generate natural and realistic trimming results.
[0085] In the above technical solution, in order to better use the above method, this application proposes a neural network for lip line retouching guided by contrast learning. Each module in the network corresponds to each step of the above method. The specific principle has been described above and will not be repeated here. For the overall process, please refer to Figure 2 .
[0086] Based on the first embodiment and the second embodiment, the present invention has the following advantages:
[0087] 1. Comprehensive application of deep learning and contrastive learning: The present invention adopts a technical solution that combines deep learning and contrastive learning, making full use of the advantages of deep learning in feature extraction. At the same time, it learns the similarities and differences of lip textures through the contrastive learning mechanism, thereby improving the accuracy and robustness of feature representation.
[0088] 2. The segmentation module accurately extracts the target area: The segmentation module is used to extract the lips in the image, which can accurately locate and segment the target area, avoid the accumulation of errors, and save the computing resources of subsequent modules.
[0089] 3. Personalized retouching results: The trimmer module realizes personalized retouching of lip texture based on the feature representation obtained by contrastive learning. The natural and realistic retouching results are generated through the generative adversarial network (GAN) structure, making the retouching results more natural, realistic, and more in line with user needs.
[0090] Embodiment 3
[0091] See also Figure 3 , a device for comparative learning-guided lip wrinkle reshaping, comprising the following modules connected in sequence:
[0092] Segmentation module: used to segment the acquired image to be processed to obtain the lip area;
[0093] A contrastive learning module, for using a feature extractor optimized by a contrastive learning mechanism based on the lip region;
[0094] Feature extraction module: used to extract the texture features of the lip area through the optimized feature extractor;
[0095] Trimming module: used to input the texture feature into a trimmer to modify the lip area.
[0096] In the above technical scheme, in order to better use the above method, the present application proposes a device for lip line correction guided by comparative learning, and each module corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.
[0097] Embodiment 4
[0098] A device for comparative learning-guided lip wrinkle modification, comprising:
[0099] at least one processor and a memory communicatively coupled to the at least one processor;
[0100] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in one of the embodiments.
[0101] In the above technical solution, in order to better run and process the method described in one of the embodiments, the above method is stored in a memory, and a processor is used to execute the stored method. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.
[0102] Embodiment 5
[0103] A computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the method described in one of the embodiments is implemented.
[0104] In the above technical solution, in order to better run and use the method described in one of the embodiments, the above method is stored in a computer-readable storage medium, and the above method is implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.
[0105] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A lip wrinkle correction method guided by comparative learning, characterized in that: The method comprises: Obtain the image to be processed and segment it to obtain the lip area; Based on the lip region, a feature extractor optimized by contrastive learning mechanism is used; Extract the texture features of the lip area through the optimized feature extractor; The texture feature is input into a trimmer to modify the lip area.
2. A lip wrinkle correction method guided by comparative learning as claimed in claim 1, characterized in that: Based on the lip region, a feature extractor optimized by contrastive learning mechanism is used, specifically: Data enhancement is performed based on the lip region to obtain positive and negative samples; wherein the positive samples are different enhanced versions based on the lip region, and the negative samples are enhanced versions of different lip regions; The feature extractor is used to extract lip texture features of positive and negative samples, and mapped to a low-dimensional embedding space, and a contrast loss function is defined based on the similarities and differences between the lip texture features; The feature extractor is optimized based on the contrastive loss function.
3. A comparative learning guided lip wrinkle correction method as claimed in claim 2, characterized in that: The contrast loss function adopts the InfoNCE loss function.
4. The lip wrinkle correction method guided by comparative learning as claimed in claim 2, characterized in that: Based on the similarities and differences between lip texture features, specifically: For each pair of positive samples, the distance between their feature vectors in the embedding space is minimized, the cosine similarity is used to measure the similarity of the feature vectors of the positive sample pairs, and the similarity probability of the positive sample pairs is calculated through the Softmax function; For each pair of negative samples, the distance between their feature vectors in the embedding space is maximized, the cosine similarity is used to measure the similarity of the feature vectors of the negative sample pairs, and the similarity probability of the negative sample pairs is calculated by the Softmax function.
5. The lip wrinkle correction method guided by comparative learning as claimed in claim 1, characterized in that: The trimmer adopts a generative adversarial network structure.
6. A neural network for contrastive learning-guided lip wrinkle retouching, characterized in that: include: A segmentation subnetwork is configured to segment the acquired image to be processed to obtain a lip region; A contrastive learning subnetwork, which is configured to be a feature extractor optimized by a contrastive learning mechanism based on the lip region; A feature extraction subnetwork, configured to extract texture features of the lip region through an optimized feature extractor; A trimming subnetwork is configured to modify the lip region based on the texture feature.
7. A device for lip wrinkle modification guided by comparative learning, characterized in that: Includes the following modules connected in sequence: Segmentation module: used to segment the acquired image to be processed to obtain the lip area; A contrastive learning module, for using a feature extractor optimized by a contrastive learning mechanism based on the lip region; Feature extraction module: used to extract the texture features of the lip area through the optimized feature extractor; Trimming module: used to input the texture feature into a trimmer to modify the lip area.
8. A device for lip wrinkle modification guided by comparative learning, characterized in that: include: at least one processor and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as claimed in any one of claims 1 to 5.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.