Image processing method, device, electronic device and storage medium
By decomposing face texture and reducing texture base specific gravity on face images, the problems of poor face authenticity and loss of personality characteristics after beauty treatment in the prior art are solved, and directional weakening of defects and improving user experience are achieved.
Patent Information
- Application Number
- CN202110485161.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-04-30
- Publication Date
- 2024-12-06
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
When the prior art is used to beautify the facial images, it is impossible to accurately locate the defect location and directional removal, resulting in poor authenticity of the face after the beauty treatment and loss of personality characteristics, resulting in high similarity after the facial image processing of different users. , poor user experience.
By decomposing the face texture of the face image, the multiple frequency texture substrates that make up the face texture and their specific gravity are determined, and the specific gravity of the first texture substrate that meets the preset filtering conditions are reduced to generate a target face image.
It realizes directional weakening of facial flaws, retaining the real features of the face to the greatest extent, reducing the gap between the image after beauty processing and the original face image, reducing the similarity of face images for different users after facial images, and improving the user experience.
Smart Images

Figure CN113177879B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, in particular to the field of artificial intelligence technologies such as computer vision, augmented reality and deep learning, and more particularly to image processing methods, devices, electronic devices and storage media. Background Art
[0002] With the increasing popularity of mobile Internet and the increasing number of social media users, more and more people choose to take photos of their daily lives and share them through social networks. At the same time, in order to make the captured images more attractive, people usually perform beautification processing on the images, such as reducing wrinkles, eliminating spots, and smoothing the skin.
[0003] However, in the related art, when beautifying facial images, since it is impossible to accurately locate the location of defects and remove them in a targeted manner, the entire facial area is usually subjected to processes such as skin smoothing, resulting in poor authenticity of the face after beautification processing and a large gap with the original facial image. In addition, due to the loss of individual characteristics, the facial images of different users are very similar after processing, resulting in a poor user experience. Summary of the invention
[0004] The present disclosure provides an image processing method, an apparatus, an electronic device, a storage medium, and a computer program product.
[0005] According to one aspect of the present disclosure, there is provided an image processing method, comprising: acquiring a face image to be processed; performing face texture decomposition on the face image to determine texture bases of multiple frequencies constituting the face texture, and proportions of the multiple texture bases; reducing the proportion of a first texture base among the multiple texture bases, wherein the first texture base is a texture base that satisfies a preset filtering condition; and generating a target face image based on the multiple texture bases and the corresponding processed proportions.
[0006] According to another aspect of the present disclosure, there is provided an image processing device, comprising: an acquisition module for acquiring a face image to be processed; a determination module for performing face texture decomposition on the face image to determine a plurality of frequency texture bases constituting the face texture, and a proportion of the plurality of texture bases; a processing module for reducing the proportion of a first texture base among the plurality of texture bases, wherein the first texture base is a texture base that satisfies a preset filtering condition; and a generation module for generating a target face image based on the plurality of texture bases and the corresponding processed proportions.
[0007] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the image processing method as described above.
[0008] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the image processing method as described above.
[0009] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the image processing method as described above.
[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0012] Figure 1 is a flowchart of an image processing method according to a first embodiment of the present disclosure;
[0013] Figure 2 is a flowchart of an image processing method according to a second embodiment of the present disclosure;
[0014] Figure 3 is a flowchart of an image processing method according to a third embodiment of the present disclosure;
[0015] Figure 4 is a schematic structural diagram of an image processing device according to a fourth embodiment of the present disclosure;
[0016] Figure 5 is a structural schematic diagram of an image processing device according to a fifth embodiment of the present disclosure;
[0017] Figure 6 It is a block diagram of an electronic device used to implement the image processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0018] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0019] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0020] It is understandable that when beautifying facial images in related technologies, since it is impossible to accurately locate the location of defects for targeted removal, the entire facial area is usually subjected to processes such as skin smoothing, resulting in poor authenticity of the face after beautification processing and a large gap with the original facial image. In addition, due to the loss of individual characteristics, the facial images of different users are very similar after processing, resulting in a poor user experience.
[0021] In view of the above technical problems, the present disclosure proposes an image processing method. After obtaining a face image to be processed, the face image is decomposed into face textures to determine the texture bases of multiple frequencies constituting the face texture and the weights of the multiple texture bases, and then the weight of the first texture base that meets the preset filtering conditions among the multiple texture bases is reduced, and then a target face image is generated according to the multiple texture bases and the corresponding processed weights. Thus, the face defects are weakened in a directional manner, and the real features of the face can be retained to the greatest extent, the authenticity of the face after beautification is improved, the gap between the processed image and the original face image is reduced, and since the individual features will not be lost, the similarity of the processed face images of different users is reduced, thereby improving the user experience.
[0022] The following describes an image processing method, an apparatus, an electronic device, a non-transitory computer-readable storage medium, and a computer program product according to embodiments of the present disclosure with reference to the accompanying drawings.
[0023] First combine Figure 1 , the image processing method provided by the present invention is described in detail.
[0024] Figure 1 It is a flowchart of an image processing method according to the first embodiment of the present disclosure. It should be noted that the image processing method provided in this embodiment is performed by an image processing device, wherein the device can be implemented by software and / or hardware, and the device can be configured in an electronic device, and the electronic device can include but is not limited to a terminal, a server, etc., to achieve directional weakening of facial defects and retain the true features of the face to the greatest extent.
[0025] The electronic device may be any computing device capable of performing image processing, such as a laptop computer, a smart phone, a wearable device, and the like, and the present disclosure does not impose any limitation on this.
[0026] like Figure 1 As shown, the image processing method may include the following steps:
[0027] Step 101, obtaining a face image to be processed.
[0028] The image to be processed may be captured by a camera in the electronic device where the image processing device is located, or may be acquired by the image processing device from other devices, which is not limited in the present disclosure.
[0029] The face image to be processed may be a two-dimensional face image, and the face image may include a face area.
[0030] It should be noted that the image processing device in the embodiments of the present disclosure can obtain the facial image to be processed through various public, legal and compliant methods, for example, it can obtain the facial image to be processed from a public data set, or it can obtain the facial image to be processed from the user after the user's authorization. The present disclosure does not impose any restrictions on this.
[0031] In an exemplary embodiment, after authorization by user A, the camera device of the electronic device where the image processing device is located can be turned on, and an image including the facial area of user A can be captured based on the camera device, and the image can be used as the facial image to be processed; or, multiple video frames of the scene where the electronic device is located can be captured based on the camera device, and an image including the facial area of user A can be obtained by parsing the multiple video frames, and the image can be used as the facial image to be processed. The present disclosure does not impose any restrictions on this.
[0032] Step 102, performing facial texture decomposition on the facial image to determine the texture bases of multiple frequencies constituting the facial texture and the proportions of the multiple texture bases.
[0033] The texture bases of multiple frequencies that make up the facial texture can be understood as multiple basic texture images, which are related to the frequency, and each texture image corresponds to a frequency, which can describe the facial base components. The facial base components can be, for example, the texture and contour of the face itself.
[0034] In an exemplary embodiment, the texture bases of multiple frequencies constituting the facial texture may be acquired in advance.
[0035] It is understandable that, since the principal component analysis method can transform a set of variables that may be correlated into a set of linearly uncorrelated variables through orthogonal transformation, where the transformed set of variables is called principal components, then, in the embodiments of the present disclosure, the principal component analysis method can be used to obtain multiple linearly uncorrelated texture bases. Specifically, the principal component analysis method can be used to perform Gaussian distribution fitting on multiple sample face images to obtain a parameterized face model, thereby decomposing multiple sample face images into texture bases of multiple frequencies that constitute face textures, while reducing the variable dimension.
[0036] It is understandable that for different facial images, the proportions of the texture bases of multiple frequencies that make up the facial texture are different. For example, assuming that the texture bases of multiple frequencies that make up the facial texture include: texture base T1 corresponding to frequency f1, texture base T2 corresponding to frequency f2, and texture base T3 corresponding to frequency f3, then for facial image a, the proportion of texture base T1 can be a1, the proportion of texture base T2 can be a2, and the proportion of texture base T3 can be a3. For facial image b, the proportion of texture base T1 can be b1, the proportion of texture base T2 can be b2, and the proportion of texture base T3 can be b3, where a1, a2, a3 and b1, b2, b3 have different values. Then, in the disclosed embodiment, after obtaining the facial image to be processed, the facial texture of the facial image to be processed can be decomposed to determine the proportions of the multiple texture bases corresponding to the facial image to be processed.
[0037] In an exemplary embodiment, for example, the least squares method or other curve fitting methods can be used to compare the facial texture information of the facial image to be processed with multiple texture bases to determine the proportion of the multiple texture bases when the multiple texture bases constitute the facial texture of the facial image to be processed. The present disclosure does not limit the method of determining the proportion of the texture bases of multiple frequencies that constitute the facial texture.
[0038] Step 103 : reducing the specific gravity of a first texture base among the multiple texture bases, wherein the first texture base is a texture base that meets a preset filtering condition.
[0039] Step 104: Generate a target face image according to the multiple texture bases and the corresponding processed weights.
[0040] The preset filtering conditions are preset conditions that must be satisfied by the texture base that needs to be removed or weakened, and the preset filtering conditions can be set as needed.
[0041] It can be understood that for a face image, the facial texture information of the face image contains texture features of various frequencies, among which high-frequency texture features contain information about fine details, such as skin pores, fine lines, skin defects, etc., and low-frequency texture features contain information related to tone and color, such as shadow and highlight areas.
[0042] In an exemplary embodiment, since facial defects correspond to high-frequency texture features, in order to remove or weaken the facial defects in the facial image to be processed, the preset filtering condition can be set to a frequency higher than a preset frequency threshold, so that by reducing the proportion of the first texture base with a frequency higher than the preset frequency threshold among multiple texture bases, the high-frequency texture features in the facial texture information of the facial image to be processed can be removed or weakened in a targeted manner, thereby removing or weakening the facial defects in the target facial image generated according to the multiple texture bases and the corresponding processed proportions.
[0043] Alternatively, in an exemplary embodiment, since a portion of the texture bases of multiple frequencies that constitute the facial texture may be unimportant, the preset filtering condition can be set to an importance lower than a preset importance threshold, so that by reducing the proportion of the first texture base whose importance is lower than the preset importance threshold among the multiple texture bases, unimportant texture features in the facial texture information of the facial image to be processed can be directed to be removed or weakened, thereby beautifying the target facial image generated based on the multiple texture bases and the corresponding processed proportions. For example, the noise in the facial image to be processed can be removed, making the face in the target facial image smoother.
[0044] By decomposing the facial texture of the facial image and separating the facial feature information, and removing or weakening the facial blemishes by reducing the proportion of the first texture base that meets the preset filtering conditions, the directional removal or weakening of facial blemishes is achieved. Moreover, since the proportion of the texture base that does not meet the preset filtering conditions is not processed, the real features of the face can be retained to the greatest extent, the authenticity of the face after beauty processing is improved, the gap between the processed image and the original facial image is reduced, and since individual features will not be lost, the similarity of facial images of different users after processing is reduced, thereby improving the user experience.
[0045] The image processing method provided by the embodiment of the present disclosure obtains the face image to be processed, first decomposes the face texture of the face image, determines the texture bases of multiple frequencies that constitute the face texture, and the proportions of the multiple texture bases, and then reduces the proportion of the first texture base among the multiple texture bases, wherein the first texture base is a texture base that meets the preset filtering conditions, and then generates a target face image based on the multiple texture bases and the corresponding processed proportions. In this way, the directional weakening of face defects is achieved, and the real features of the face can be retained to the greatest extent, the authenticity of the face after beautification is improved, the gap between the processed image and the original face image is reduced, and because the individual features will not be lost, the similarity of the processed face images of different users is reduced, and the user experience is improved.
[0046] Combine the following Figure 2 , further illustrating the image processing method provided by the present invention.
[0047] Figure 2 FIG. 1 is a flow chart of an image processing method according to a second embodiment of the present disclosure. Figure 2 As shown, the image processing method may include the following steps:
[0048] Step 201, obtaining a face image to be processed.
[0049] The specific implementation process and principle of step 201 may refer to the description of the above embodiment and will not be repeated here.
[0050] Step 202, performing facial texture decomposition on the facial image to determine multiple texture bases constituting the facial texture and the proportions of the multiple texture bases.
[0051] In an exemplary embodiment, after obtaining a face image to be processed, facial texture information of the face image may be extracted first, and then facial texture information may be decomposed to determine multiple texture bases and corresponding proportions.
[0052] Among them, facial texture information contains texture features of various frequencies, among which high-frequency texture features contain information about fine details, such as skin pores, fine lines, skin defects, etc., and low-frequency texture features contain information related to tone and color, such as shadow and highlight areas.
[0053] In an exemplary embodiment, a principal component analysis method can be used to perform Gaussian distribution fitting on multiple sample face images to obtain a parameterized face model, thereby decomposing the multiple sample face images into texture bases of multiple frequencies that constitute the face texture, while reducing the variable dimension, and using, for example, the least squares method to compare the face texture information of the face image to be processed with the multiple texture bases to determine the proportion of the multiple texture bases when the multiple texture bases constitute the face texture of the face image to be processed.
[0054] After extracting the facial texture information of the face image, the facial texture information is decomposed to determine multiple texture bases and their corresponding proportions. The texture decomposition method is used to construct the texture base that constitutes the facial texture and separate the facial feature information, which provides conditions for directional removal or weakening of facial defects and retaining the true features of the face to the greatest extent.
[0055] Step 203: determine the reduction ratio corresponding to the preset filtering condition.
[0056] Step 204 , reducing the proportion of a first texture base among the plurality of texture bases according to the reduction ratio, wherein the first texture base is a texture base that meets a preset filtering condition.
[0057] Step 205 , generating a target face image according to the multiple texture bases and the corresponding processed weights.
[0058] The preset filtering conditions may include at least one of the following conditions: the frequency is higher than a preset frequency threshold, and the entropy determined according to the texture base and the specific gravity is greater than a preset quantity threshold.
[0059] In an exemplary embodiment, since facial defects correspond to high-frequency texture features, in order to remove or weaken the facial defects in the facial image to be processed, the preset filtering condition can be set to a frequency higher than a preset frequency threshold, so that by reducing the proportion of the first texture base with a frequency higher than the preset frequency threshold among multiple texture bases, the high-frequency texture features in the facial texture information of the facial image to be processed can be removed or weakened in a targeted manner, thereby removing or weakening the facial defects in the target facial image generated according to the multiple texture bases and the corresponding processed proportions.
[0060] Alternatively, since entropy is a standard for measuring the complexity of facial images, the more facial textures there are in a facial image, the more complex the surface features are and the more noise it contains, the greater the entropy; the smoother the facial skin in the facial image, the lower the complexity of the surface features and the less noise it contains, the smaller the entropy. In this case, the preset filtering condition can be set to an entropy greater than a preset threshold value, thereby determining the entropy corresponding to each texture base based on the texture base and the corresponding proportion, and reducing the proportion of the first texture base whose entropy is greater than the preset threshold value, thereby directionally removing or weakening features close to noise in the facial image to be processed, thereby making the face in the target facial image generated based on multiple texture bases and the corresponding processed proportions smoother.
[0061] In an exemplary embodiment, the preset filtering condition can also be set to a frequency higher than a preset frequency threshold and an entropy greater than a preset quantity threshold, so that by reducing the proportion of the first texture base among multiple texture bases whose frequency is higher than the preset frequency threshold and whose entropy is greater than the preset frequency threshold, the high-frequency texture features and noise-close features in the facial image to be processed can be removed or weakened in a targeted manner, thereby removing or weakening the facial defects in the target facial image generated according to multiple texture bases and the corresponding processed proportions, and making the face smoother.
[0062] By setting the preset filtering condition to a frequency higher than a preset frequency threshold or an entropy greater than a preset quantity threshold, and reducing the proportion of the first texture base that meets the preset filtering condition among multiple texture bases, the high-frequency texture features or features close to noise in the facial image to be processed are directional removed or weakened.
[0063] In an exemplary embodiment, when reducing the proportion of a first texture base that meets a preset filtering condition among multiple texture bases, the reduction ratio corresponding to the preset filtering condition can be first determined, and then the proportion of the first texture base can be reduced according to the reduction ratio.
[0064] The reduction ratio can be set as needed, for example, 20%, 10%, etc.
[0065] For example, assuming that the preset filtering condition is that the frequency is higher than the preset frequency threshold, then after determining the texture bases of multiple frequencies that constitute the facial texture, and the proportion of the multiple texture bases, the reduction ratio corresponding to the preset filtering condition can be determined. Assuming the reduction ratio is 20%, the proportion of the first texture base among the multiple texture bases whose frequency is higher than the preset frequency threshold can be reduced to 20% of the original, thereby achieving the reduction of the proportion of the first texture base.
[0066] By determining the reduction ratio corresponding to the preset filtering condition and reducing the proportion of the first texture base according to the reduction ratio, it is possible to reduce the proportion of the first texture to different degrees as needed, so as to flexibly remove facial defects or noise features in the facial image to be processed or weaken them to the required degree as needed.
[0067] The image processing method provided by the embodiment of the present disclosure obtains the face image to be processed, decomposes the face texture of the face image, determines the multiple texture bases that constitute the face texture and the proportion of the multiple texture bases, and then determines the reduction ratio corresponding to the preset filtering condition, and reduces the proportion of the first texture base among the multiple texture bases according to the reduction ratio, wherein the first texture base is a texture base that meets the preset filtering condition, and then generates a target face image based on the multiple texture bases and the corresponding processed proportions, thereby achieving directional weakening of face defects and being able to retain the real features of the face to the greatest extent, thereby improving the authenticity of the face after beautification, reducing the gap between the processed image and the original face image, and since individual features will not be lost, reducing the similarity of the processed face images of different users, thereby improving the user experience.
[0068] From the above analysis, it can be seen that in the embodiment of the present disclosure, after the proportion of the first texture base that meets the preset filtering conditions among multiple texture bases is reduced, a target facial image can be generated based on the multiple texture bases and the corresponding processed proportions. The following is a further explanation of the process of generating a target facial image based on multiple texture bases and the corresponding processed proportions in the image processing method provided by the present disclosure.
[0069] Figure 3 FIG. 1 is a flow chart of an image processing method according to a third embodiment of the present disclosure. Figure 3 As shown, the image processing method may include the following steps:
[0070] Step 301, obtaining a face image to be processed.
[0071] Step 302, performing facial texture decomposition on the facial image to determine the texture bases of multiple frequencies constituting the facial texture and the proportions of the multiple texture bases.
[0072] Step 303 : reducing the specific gravity of a first texture base among the multiple texture bases, wherein the first texture base is a texture base that meets a preset filtering condition.
[0073] The specific implementation process and principle of the above steps 301-303 can be referred to the description of the above embodiment and will not be repeated here.
[0074] Step 304: Generate target face texture information according to the multiple texture bases and the corresponding processed weights.
[0075] In an exemplary embodiment, after reducing the proportion of the first texture base among the multiple texture bases, target face texture information can be generated according to the multiple texture bases and the proportion corresponding to each texture base.
[0076] In an exemplary embodiment, a plurality of texture bases and the corresponding processed weights may be weighted and summed to obtain the target face texture information. The weight of each texture base is used as the weight for the weighted summation. Alternatively, other methods may be used to generate the target face texture information based on a plurality of texture bases and the weights corresponding to each texture base, which is not limited in the present disclosure.
[0077] Step 305: Generate a target face image according to the target face texture information and the face image.
[0078] In an exemplary embodiment, after the target face texture information is generated according to a plurality of texture bases and the corresponding processed weights, a target face image can be generated according to the target face texture information and the face image.
[0079] In an exemplary embodiment, a target facial image can be generated based on the target facial texture information and the facial image in the following manner: determine a facial model of the facial image; generate a target facial area image based on the facial model and the target facial texture information; replace the facial area image in the facial image with the target facial area image to obtain the target facial image.
[0080] The face model is a three-dimensional face model. The target face region image is an image including a face region, and the face region contains the target face texture information.
[0081] Specifically, facial features can be extracted from the facial image to be processed, and a facial model corresponding to the facial image can be constructed based on the extracted facial features. Then, a target facial area image can be generated based on the facial model and the target facial texture information, and the facial area image in the facial image to be processed can be determined. Then, the facial area image in the facial image to be processed can be replaced with the target facial area image to obtain the target facial image.
[0082] It should be noted that in the embodiment of the present disclosure, the facial model determined based on the facial image includes the facial information of the user corresponding to the facial image, but the construction of the facial model is performed after the user's authorization, and its construction process complies with relevant laws and regulations.
[0083] By generating target face texture information according to multiple texture bases and corresponding processed weights, determining the face model of the face image, generating a target face area image according to the face model and the target face texture information, and then replacing the face area image in the face image with the target face area image to obtain the target face image, it is achieved that the target face image is generated according to multiple texture bases and corresponding processed weights, and in the generated target face image, only the face area is beautified relative to the original face image, and other areas except the face area are the same as the original face image, thereby further improving the authenticity of the face image after beauty processing and reducing the gap between the processed image and the original face image.
[0084] The image processing method provided by the embodiment of the present disclosure obtains a face image to be processed, performs face texture decomposition on the face image, determines multiple texture bases constituting the face texture and the proportions of the multiple texture bases, reduces the proportion of the first texture base among the multiple texture bases, generates target face texture information according to the multiple texture bases and the corresponding processed proportions, and then generates a target face image according to the target face texture information and the face image, thereby achieving directional weakening of face defects and being able to retain the real features of the face to the greatest extent, improving the authenticity of the face after beautification, reducing the gap between the processed image and the original face image, and because individual features will not be lost, reducing the similarity of the processed face images of different users, thereby improving the user experience.
[0085] Combine the following Figure 4 , the image processing device provided by the present invention is described.
[0086] Figure 4 is a schematic structural diagram of an image processing device according to a fourth embodiment of the present disclosure.
[0087] like Figure 4 As shown, the image processing device 400 provided by the present disclosure includes: an acquisition module 401, a determination module 402, a processing module 403 and a generation module 404.
[0088] Wherein, the acquisition module 401 is used to acquire the face image to be processed;
[0089] A determination module 402 is used to perform facial texture decomposition on the facial image, determine the texture bases of multiple frequencies constituting the facial texture, and the proportions of the multiple texture bases;
[0090] A processing module 403 is used to reduce the specific gravity of a first texture base among the multiple texture bases, wherein the first texture base is a texture base that meets a preset filtering condition;
[0091] The generating module 404 is used to generate a target face image according to a plurality of texture bases and corresponding processed weights.
[0092] It should be noted that the image processing device provided in this embodiment can execute the image processing method of the aforementioned embodiment to achieve directional weakening of facial defects and retain the real features of the face to the greatest extent.
[0093] It should be noted that the above description of the embodiment of the image processing method is also applicable to the image processing device provided in the present disclosure, and will not be repeated here.
[0094] The image processing device provided by the embodiment of the present disclosure obtains the face image to be processed, first decomposes the face texture of the face image, determines the texture bases of multiple frequencies that constitute the face texture, and the proportions of the multiple texture bases, and then reduces the proportion of the first texture base among the multiple texture bases, wherein the first texture base is a texture base that meets the preset filtering conditions, and then generates a target face image based on the multiple texture bases and the corresponding processed proportions. In this way, the directional weakening of face defects is achieved, and the real features of the face can be retained to the greatest extent, the authenticity of the face after beautification is improved, the gap between the processed image and the original face image is reduced, and because the individual features will not be lost, the similarity of the processed face images of different users is reduced, thereby improving the user experience.
[0095] Combine the following Figure 5 , the image processing device provided by the present invention is described.
[0096] Figure 5 is a schematic structural diagram of an image processing device according to the fifth embodiment of the present disclosure.
[0097] like Figure 5 As shown, the image processing device 500 may specifically include: an acquisition module 501, a determination module 502, a processing module 503 and a generation module 504, wherein: Figure 5 The acquisition module 501, the determination module 502, the processing module 503 and the generation module 504 are Figure 4 The acquisition module 401, the determination module 402, the processing module 403 and the generation module 404 have the same function and structure.
[0098] In an exemplary embodiment, if Figure 5 As shown, the above-mentioned determination module 502 includes:
[0099] Extraction unit 5021, used to extract facial texture information of the facial image;
[0100] The first determining unit 5022 is used to perform facial texture decomposition on the facial texture information to determine multiple texture bases and corresponding weights.
[0101] In an exemplary embodiment, the preset filtering condition includes at least one of the following conditions: the frequency is higher than a preset frequency threshold, and the entropy determined according to the texture base and the specific gravity is greater than a preset quantity threshold.
[0102] In an exemplary embodiment, the generating module 504 includes:
[0103] A first generating unit 5041 is used to generate target face texture information according to a plurality of texture bases and corresponding processed weights;
[0104] The second generating unit 5042 is used to generate a target face image according to the target face texture information and the face image.
[0105] In an exemplary embodiment, the second generating unit includes:
[0106] A determination subunit, used for determining a face model of a face image;
[0107] A generating subunit, used for generating a target face region image according to a face model and target face texture information;
[0108] The processing subunit is used to replace the face area image in the face image with the target face area image to obtain the target face image.
[0109] In an exemplary embodiment, the processing module 503 includes:
[0110] The second determining unit 5031 is used to determine a reduction ratio corresponding to a preset filtering condition;
[0111] The processing unit 5032 is used to reduce the specific gravity of the first texture base according to the reduction ratio.
[0112] It should be noted that the above description of the embodiment of the image processing method is also applicable to the image processing device provided in the present disclosure, and will not be repeated here.
[0113] The image processing device provided by the embodiment of the present disclosure obtains the face image to be processed, first decomposes the face texture of the face image, determines the texture bases of multiple frequencies that constitute the face texture, and the proportions of the multiple texture bases, and then reduces the proportion of the first texture base among the multiple texture bases, wherein the first texture base is a texture base that meets the preset filtering conditions, and then generates a target face image based on the multiple texture bases and the corresponding processed proportions. In this way, the directional weakening of face defects is achieved, and the real features of the face can be retained to the greatest extent, the authenticity of the face after beautification is improved, the gap between the processed image and the original face image is reduced, and because the individual features will not be lost, the similarity of the processed face images of different users is reduced, thereby improving the user experience.
[0114] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0115] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0116] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0117] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0118] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the image processing method in any other appropriate manner (e.g., by means of firmware).
[0119] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0120] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0121] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0122] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0123] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.
[0124] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.
[0125] The embodiments of the present disclosure relate to artificial intelligence technology fields such as computer vision, augmented reality, and deep learning.
[0126] Among them, artificial intelligence (AI) is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision, speech recognition technology, natural language processing technology, machine learning / deep learning, big data processing technology, knowledge graph technology, etc.
[0127] Computer vision refers to the use of cameras and computers to replace the human eye to identify, track and measure targets, and further perform graphic processing so that the computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection.
[0128] Augmented Reality (AR) technology is a technology that cleverly integrates virtual information with the real world. It widely uses a variety of technical means such as multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, and sensing. It simulates computer-generated virtual information such as text, images, three-dimensional models, music, and videos, and applies them to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world.
[0129] Deep learning is the process of learning the inherent laws and representation levels of sample data. The information obtained in the learning process is very helpful for interpreting data such as text, images, and sounds. The ultimate goal of deep learning is to enable machines to have analytical learning capabilities like humans and to recognize data such as text, images, and sounds.
[0130] According to the technical solution of the embodiment of the present disclosure, after obtaining the face image to be processed, the face texture of the face image is first decomposed to determine the texture bases of multiple frequencies that constitute the face texture and the proportions of the multiple texture bases, and then the proportion of the first texture base among the multiple texture bases is reduced, wherein the first texture base is a texture base that meets the preset filtering conditions, and then the target face image is generated according to the multiple texture bases and the corresponding processed proportions. In this way, the directional weakening of face defects is achieved, and the real features of the face can be retained to the greatest extent, the authenticity of the face after beautification is improved, the gap between the processed image and the original face image is reduced, and because the individual features will not be lost, the similarity of the processed face images of different users is reduced, and the user experience is improved.
[0131] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0132] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. An image processing method, comprising: Obtaining the face image to be processed; Performing facial texture decomposition on the facial image to determine texture bases of multiple frequencies constituting the facial texture and proportions of the multiple texture bases, wherein the texture bases of multiple frequencies constituting the facial texture are multiple texture images as a basis, the texture images are related to the frequencies, and each of the texture images corresponds to a frequency; Performing a process of reducing the specific gravity of a first texture base among the plurality of texture bases, wherein the first texture base is a texture base that meets a preset filtering condition; Generate a target face image according to the plurality of texture bases and the corresponding processed weights; The step of performing facial texture decomposition on the facial image to determine the texture bases of multiple frequencies constituting the facial texture and the proportions of the multiple texture bases includes: Extracting facial texture information of the facial image, wherein the facial texture information of the facial image includes texture features of various frequencies; Perform facial texture decomposition on the facial texture information to determine a plurality of texture bases and corresponding proportions.
2. The method according to claim 1, wherein: The preset filtering condition includes at least one of the following conditions: the frequency is higher than a preset frequency threshold, and the entropy determined according to the texture base and the specific gravity is greater than a preset quantity threshold.
3. The method according to claim 1, wherein: The step of generating a target face image according to the plurality of texture bases and the corresponding processed weights includes: Generating target face texture information according to the plurality of texture bases and the corresponding processed weights; The target face image is generated according to the target face texture information and the face image.
4. The method according to claim 3, wherein: The step of generating the target face image according to the target face texture information and the face image comprises: Determining a face model of the face image; Generate a target face area image according to the face model and the target face texture information; The face region image in the face image is replaced with the target face region image to obtain the target face image.
5. The method according to claim 1 or 2, wherein: The method of reducing the specific gravity of the first textured substrate among the plurality of textured substrates comprises: Determine the reduction ratio corresponding to the preset filtering condition; The specific gravity of the first textured substrate is reduced according to the reduction ratio.
6. An image processing device, comprising: An acquisition module, used for acquiring a face image to be processed; A determination module, configured to perform facial texture decomposition on the facial image, determine a plurality of frequency texture bases constituting the facial texture, and a proportion of the plurality of texture bases, wherein the plurality of frequency texture bases constituting the facial texture are a plurality of basic texture images, the texture images are related to the frequencies, and each of the texture images corresponds to a frequency; A processing module, configured to reduce the specific gravity of a first texture base among the plurality of texture bases, wherein the first texture base is a texture base that meets a preset filtering condition; A generating module, used for generating a target face image according to the plurality of texture bases and the corresponding processed weights; Wherein, the determination module includes: An extraction unit, used to extract facial texture information of the facial image, wherein the facial texture information of the facial image includes texture features of various frequencies; The first determination unit is used to perform facial texture decomposition on the facial texture information to determine a plurality of texture bases and corresponding proportions.
7. The device according to claim 6, wherein: The preset filtering condition includes at least one of the following conditions: the frequency is higher than a preset frequency threshold, and the entropy determined according to the texture base and the specific gravity is greater than a preset quantity threshold.
8. The device according to claim 6, wherein: The generating module comprises: A first generating unit, configured to generate target face texture information according to the plurality of texture bases and corresponding processed weights; The second generating unit is used to generate the target face image according to the target face texture information and the face image.
9. The device according to claim 8, wherein: The second generating unit comprises: A determination subunit, used to determine a face model of the face image; A generating subunit, used for generating a target face region image according to the face model and the target face texture information; The processing subunit is used to replace the face area image in the face image with the target face area image to obtain the target face image.
10. The device according to claim 6 or 7, wherein: The processing module comprises: A second determining unit, used to determine a reduction ratio corresponding to the preset filtering condition; A processing unit is used to reduce the specific gravity of the first textured substrate according to the reduction ratio.
11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.
12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.
13. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.
