Image processing method, device, medium and electronic device based on artificial intelligence

Abstract

The present disclosure provides an image processing method, device, medium and electronic device based on artificial intelligence, which relates to the field of artificial intelligence. The method includes: obtaining an original face image, extracting features from the original face image to obtain a face area image; inputting the face area image into a generative network model, extracting features from the face area image through the generative network model, and generating a first image based on the extracted features; wherein the features of the target object in the first image are different from the features of the target object in the original face image; aligning and superimposing the first image with the original face image to obtain a second image. The present disclosure can process the received face image in real time, and can also keep the resolution of the generated image the same as that of the original face image, thereby improving the quality of the generated image and thus improving the user experience.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based image processing method, an artificial intelligence-based image processing device, a computer storage medium, and an electronic device. Background Art

[0002] With the changes in computer technology and the rapid development of artificial intelligence, artificial intelligence technology is widely used in various electronic products, such as smart phones, portrait processing software, sweeping robots, etc.

[0003] At present, people like to use portrait processing software such as beauty camera when taking photos, hoping to make changes to the portraits they take, such as modifying and optimizing the eyes, nose, mouth, hair and other parts. However, existing portrait processing software finds it difficult to simulate realistic three-dimensional effects in real time when the face angle changes, and there are fewer special effects to choose from, and the resolution of the generated image is lower than the original image, which reduces the user experience.

[0004] In view of this, there is an urgent need to develop a new image processing method based on artificial intelligence in this field.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The embodiments of the present disclosure provide an artificial intelligence-based image processing method, an artificial intelligence-based image processing device, a computer storage medium, and an electronic device, which can simulate a three-dimensional effect on a facial image in real time, at least to a certain extent, and maintain the resolution of each part of the generated image, thereby further improving the user experience.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to one aspect of an embodiment of the present disclosure, there is provided an artificial intelligence-based image processing method, comprising: obtaining an original face image, performing feature extraction on the original face image to obtain a face area image; inputting the face area image into a generative network model, performing feature extraction on the face area image through the generative network model, and generating a first image based on the extracted features; wherein features of a target object in the first image are different from features of the target object in the original face image; and aligning and superimposing the first image with the original face image to obtain a second image.

[0009] According to one aspect of an embodiment of the present disclosure, there is provided an artificial intelligence-based image processing device, comprising: an image acquisition module, used to acquire an original face image, perform feature extraction on the original face image to acquire a face area image; an image processing module, used to input the face area image into a generative network model, perform feature extraction on the face area image through the generative network model, and generate a first image based on the extracted features; wherein features of a target object in the first image are different from features of the target object in the original face image; and an image overlay module, used to align and overlay the first image with the original face image to acquire a second image.

[0010] In some embodiments of the present disclosure, based on the aforementioned scheme, the image acquisition module is configured to: detect the facial area in the original facial image, and perform feature registration on the facial area to determine the facial feature points; determine the feature area corresponding to the facial feature points based on the position information of the facial feature points; expand the feature area by a preset multiple, and determine the facial area image based on the expanded feature area.

[0011] In some embodiments of the present disclosure, based on the aforementioned scheme, the image processing module includes: a channel image generation unit, used to perform feature extraction on the face area image to obtain contour features and facial features, obtain an RGB channel image and an α channel image based on the contour features and the facial features, and use the RGB channel image, the α channel image and the RGB channel image as the first image; or, used to perform feature extraction on the face area image to obtain contour features and facial features, obtain an RGB channel image based on the contour features and the facial features, and use the RGB channel image as the first image.

[0012] In some embodiments of the present disclosure, based on the aforementioned scheme, the image overlay module includes: an image alignment unit, used to align the first image and the original face image according to the facial feature points; and an image synthesis unit, used to overlay and synthesize the aligned first image with the original face image to obtain the second image.

[0013] In some embodiments of the present disclosure, the first image includes an RGB channel image and an α channel image; based on the aforementioned scheme, the image synthesis unit is configured to: use the α channel image as a weight, and weightedly sum the RGB channel image and the original face image to obtain the second image.

[0014] In some embodiments of the present disclosure, based on the aforementioned scheme, the image processing device also includes: a sample acquisition module, used to acquire face area image samples and synthetic image samples corresponding to the face area image samples, wherein the synthetic image samples are obtained by processing the target objects in the face area image samples; a model training module, used to train a face image model including a generative network to be trained and a discriminative network to be trained according to the face area image samples and the synthetic image samples, and use the trained generative network to be trained as the generative network model.

[0015] In some embodiments of the present disclosure, based on the aforementioned scheme, the sample acquisition module includes: a feature point acquisition unit, used to acquire face image samples, detect and align the face image samples to acquire face feature points corresponding to the face image samples; a face area image sample acquisition unit, used to determine target feature points from the face feature points, and extract the face area image samples corresponding to different postures according to the target feature points; a region position map acquisition unit, used to extract features of the face area image samples through a region prediction module to acquire a region position map corresponding to the target object; a synthetic image sample acquisition unit, used to acquire a two-dimensional texture image corresponding to the target object, and superimpose the face area image samples, the region position map and the two-dimensional texture image to obtain a synthetic image sample corresponding to the face area image samples.

[0016] In some embodiments of the present disclosure, based on the aforementioned scheme, the face area image sample acquisition unit is configured to: determine the target feature area according to the target feature points; fix the position of the target feature area unchanged, acquire face images corresponding to different postures, and use the face images as the face area image samples.

[0017] In some embodiments of the present disclosure, based on the aforementioned scheme, the area location map acquisition unit is configured to: perform image segmentation on the face area image samples to obtain the target feature points; determine the area corresponding to the target object based on the target feature points and a preset distance, and obtain the area location map based on the area corresponding to the target object.

[0018] In some embodiments of the present disclosure, based on the aforementioned scheme, the model training module includes: an image generation unit to be detected, used to input the face area image sample into the face image model, perform feature extraction on the face area image sample through the generation network to be trained, and generate an image to be detected containing the optimized target object based on the extracted features; a reality acquisition unit, used to input the image to be detected and the synthetic image sample into the discriminant network to be trained, perform feature extraction and feature comparison on the image to be detected and the synthetic image sample through the discriminant network to be trained to obtain the reality of the image to be detected; a parameter adjustment unit, used to adjust the parameters of the face image model according to the reality to make the face image model converge.

[0019] In some embodiments of the present disclosure, based on the above scheme, the image generation unit to be detected is configured to: extract features from the face area image sample, and generate RGB channel image samples and α channel image samples based on the extracted features; superimpose the α channel image samples and the RGB channel image samples to obtain the image to be detected.

[0020] In some embodiments of the present disclosure, based on the aforementioned scheme, the image acquisition module is configured to: use the received single-frame face image as the original face image; or, extract the face information in each image frame contained in the received video to obtain the original face image.

[0021] According to one aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the image processing method based on artificial intelligence as described in the above embodiment is implemented.

[0022] According to one aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising one or more processors; and a storage device for storing one or more programs, which, when the one or more programs are executed by the one or more processors, enables the one or more processors to execute the artificial intelligence-based image processing method as described in the above embodiment.

[0023] In the technical solutions provided in some embodiments of the present disclosure, firstly, a face region image is obtained by extracting features from an original face image; then, the face region image is input into a generative network model, and features are extracted from the face region image through the generative network model, and a first image is generated based on the extracted features; the features of the target object in the first image are different from the features of the target object in the original face image; finally, the first image is aligned and superimposed with the original face image to obtain a second image. The technical solution of the present disclosure can process the received face image in real time, change the target object therein, and can also keep the resolution of the second image the same as that of the original face image, thereby improving the image quality and thus improving the user experience.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0026] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of the embodiments of the present disclosure can be applied;

[0027] Figure 2 A-2B schematically shows an interface diagram of generating long hair by non-real-time single-frame image processing in the related art;

[0028] Figure 3 A-3C schematically shows an interface diagram of real-time generation of two-dimensional hair in the related art;

[0029] Figure 4 A-4C schematically shows an interface diagram of real-time generation of two-dimensional hair in the related art;

[0030] Figure 5 A-5C schematically shows an interface diagram of real-time generation of three-dimensional hair in the related art;

[0031] Figure 6 The following schematically shows a flow chart of an image processing method based on artificial intelligence according to an embodiment of the present disclosure;

[0032] Figure 7 The following schematically shows a process flow diagram of obtaining a face region image according to an embodiment of the present disclosure;

[0033] Figure 8 The following schematically shows a flow chart of obtaining face area image samples and synthesized image samples according to an embodiment of the present disclosure;

[0034] Fig. 9 A-9F schematically shows an interface diagram of long hair composite image samples in different postures according to an embodiment of the present disclosure;

[0035] Fig.10 The following schematically shows a flow chart of training a face image model according to an embodiment of the present disclosure;

[0036] Fig.11 The following schematically shows a flow chart of generating a facial image containing long hair according to an embodiment of the present disclosure;

[0037] Fig.12 A block diagram of an image processing device based on artificial intelligence according to an embodiment of the present disclosure is schematically shown;

[0038] Fig.13 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0039] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete and will fully convey the concept of the example embodiments to those skilled in the art.

[0040] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0041] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0042] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0043] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of the embodiments of the present disclosure can be applied is shown.

[0044] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as a wired communication link, a wireless communication link, and the like.

[0045] It should be understood that Figure 1 The number of terminal devices, networks and servers in the example is only illustrative. According to actual needs, any number of terminal devices, networks and servers may be provided. For example, the server 103 may be a server cluster composed of multiple servers. The terminal device 101 may be a shooting device with an imaging unit such as a video camera, a still camera, or a smart phone, and an original face image containing a target object may be obtained through the terminal device 101.

[0046] In one embodiment of the present disclosure, after the terminal device 101 obtains the original facial image containing the target object, the original facial image can be sent to the server 103 through the network 102. When the server 103 obtains the original facial image, face detection and registration can be performed on the original facial image to obtain facial feature points; then, based on the facial feature points, a facial region image with all postures aligned can be calculated and extracted; then, features of the facial region image are extracted through a trained generative network model, and a first image is generated based on the extracted features. The first image is an image generated after the target object in the facial region image is transformed, such as changing the short hair in the facial region image to long hair, changing the small eyes to big eyes, etc.; finally, the first image with the transformed target object is aligned and superimposed with the original facial image to obtain a second image. The second image is a facial image with the transformed target object and the original resolution is maintained. Furthermore, in the embodiments of the present disclosure, after obtaining the original face image, the facial makeup in the original face image can be transformed, for example, male makeup can be transformed into female makeup, female makeup can be transformed into male makeup, and so on, and then the above operation can be performed on the original face image after the facial makeup is transformed to obtain a second image in which the facial makeup and the target object are changed and the original resolution is maintained. The technical solution of the embodiments of the present disclosure can simulate and generate a three-dimensional effect for the target object in the received original face image in real time, and as the face angle changes, the three-dimensional effect of the target object also changes accordingly; at the same time, images with various shapes and makeup can be obtained while maintaining the original resolution, further improving the user experience.

[0047] It should be noted that the image processing method based on artificial intelligence provided in the embodiment of the present disclosure is generally executed by a server, and accordingly, the image processing device based on artificial intelligence is generally set in the server. However, in other embodiments of the present disclosure, the image processing method based on artificial intelligence provided in the embodiment of the present disclosure may also be executed by a terminal device.

[0048] In the related art in this field, taking the real-time generation of hair from a face image as an example, there are three methods: (1) generating long hair by non-real-time single-frame image processing; (2) generating two-dimensional hair in real time; (3) generating three-dimensional hair in real time. Specifically:

[0049] Non-real-time single-frame image processing to generate long hair can obtain a more realistic three-dimensional effect. Figure 2 A-2B shows a schematic diagram of the interface for generating long hair by non-real-time single-frame image processing, such as Figure 2 A shows the original face image with short hair. After processing, the short hair is transformed into long hair with a three-dimensional effect. Figure 2 As shown in B, this method can only process a single frame, cannot simulate in real time, and the simulation efficiency is very low.

[0050] Although the method of generating two-dimensional hair in real time can achieve real-time processing, it can only generate two-dimensional hair, not three-dimensional hair, and the effect of side face is poor. Figure 3 A-3C and Figure 4 A-4C shows a schematic diagram of the interface for generating two-dimensional hair in real time. Figure 3 A-3B and Figure 4 As shown in A-4C, Figure 3 A and Figure 4 A is the long hair effect generated when the face is turned right. It can be seen that the long hair does not fit the contour of the face and the effect is poor; Figure 3 B and Figure 4 B is the long hair effect generated when the face is front-facing. It can be seen that the long hair fits the face contour well. Figure 3 C and Figure 4 C is the long hair effect generated when the face is turned left, which is the same as the long hair effect when the face is turned right. The long hair does not fit the facial contour and the effect is poor.

[0051] The method of generating three-dimensional hair in real time can generate three-dimensional effects in real time and can also achieve facial gender transformation, but the hair color in the generated image is single. Figure 5 A-5C shows a schematic diagram of an interface for generating three-dimensional hair in real time. Figure 5 As shown in A-5C, Figure 5 A is the long hair effect generated when the face is turned right. Figure 5 B is the long hair effect generated when the face is frontal. Figure 5 C is the long hair effect generated when the face is turned left. It can be seen that the generated long hair fits the facial contour more closely and has a three-dimensional effect. However, the hair color is single and can only be the hair color in the original face image, which cannot provide users with multiple choices.

[0052] In addition, the above three methods have a common disadvantage, that is, the corresponding applications are large, usually requiring dozens, tens or even hundreds of megabytes of storage space. For terminal devices with small storage space, it is difficult to install and effectively load and run them. Even if they are installed, the post-processing efficiency is very low.

[0053] In view of the problems existing in the related art, the embodiment of the present disclosure provides an image processing method based on artificial intelligence, which is implemented based on machine learning. Machine learning is a type of artificial intelligence. Artificial Intelligence (AI) is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making.

[0054] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0055] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing so that computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0056] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0057] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0058] The solution provided by the embodiments of the present disclosure relates to an artificial intelligence image processing technology, which is specifically described by the following embodiments:

[0059] The embodiment of the present disclosure first proposes an image processing method based on artificial intelligence. The following takes real-time generation of long hair as an example to elaborate on the implementation details of the technical solution of the embodiment of the present disclosure:

[0060] Figure 6 The flowchart of the image processing method based on artificial intelligence according to an embodiment of the present disclosure is schematically shown. The image processing method based on artificial intelligence can be executed by a server, which can be Figure 1 The server 103 shown in FIG. Figure 6 As shown, the artificial intelligence-based image processing method includes at least steps S610 to S630, which are described in detail as follows:

[0061] In step S610, an original face image is obtained, and feature extraction is performed on the original face image to obtain a face region image.

[0062] In one embodiment of the present disclosure, an original facial image can be obtained through the terminal device 101. The original facial image can be formed by photographing a face containing a target object through the terminal device 101 and imaging the face according to the captured image signal through the imaging unit therein; it can also be a facial image containing the target object downloaded from the network by the terminal device 101; of course, it can also be a facial image containing the target object locally stored by the terminal device 101, etc., wherein the target object can be any one of the five senses, or can be objects such as hair and face shape, which are not specifically limited in the present embodiment. In order to clearly understand the technical solution of the embodiment of the present disclosure, hair is used as the target object for explanation. After obtaining the original facial image sent by the terminal device 101, feature extraction can be performed on the original facial image to obtain a facial area image.

[0063] Figure 7 FIG. 4 shows a schematic diagram of the process of obtaining a face area image, such as Figure 7 As shown, the process of obtaining the face area image includes at least steps S701-S703, specifically:

[0064] In step S701, the face region in the original face image is detected, and feature registration is performed on the face region to determine face feature points.

[0065] In one embodiment of the present disclosure, the original face image can be input into a trained face detection model, which includes a detection sub-model and a registration sub-model, wherein the detection sub-model is used to detect the face area in the original face image, and the registration sub-model is used to perform feature registration on the face area to determine the face feature points. The face feature points can specifically include the center point of the left and right eyes, the left and right corners of the mouth, and the tip of the nose.

[0066] In step S702, the feature area corresponding to the facial feature point is determined according to the position information of the facial feature point.

[0067] In one embodiment of the present disclosure, after determining the facial feature points, the position information of the facial feature points can be obtained, such as the position information of the center points of the left and right eyes, the position information of the left and right corners of the mouth, and the position information of the tip of the nose. According to the position information of the facial feature points, the feature areas corresponding to the facial feature points can be determined. For example, taking the center point (x, y) of the left eye as a reference, (x-10mm, x+10mm) is used as the position information of the two ends of the left eye, and (y-5mm, y+5mm) is used as the position information of the upper and lower vertices of the left eye. Then, according to the position information of the two ends of the left eye and the position information of the upper and lower vertices, the feature area of ​​the left eye can be determined. It is worth noting that different people have different eye sizes. Therefore, when determining the position information of the two ends of the eyes and the upper and lower vertices, values ​​different from 10mm and 5mm can be used. Similarly, according to the position information of the left and right corners of the mouth and the position information of the tip of the nose, the feature area corresponding to the mouth and the feature area corresponding to the nose can be determined.

[0068] In step S703, the feature region is enlarged by a preset multiple, and a face region image is determined based on the enlarged feature region.

[0069] In one embodiment of the present disclosure, after obtaining the feature area corresponding to the facial feature points, the feature area can be enlarged by a preset multiple to obtain a facial area image. For example, the feature area can be enlarged by three times, four times, etc., as long as the entire face and the partial area outside the facial contour are covered.

[0070] In step S620, the face area image is input into the generative network model, features of the face area image are extracted through the generative network model, and a first image is generated based on the extracted features; wherein the features of the target object in the first image are different from the features of the target object in the original face image.

[0071] In one embodiment of the present disclosure, after acquiring a face region image, the face region image can be input into a trained machine learning model to extract features from the face region image, and generate a first image based on the extracted features, wherein the machine learning model can be a generative network model in a generative adversarial network model, and after training the generative adversarial network model, the trained generative network model can be used as the final model for face image processing, which can reduce the size of the model and improve image processing efficiency. Feature extraction is performed on the face region image through the trained generative network model, and a first image with long hair is generated based on the extracted features. It is worth noting that the hair in the original face image or the face area image can be long hair, short hair, or no hair, and the hair in the first image is different from the hair in the original face image or the face area image. For example, the face area image is a face area image of a male with short hair. The generative network model can transform the male's short hair into long hair. Similarly, the generative network model can also transform women's long hair or short hair into short hair or longer long hair. The generated hair can also have various styles, such as curly hair, straight hair, bangs, side bangs, ancient hairstyle with a bun, etc., and the hair can also have any hair color.

[0072] In one embodiment of the present disclosure, when feature extraction is performed on a face area image by generating a network model, feature extraction can be performed on the face area image to obtain contour features and facial features, and based on the extracted contour features and facial features, an RGB channel image and an α channel image corresponding to the face area image are obtained, and the RGB channel image and the α channel image are the first image.

[0073] It is worth noting that in the embodiment of the present disclosure, the input face area image can also be feature extracted by generating a network model to obtain contour features and facial features, and an RGB channel image corresponding to the face area image can be obtained based on the extracted contour features and facial features. The RGB channel image is the first image.

[0074] In one embodiment of the present disclosure, in order to obtain a stable generative network model, it is necessary to train a face image model, which is a generative adversarial network model, which includes a generative network to be trained and a discriminative network to be trained. The trained generative network model is obtained by training the face image model. Generative adversarial network (GAN) is a method of unsupervised learning, which learns by letting two neural networks compete with each other. The generative adversarial network consists of a generative network and a discriminative network. The generative network randomly samples from the latent space as input, and its output results need to imitate the real samples in the training set as much as possible. The input of the discriminative network is the real sample and the output of the generative network. The purpose is to distinguish the output of the generative network from the real sample as much as possible, and the generative network should deceive the discriminative network as much as possible. The two networks compete with each other, constantly adjust the parameters, and finally generate a fake image.

[0075] In one embodiment of the present disclosure, training samples are first obtained, and then the generative adversarial network model is trained using the training samples. The training samples include face region image samples and synthetic image samples corresponding to the face region image samples. The synthetic image samples are obtained by transforming the target object in the face region image samples. For example, the image generated by transforming the hair in the face region image samples is the synthetic image sample. After obtaining the face image samples and the synthetic image samples, the face image model can be trained according to the face image samples and the synthetic image samples, and the trained generative network to be trained is used as the generative network model for subsequent image processing.

[0076] In one embodiment of the present disclosure, when the training data is not paired data and the training data is not aligned, unsupervised GAN training will be more difficult, the required network will be relatively large, and it will not be able to meet the real-time requirements. Therefore, in order to reduce the difficulty of training and improve network efficiency, GAN can be trained by constructing paired training data, and the use of paired training data can control the alignment of background pixel values, so that the generated results are more natural after being integrated into videos and images. Paired real long and short hair data, that is, the long and short hair data of the same person is very difficult to obtain. Usually, the long and short hair data of the same person can be obtained as paired training data by wearing a wig. High-precision three-dimensional modeling can also be used to synthesize three-dimensional hair effects. However, these methods are either too labor-intensive or too computationally expensive and difficult to implement. Therefore, in order to reduce labor and computational costs, the embodiment of the present disclosure uses two-dimensional textures to simulate the rotational changes of three-dimensional hair.

[0077] Figure 8 FIG. 4 shows a flow chart of obtaining face area image samples and synthesized image samples, as shown in FIG. Figure 8 As shown, the process at least includes steps S801-S804, specifically:

[0078] In step S801, a face image sample is obtained, and detection and registration are performed on the face image sample to obtain face feature points corresponding to the face image sample.

[0079] In one embodiment of the present disclosure, similar to step S701, the acquired face image sample can be input into a trained face detection model, the face area in the face image sample can be detected by the detection sub-model in the face detection model, and then the detected face area can be feature aligned by the alignment sub-model in the face detection model to determine the face feature points corresponding to the face image sample.

[0080] In step S802, target feature points are determined from the facial feature points, and facial region image samples corresponding to different postures are extracted based on the target feature points.

[0081] In one embodiment of the present disclosure, relatively rigid points can be selected from the determined facial feature points as target feature points, and a target feature region can be determined based on the target feature points; then the position of the target feature region is fixed unchanged, and facial region images corresponding to different postures are obtained, and the obtained facial region images are used as facial region image samples. For example, the center points of the left and right eyes can be used as target feature points, and the left and right eye regions can be determined based on the center points of the left and right eyes. While ensuring that the positions of the left and right eye regions in the picture remain basically unchanged, facial region image samples in different postures can be extracted. For example, facial region images of different postures such as the front face, the face turned left, the face turned right, and the face raised can be extracted, and the obtained facial region images are used as facial region image samples.

[0082] In step S803, feature extraction is performed on the face region image samples by a region prediction module to obtain a region position map corresponding to the target object.

[0083] In one embodiment of the present disclosure, after obtaining a face region image sample, the face region image sample can be subjected to feature extraction through a region prediction module, thereby obtaining a region location map corresponding to the target object. For example, if the target object is hair, then a long hair region prediction module can be used to extract features from the face region image sample, and a long hair region location map corresponding to the face region image sample is obtained based on the contour features in the face region image sample; if the target object is a nose, then a nose region prediction module can be used to extract features from the face region image sample, and a nose region location map corresponding to the face region image sample is obtained based on the facial features in the face region image sample, and so on. When extracting features from the face region image through a region prediction module to obtain a region location map corresponding to the target object, the following steps can be specifically included: first, the face region image sample is segmented to obtain target feature points; then, the region corresponding to the target object is determined based on the target feature points and a preset distance, and a region location map is obtained based on the region. Specifically, when performing image segmentation on a face region image, facial features in a face region image sample can be obtained, and relatively rigid part feature points can be selected from the facial features as target feature points, such as the center point of the left and right eyes as the target feature point; then the hair region is determined based on the center point of the left and right eyes and the preset distance, and then the hair region position map is determined based on the hair region. The preset distance is a distance set based on the positional relationship between the center point of the eye and the face contour. In addition, the hair region includes multiple small regions, and the distance between each small region and the center point of the eye on the same side does not change with the change of the face angle. However, as the face angle changes, the display properties of these small regions will change accordingly. For example, when the face turns right, as the right turn angle gradually increases, the outer hair on the right side will gradually be hidden to present a realistic three-dimensional long hair effect. It is worth noting that the region position map is essentially a region template where the predicted target object is located. By superimposing the two-dimensional texture on the region template, an image of the target object that has changed can be obtained.

[0084] In step S804, a two-dimensional texture image corresponding to the target object is obtained, and the face area image sample, the area position map and the two-dimensional texture image are superimposed to obtain a composite image sample corresponding to the face area image sample.

[0085] In one embodiment of the present disclosure, Figure 2The 2D texture design can obtain the 2D texture image corresponding to the target object, such as the 2D hair texture image, the 2D nose texture image, etc. After obtaining the 2D texture image, it can be superimposed with the face area image sample and the area position map to obtain the synthetic image sample corresponding to the face area image sample. When superimposing, the 2D texture image, the face area image sample and the area position map need to be aligned according to the target feature points, and then superimposed, so as to ensure the realism of the synthetic image sample; in addition, the 2D texture image is only displayed in the area position map to obtain the face image of the target object that has changed. Taking the real-time generation of long hair as an example, after obtaining the long hair area position map, the two-dimensional hair texture can be designed, and the long hair area position map, the two-dimensional hair texture and the face area image are aligned and superimposed through the texture superposition module to obtain a synthetic image sample with long hair. Furthermore, when designing a two-dimensional hair texture image covering the entire image, the outer transparency can also be designed. The outer transparency can increase the three-dimensional sense and realism of the hair. Accordingly, when aligning and superimposing, the long hair area position map, the two-dimensional hair texture, the outer transparency and the face area image can be input into the texture superposition module, and the three-dimensional long hair synthetic image sample can be obtained through image superposition. Accordingly, by performing the above operation on the face area image of each posture, a long hair synthetic image sample with a three-dimensional rotation effect can be obtained. Fig. 9 A-9F shows a schematic diagram of the interface of long hair synthetic image samples in different postures, such as Fig. 9 As shown in A-9F, each image shows the long hair effect under different facial postures. The generated long hair presents a three-dimensional rotation effect as the head rotates, and the edges and ends of the hair appear more three-dimensional and realistic due to the effect of outer transparency.

[0086] In one embodiment of the present disclosure, when generating a two-dimensional hair texture image, two-dimensional hair textures with different colors and / or hairstyles can be generated to form multiple groups of paired training data with different colors and / or hairstyles, and the generative network model trained according to the paired training data with different colors and / or hairstyles can generate hair with multiple colors and / or hairstyles when performing image processing on a new facial image, thereby further increasing the types of special effects that can be generated by the model and improving the user experience.

[0087] In one embodiment of the present disclosure, after obtaining face area image samples and synthetic image samples, a face image model including a generative network to be trained and a discriminative network to be trained can be trained based on the face area image samples and the synthetic image samples. Fig.10 A schematic diagram of the process of training a face image model is shown, such as Fig.10As shown, a face area image sample is input into a generative network 1001 to be trained, and features of the face area image sample are extracted through the generative network 1001 to be trained, and an image to be detected containing a transformed target object is generated based on the extracted features. For example, the input face area image sample is a short-haired male, and the generated image to be detected is a long-haired male; then the image to be detected and a synthetic image sample corresponding to the face area image sample are input into a discriminative network 1002 to be trained, and features of the image to be detected and the synthetic image sample are extracted and compared through the discriminative network 1002 to be trained to obtain the authenticity of the image to be detected; finally, the parameters of the face image model are adjusted according to the authenticity to make the face image model converge.

[0088] In one embodiment of the present disclosure, when extracting features from a face region image sample through a generative network to be trained, RGB channel image samples and α channel image samples can be generated, and the image to be detected can be obtained by superimposing the RGB channel image samples and the α channel image samples. Of course, only RGB channel image samples can be generated through the generative network to be trained, and the RGB channel image samples are the images to be detected, but only using RGB channel image samples will result in low resolution accuracy of the images to be detected, which in turn leads to low resolution of the face images generated by the trained generative network model and poor user experience. By training the model based on the image to be detected generated by superimposing the RGB channel image samples and the α channel image samples, the model no longer needs to learn to generate irrelevant parts such as the background of the picture, which reduces the difficulty of training and improves the resolution accuracy of the generated image. Furthermore, the face image sample can also be directly input as an input image into the generative network to be trained, so that the background and face resolution of the generated image to be detected can be kept consistent with the input face image sample.

[0089] In one embodiment of the present disclosure, after the training of the face image model is completed, the trained generative network to be trained can be used as a generative network model to process the subsequently received face images to obtain the face images containing the transformed target object. Only the generative network model is used for image processing, which avoids the use of models of hundreds of megabytes or even several gigabytes in size for image processing, thereby improving the efficiency of image processing. The generative network model in the present disclosure is only 1.4M in size, has a high computing speed, and can achieve a real-time processing efficiency of 20 frames per second (20FPS) on the mobile terminal. Even for mid- and low-end mobile terminals, it can achieve a real-time processing efficiency of 10FPS.

[0090] In step S630, the first image is aligned and superimposed with the original face image to obtain a second image.

[0091] In one embodiment of the present disclosure, after generating the first image, in order to maintain the original resolution, the first image may be aligned and superimposed with the original face image to obtain a second image. During the alignment and superposition, the alignment and superposition may be performed based on relatively rigid target feature points among the face feature points, or based on all the face feature points. If the first image includes an RGB channel image and an α channel image, when the aligned first image is superimposed and synthesized with the original face image, the α channel image may be used as a weight to perform weighted summation of the RGB channel image and the original face image to obtain the second image. Specifically, formula (1) may be used for weighted summation:

[0092] I out =I RGB ×α+I in ×(1-α) (1)

[0093] Among them, I out is the second image, I RGB is the RGB channel image, I in is the original face image, and α is the α channel image.

[0094] It is worth noting that if the first image is only an RGB image, it is only necessary to align and superimpose the RGB channel image with the original face image. Of course, weights can also be set to perform weighted superposition of the two. The weights can be set according to actual needs, and the embodiments of the present disclosure do not make specific limitations on this.

[0095] In one embodiment of the present disclosure, the original facial image can be a single-frame facial image, or it can be a facial image obtained by extracting facial information from each image frame in a received video. That is to say, the technical solution of the present disclosure can perform real-time processing on a single-frame facial image to obtain a facial image in which the target object has changed, and it can also perform real-time processing on continuous facial images in a video to obtain a facial image in which the target object has changed.

[0096] In one embodiment of the present disclosure, taking the simulation of generating long hair for a short-haired male as an example, Fig.11 FIG. 4 shows a flow chart of generating a face image containing long hair, such as Fig.11As shown, in step S1101, a facial image or video frame of a short-haired male is obtained; in step S1102, the face in the facial image or video frame is detected and registered to obtain facial feature points; in step S1103, facial region images aligned in various postures are extracted based on the facial feature points; in step S1104, the facial region image is input into a generative network model to obtain an RGB channel image and an α channel image, wherein the RGB channel image and the α channel image are both channel images containing long hair; in step S1105, the RGB channel image, the α channel image and the facial image or video frame are synthesized by a synthesis module to form a facial image or video frame of a long-haired male.

[0097] In one embodiment of the present disclosure, when using image processing software, the user may not only be limited to changing a certain feature such as hair, eyes, nose, etc., but may also want to change facial makeup, such as changing a male facial makeup to a female facial makeup, changing a female facial makeup without makeup to a facial makeup with heavy makeup, etc. In view of this, in the technical solution of the embodiment of the present disclosure, after obtaining a single frame of facial image or video frame, the facial makeup of the facial image therein can be changed, and then the facial image after the facial makeup is changed can be processed as follows: Figure 8 The operations shown are performed to obtain a facial image with long hair corresponding to the facial image after the facial makeup is changed; then a facial image model can be trained according to the facial image after the facial makeup is changed and the facial image with long hair corresponding to the facial image after the facial makeup is changed to obtain a trained generative network model, and then the trained generative network model can be used to generate a facial image with changed facial makeup and long hair corresponding to the new facial image after receiving a new facial image.

[0098] In one embodiment of the present disclosure, in addition to simulating the hair in a face image in real time, the eyes, nose, mouth, ears and other parts in the face image can also be simulated, for example, single eyelids can be converted into double eyelids, flat noses can be converted into hooked noses, ears can be converted into rabbit ears, etc., as long as the corresponding area prediction module is used to predict the posture-aligned face area image to obtain the area position map of the corresponding part, and a two-dimensional texture image corresponding to the part to be changed is designed, and then the two-dimensional texture image, the area position map and the face area image are superimposed by the texture superposition module to obtain a synthetic face image corresponding to the input face image, and the face image / face area image and the synthetic face image constitute the paired training data. Then, the generative adversarial network model is trained according to the paired training data to obtain a generative network model, and the generative network model can perform corresponding transformations on the relevant parts in the received face image to generate a face image with transformed features of the relevant parts.

[0099] On the one hand, the artificial intelligence-based image processing method in the embodiment of the present disclosure can process facial images in real time to simulate a three-dimensional effect. Even when the face angle changes, the transformed target object can be rotated accordingly. On the other hand, by aligning and superimposing the image generated by the generative network model with the original face image, the resolution of the face and background can be maintained, thereby improving the quality of the generated image. On the other hand, the model in the embodiment of the present disclosure is only a generative network model in the generative adversarial network model. The model is small and has high image processing efficiency. It can achieve the effect of transforming the target object while recording the video, further improving the user experience.

[0100] The following describes an apparatus embodiment of the present disclosure, which can be used to execute the image processing method based on artificial intelligence in the above-mentioned embodiment of the present disclosure. For details not disclosed in the apparatus embodiment of the present disclosure, please refer to the embodiment of the image processing method based on artificial intelligence in the above-mentioned embodiment of the present disclosure.

[0101] Fig.12 A block diagram of an artificial intelligence-based image processing apparatus according to an embodiment of the present disclosure is schematically shown.

[0102] Reference Fig.12 As shown, an artificial intelligence-based image processing device 1200 according to an embodiment of the present disclosure includes: an image acquisition module 1201, an image processing module 1202 and an image overlay module 1203.

[0103] Among them, the image acquisition module 1201 is used to acquire the original face image, perform feature extraction on the original face image to obtain a face area image; the image processing module 1202 is used to input the face area image into the generative network model, perform feature extraction on the face area image through the generative network model, and generate a first image based on the extracted features; wherein the features of the target object in the first image are different from the features of the target object in the original face image; the image overlay module 1203 is used to align and overlay the first image with the original face image to obtain a second image.

[0104] In one embodiment of the present disclosure, the image acquisition module 1201 is configured to: detect the facial area in the original facial image, and perform feature registration on the facial area to determine the facial feature points; determine the feature area corresponding to the facial feature points based on the position information of the facial feature points; expand the feature area by a preset multiple, and determine the facial area image based on the expanded feature area.

[0105] In one embodiment of the present disclosure, the image processing module 1202 includes: a channel image generation unit, which is used to perform feature extraction on the face area image to obtain contour features and facial features, obtain an RGB channel image and an α channel image based on the contour features and the facial features, and use the RGB channel image and the α channel image as the first image; or, it is used to perform feature extraction on the face area image to obtain contour features and facial features, obtain an RGB channel image based on the contour features and the facial features, and use the RGB channel image as the first image.

[0106] In one embodiment of the present disclosure, the image overlay module 1203 includes: an image alignment unit, used to align the first image and the original face image according to the facial feature points; and an image synthesis unit, used to overlay and synthesize the aligned first image with the original face image to obtain the second image.

[0107] In one embodiment of the present disclosure, the first image includes an RGB channel image and an α channel image; the image synthesis unit is configured to: use the α channel image as a weight, and perform weighted summation on the RGB channel image and the original face image to obtain the second image.

[0108] In one embodiment of the present disclosure, the image processing device 1200 also includes: a sample acquisition module, used to acquire face area image samples and synthetic image samples corresponding to the face area image samples, wherein the synthetic image samples are obtained by processing the target object in the face area image samples; a model training module, used to train a face image model including a generative network to be trained and a discriminative network to be trained according to the face area image samples and the synthetic image samples, and use the trained generative network to be trained as the generative network model.

[0109] In one embodiment of the present disclosure, the sample acquisition module includes: a feature point acquisition unit, which is used to acquire face image samples, detect and align the face image samples to acquire face feature points corresponding to the face image samples; a face area image sample acquisition unit, which is used to determine target feature points from the face feature points, and extract the face area image samples corresponding to different postures according to the target feature points; a region position map acquisition unit, which is used to extract features of the face area image samples through a region prediction module to acquire a region position map corresponding to the target object; a synthetic image sample acquisition unit, which is used to acquire a two-dimensional texture image corresponding to the target object, and superimpose the face area image samples, the region position map and the two-dimensional texture image to obtain a synthetic image sample corresponding to the face area image samples.

[0110] In one embodiment of the present disclosure, the face area image sample acquisition unit is configured to: determine the target feature area based on the target feature points; fix the position of the target feature area unchanged, acquire face images corresponding to different postures, and use the face images as the face area image samples.

[0111] In one embodiment of the present disclosure, the area location map acquisition unit is configured to: perform image segmentation on the face area image sample to obtain the target feature points; determine the area corresponding to the target object based on the target feature points and a preset distance, and obtain the area location map based on the area corresponding to the target object.

[0112] In one embodiment of the present disclosure, the model training module includes: an image generation unit to be detected, used to input the face area image sample into the face image model, perform feature extraction on the face area image sample through the generation network to be trained, and generate an image to be detected containing the optimized target object based on the extracted features; a reality acquisition unit, used to input the image to be detected and the synthetic image sample into the discriminant network to be trained, perform feature extraction and feature comparison on the image to be detected and the synthetic image sample through the discriminant network to be trained to obtain the reality of the image to be detected; a parameter adjustment unit, used to adjust the parameters of the face image model according to the reality to make the face image model converge.

[0113] In one embodiment of the present disclosure, the image generation unit to be detected is configured to: extract features from the face area image sample, and generate RGB channel image samples and α channel image samples based on the extracted features; superimpose the α channel image samples and the RGB channel image samples to obtain the image to be detected.

[0114] In one embodiment of the present disclosure, the image acquisition module 1201 is configured to: use the received single-frame face image as the original face image; or extract the face information in each image frame contained in the received video to obtain the original face image.

[0115] Fig.13 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown.

[0116] It should be noted that Fig.13 The computer system 1300 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0117] like Fig.13As shown, the computer system 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1302 or the program loaded from the storage part 1308 to the random access memory (RAM) 1303 to implement the image annotation method described in the above embodiment. In RAM 1303, various programs and data required for system operation are also stored. CPU 1301, ROM 1302 and RAM 1303 are connected to each other through bus 1304. Input / output (I / O) interface 1305 is also connected to bus 1304.

[0118] The following components are connected to the I / O interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed so that a computer program read therefrom is installed into the storage section 1308 as needed.

[0119] In particular, according to an embodiment of the present disclosure, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1309, and / or installed from a removable medium 1311. When the computer program is executed by a central processing unit (CPU) 1301, various functions defined in the system of the present disclosure are executed.

[0120] It should be noted that the computer-readable medium shown in the embodiment of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0121] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0123] As another aspect, the present disclosure further provides a computer-readable medium, which may be included in the image processing device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0124] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0125] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0126] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0127] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method based on artificial intelligence, characterized in that: include: Acquire a facial image sample, and perform detection and registration on the facial image sample to obtain facial feature points corresponding to the facial image sample; Determine target feature points from the facial feature points, and extract facial region image samples corresponding to different postures according to the target feature points; Performing feature extraction on the face region image sample by a region prediction module to obtain a region position map corresponding to a target object, wherein the target object is hair; Acquire a two-dimensional texture image corresponding to the target object, and superimpose the face area image sample, the area position map, the outer transparency and the two-dimensional texture image through a texture superposition module to obtain a synthetic image sample corresponding to the face area image sample, wherein the synthetic image sample is obtained by processing the target object in the face area image sample; Inputting the face area image sample into a face image model including a generative network to be trained and a discriminative network to be trained, extracting features of the face area image sample through the generative network to be trained, and generating RGB channel image samples and α channel image samples based on the extracted features; Superimposing the α channel image sample and the RGB channel image sample to obtain an image to be detected containing the optimized target object; Inputting the image to be detected and the synthetic image sample into the discriminant network to be trained, and performing feature extraction and feature comparison on the image to be detected and the synthetic image sample through the discriminant network to be trained to obtain the authenticity of the image to be detected; Adjusting the parameters of the face image model according to the authenticity to make the face image model converge, and using the trained generative network to be trained as a generative network model; Acquire an original face image, and perform feature extraction on the original face image to acquire a face region image; Inputting the face region image into the generative network model, performing feature extraction on the face region image through the generative network model to obtain contour features and facial features, obtaining an RGB channel image and an α channel image based on the contour features and the facial features, and using the RGB channel image and the α channel image as a first image; wherein the hair style in the first image is different from the hair style in the original face image; The first image is aligned and superimposed with the original face image to obtain a second image, so as to achieve an effect of transforming the target object while recording the video.

2. The image processing method based on artificial intelligence according to claim 1, characterized in that: The step of extracting features from the original face image to obtain a face region image comprises: Detecting a face region in the original face image, and performing feature registration on the face region to determine face feature points; Determine the feature area corresponding to the facial feature point according to the position information of the facial feature point; The characteristic region is enlarged by a preset multiple, and the face region image is determined according to the enlarged characteristic region.

3. The image processing method based on artificial intelligence according to claim 2, characterized in that: The step of aligning and superimposing the first image with the original face image to obtain a second image includes: Aligning the first image with the original face image according to the face feature points; The aligned first image is superimposed and synthesized with the original face image to obtain the second image.

4. The image processing method based on artificial intelligence according to claim 3, characterized in that: The first image includes an RGB channel image and an α channel image; The step of superimposing and synthesizing the aligned first image and the original face image to obtain the second image includes: The alpha channel image is used as a weight, and weighted sum is performed on the RGB channel image and the original face image to obtain the second image.

5. The image processing method based on artificial intelligence according to claim 1, characterized in that: The extracting of face area image samples corresponding to different postures according to the target feature points includes: Determine a target feature area according to the target feature points; The position of the target feature area is fixed, and face images corresponding to different postures are obtained, and the face images are used as the face area image samples.

6. The image processing method based on artificial intelligence according to claim 1, characterized in that: The extracting features of the face region image samples by the region prediction module to obtain a region position map corresponding to the target object includes: Performing image segmentation on the face area image sample to obtain the target feature points; An area corresponding to the target object is determined according to the target feature points and a preset distance, and the area position map is obtained according to the area corresponding to the target object.

7. The image processing method based on artificial intelligence according to claim 1, characterized in that: The obtaining of the original face image comprises: Using the received single-frame face image as the original face image; or, The face information in each image frame contained in the received video is extracted to obtain the original face image.

8. An image processing device based on artificial intelligence, characterized in that: include: A feature point acquisition unit, used to acquire a face image sample, and perform detection and registration on the face image sample to acquire face feature points corresponding to the face image sample; A face area image sample acquisition unit, used to determine target feature points from the face feature points, and extract face area image samples corresponding to different postures according to the target feature points; A region position map acquisition unit, used for performing feature extraction on the face region image sample through a region prediction module to acquire a region position map corresponding to a target object, wherein the target object is hair; a synthetic image sample acquisition unit, configured to acquire a two-dimensional texture image corresponding to the target object, and to superimpose the face region image sample, the region position map, the outer transparency and the two-dimensional texture image through a texture superposition module to obtain a synthetic image sample corresponding to the face region image sample, wherein the synthetic image sample is obtained by processing the target object in the face region image sample; A model training module, used for training a face image model including a to-be-trained generative network and a to-be-trained discriminative network according to the face region image samples and the synthetic image samples, and using the trained to-be-trained generative network as the generative network model; The model training module includes: a to-be-detected image generation unit, used to input the face area image sample into the face image model, perform feature extraction on the face area image sample through the to-be-trained generation network, and generate RGB channel image samples and α channel image samples based on the extracted features; superimpose the α channel image samples and the RGB channel image samples to obtain the to-be-detected image containing the optimized target object; a truthfulness acquisition unit, used to input the to-be-detected image and the synthetic image sample into the to-be-trained discriminant network, perform feature extraction and feature comparison on the to-be-detected image and the synthetic image sample through the to-be-trained discriminant network to obtain the truthfulness of the to-be-detected image; a parameter adjustment unit, used to adjust the parameters of the face image model according to the truthfulness to make the face image model converge; An image acquisition module is used to acquire an original face image and perform feature extraction on the original face image to acquire a face region image; An image processing module, configured to input the face region image into the generative network model, perform feature extraction on the face region image through the generative network model to obtain contour features and facial features, obtain an RGB channel image and an α channel image based on the contour features and the facial features, and use the RGB channel image and the α channel image as a first image; wherein the hair style in the first image is different from the hair style in the original face image; An image superposition module is used to align and superimpose the first image with the original face image to obtain a second image, so as to achieve the effect of transforming the target object while recording the video.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the artificial intelligence-based image processing method as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to execute the artificial intelligence-based image processing method as described in any one of claims 1 to 7.

Images