Image processing method, device, storage medium and electronic device

Abstract

The embodiment of the present invention discloses an image processing method, device, storage medium and electronic device, wherein the method comprises: obtaining an image to be processed, extracting the current attribute features, shape features, texture features and target attribute features of the image to be processed; inputting the current attribute features, shape features and target attribute features into a pre-trained shape migration model to generate shape features corresponding to the target attribute; inputting the current attribute features, texture features and target attribute features into a pre-trained texture migration model to generate texture features corresponding to the target attribute; performing feature fusion based on the texture features and shape features corresponding to the target attribute to obtain a target image corresponding to the target attribute. Transfer learning in two dimensions, shape features and texture features, improves the accuracy of transfer learning of the image to be processed and the quality of the target image.

Description

Technical Field

[0001] Embodiments of the present invention relate to autonomous driving technology, and more particularly to an image processing method, device, storage medium, and electronic device. Background Art

[0002] In recent years, deep learning has achieved very good results in many fields, from image classification, speech recognition to natural language processing, etc. At present, deep learning is widely used in the field of image processing.

[0003] However, in the process of implementing the present invention, the inventors found that there are at least the following technical problems in the prior art: in image transfer learning, the same type of transfer adopts a common mode, the individual differences of the learned images are small, and the image transfer accuracy is low. Summary of the invention

[0004] The embodiments of the present invention provide an image processing method, an apparatus, a storage medium and an electronic device to achieve image accuracy.

[0005] In a first aspect, an embodiment of the present invention provides an image processing method, including:

[0006] Acquire an image to be processed, and extract current attribute features, shape features, texture features of the image to be processed, and target attribute features of the image to be processed;

[0007] Inputting the current attribute feature, the shape feature and the target attribute feature into a pre-trained shape transfer model to generate a shape feature corresponding to the target attribute;

[0008] Inputting the current attribute feature, the texture feature and the target attribute feature into a pre-trained texture migration model to generate a texture feature corresponding to the target attribute;

[0009] Feature fusion is performed based on the texture feature corresponding to the target attribute and the shape feature to obtain a target image corresponding to the target attribute.

[0010] In a second aspect, an embodiment of the present invention further provides an image processing method:

[0011] A feature extraction module is used to obtain an image to be processed, and extract current attribute features, shape features, texture features of the image to be processed, and target attribute features of the image to be processed;

[0012] A shape feature migration module, used for inputting the current attribute feature, the shape feature and the target attribute feature into a pre-trained shape migration model to generate a shape feature corresponding to the target attribute;

[0013] A texture feature migration module, used for inputting the current attribute feature, the texture feature and the target attribute feature into a pre-trained texture migration model to generate a texture feature corresponding to the target attribute;

[0014] The target image generation module is used to perform feature fusion based on the texture feature corresponding to the target attribute and the shape feature to obtain a target image corresponding to the target attribute.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an image processing method as provided in any embodiment of the present invention is implemented.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an image processing method as provided in any embodiment of the present invention.

[0017] The technical solution provided by the embodiment of the present invention decouples the image to be processed in two dimensions of shape and texture by setting a shape transfer model and a texture transfer model, and performs transfer learning on the shape features and texture features respectively based on the set shape transfer model and texture transfer model, thereby realizing independent transfer learning of the shape features and texture features of the image to be processed, improving the transfer learning accuracy of features in each dimension, and at the same time, the transfer learning in two dimensions of shape features and texture features improves the comprehensiveness of transfer learning of the image to be processed, thereby further improving the accuracy of transfer learning of the image to be processed and the quality of the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a flow chart of an image processing method provided in Embodiment 1 of the present invention;

[0019] Figure 2 is a structural schematic diagram of a shape migration model or a texture migration model in an embodiment of the present invention;

[0020] Figure 3 is a structural diagram of an attribute migration module provided by an embodiment of the present invention;

[0021] Figure 4 is a flowchart of an image processing method provided by Embodiment 2 of the present invention;

[0022] Figure 5 is a structural schematic diagram of a texture migration model provided by an embodiment of the present invention;

[0023] Figure 6 is a schematic diagram of the structure of an attribute migration module provided in an embodiment of the present invention;

[0024] Figure 7 is a structural schematic diagram of an image processing device provided by an embodiment of the present invention;

[0025] Figure 8 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0027] Embodiment 1

[0028] Figure 1 This is a flow chart of an image processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of image transfer learning. The method can be executed by the image processing device provided in the embodiment of the invention. The control device can be implemented by software and / or hardware. The control device can be configured on electronic devices such as mobile phones, tablet computers, and computers. Specifically, the method includes the following steps:

[0029] S110, acquiring an image to be processed, and extracting current attribute features, shape features, texture features of the image to be processed, and target attribute features of the image to be processed.

[0030] S120: Input the current attribute feature, the shape feature, and the target attribute feature into a pre-trained shape transfer model to generate a shape feature corresponding to the target attribute.

[0031] S130: Input the current attribute feature, the texture feature and the target attribute feature into a pre-trained texture migration model to generate a texture feature corresponding to the target attribute.

[0032] S140: Perform feature fusion based on the texture feature and the shape feature corresponding to the target attribute to obtain a target image corresponding to the target attribute.

[0033] In this embodiment, attribute transfer processing is performed on the image to be processed through the shape transfer model and the texture transfer model, wherein the attribute can be the attribute of the image to be processed, such as image style, etc., or it can be the attribute of the image content in the image to be processed. For example, if the image content in the image to be processed is a person, the attributes include but are not limited to age, gender, etc.

[0034] In this embodiment, a shape transfer model and a texture transfer model are set, and transfer learning is performed on the image to be processed in terms of both shape features and texture features to obtain a target image after transfer learning. The image to be processed is processed from both aspects of shape features and texture features to provide comprehensiveness of transfer learning and avoid the problems of poor transfer learning quality and low image accuracy caused by single feature transfer.

[0035] Optionally, the image to be processed is a three-dimensional image. Feature extraction is performed on the image to be processed to extract the current attribute features, shape features and texture features in the image to be processed. In some embodiments, the shape feature uv map and texture feature uv texture map of the image to be processed can be obtained by a preset feature extraction model, such as a shape feature extraction model and a texture feature extraction model. The shape feature extraction model and the texture feature extraction model can be, for example, a machine learning model, specifically a neural network model, etc. The specific structure of the shape feature extraction model and the texture feature extraction model is not limited, and the extraction of shape features and texture features can be realized. In some embodiments, obtaining the shape feature and texture feature of the image to be processed includes: analyzing the image to be processed through a preset normalized coordinate coding prediction network to obtain normalized coordinate coding atlas information, searching for UV space coordinate information corresponding to the current normalized coordinate coding atlas information from the correspondence between the preset reference normalized coordinate coding atlas information and the reference UV space coordinate information, and obtaining the shape feature uv map; mapping the current normalized coordinate coding atlas information to the UV space coordinate information in sequence to form UV space texture mapping information, that is, texture feature uv texture map.

[0036] The current attribute features of the image to be processed are exclusive features of the image to be processed, and are used to improve the differences of the image to be processed in the transfer learning process. Among them, the current attribute features of the image to be processed can be obtained based on a preset attribute feature extraction model, and the attribute feature extraction model can be set according to the attribute requirements. Exemplarily, the attribute is age, and the attribute feature extraction model can be an age estimation model, which can identify the age features of the person in the image to be processed. The age feature can be a one-hot vector, that is, a vector of length n. Exemplarily, the attribute is gender, and the attribute feature extraction model can be a gender recognition model, which can recognize the gender features of the person in the image to be processed. Exemplarily, the attribute is image style, and the attribute feature extraction model can be a style recognition model, which can recognize the image style of the image to be processed.

[0037] The target attribute feature of the image to be processed is the transfer learning target of the image to be processed. For example, if the attribute is age, the target attribute feature is used to characterize the target age of transfer learning. If the attribute is gender, the target attribute feature is used to characterize the target gender of transfer learning. If the attribute is image style, the target attribute feature is used to characterize the target image style of transfer learning. Taking age as an example, the target attribute feature can be a one-hot vector of length n, where n represents the number of age groups. If the person in the image to be processed belongs to the i-th age group, the i-th dimension of the vector is 1, and the remaining dimensions are 0, which represents a general age feature. It should be noted that for other attributes, the expression form of the target attribute feature can be determined according to the representation requirements of each attribute feature, which is not limited here.

[0038] In this embodiment, a shape transfer model and a texture transfer model are respectively set to perform transfer learning on the shape features and texture features of the image to be processed. The shape transfer model and the texture transfer model can be neural network models, and the network structures of the two can be the same or different, and are obtained by training respectively. In some embodiments, the shape transfer model or the texture transfer model can be but not limited to a GAN (Generative Adversarial Networks) network or a VAE (variational autoencode).

[0039] For transfer learning of shape features, the current attribute features of the image to be processed, the shape features and the target attribute features are used as input information of the shape transfer model, and the shape transfer model performs transfer learning on the above input information to obtain the shape features corresponding to the target attribute. Similarly, for transfer learning of texture features, the current attribute features of the image to be processed, the texture features and the target attribute features are used as input information of the texture transfer model, and the texture transfer model performs transfer learning on the above input information to obtain the texture features corresponding to the target attribute.

[0040] The texture features and shape features learned separately are synthesized to obtain a target image. Optionally, the texture features and shape features may be synthesized and rendered by a render synthesis method to obtain the target image.

[0041] In this embodiment, by setting a shape transfer model and a texture transfer model, the image to be processed is decoupled in two dimensions of shape and texture, and transfer learning is performed on the shape features and texture features respectively based on the set shape transfer model and texture transfer model, thereby realizing independent transfer learning of the distribution of shape features and texture features of the image to be processed, and improving the transfer learning accuracy of features in each dimension. At the same time, the transfer learning in two dimensions of shape features and texture features improves the comprehensiveness of transfer learning of the image to be processed, and further improves the accuracy of transfer learning of the image to be processed and the quality of the target image.

[0042] Based on the above embodiment, the shape transfer model and the texture transfer model have the same network structure. Figure 2 , Figure 2 : is a structural diagram of a shape migration model or a texture migration model in an embodiment of the present invention. For the shape migration model or the texture migration model, the model includes a feature extraction module and at least one attribute migration module, the attribute migration module generates a current output feature based on the output feature of the feature extraction module or the output feature of the previous attribute migration module, the current attribute feature, and the target attribute feature, and inputs the current output feature to the next attribute migration module. It should be noted that Figure 2 This is just an example diagram. Figure 2 The number of attribute transfer modules in the method can be 3 or other numbers, which can be set according to user needs. Optionally, the feature extraction module and the attribute transfer module can include at least one convolution layer connected in sequence, or a preset number of convolution layers and other network blocks. The input feature can be a shape feature or a texture feature. In some embodiments, the output feature of the last attribute transfer module can be determined as the shape feature or texture feature corresponding to the target attribute. Figure 2 The structure shown is connected to a convolutional block or a fully connected layer, etc., but is not limited to this.

[0043] Based on the above embodiments, the structures of the attribute migration modules can be the same. For example, see Figure 3 , Figure 3It is a structural diagram of an attribute migration module provided by an embodiment of the present invention. Any attribute migration module includes: a first convolutional layer, a first style migration network, a second style migration network and a second convolutional layer; wherein, the first convolutional layer is used to perform a convolution operation on the input feature, and input the convolutional feature into the first style migration network and the second style migration network respectively; the first style migration network performs a migration operation on the current attribute feature and the convolutional feature to generate a first migration feature, and the second style migration network performs a migration operation on the target attribute feature and the convolutional feature to generate a second migration feature; the second convolutional layer performs a dimensionality reduction process on the spliced ​​convolutional feature, the first migration feature and the second migration feature to obtain the current output feature, so that the dimension of the current output feature is consistent with the dimension of the input feature.

[0044] In this embodiment, the first style transfer network and the second style transfer network can be AdaIN (Adaptive Instance Normalization) networks. The current attribute features and the target attribute features are processed by AdaIN to obtain the first transfer features and the second transfer features. Furthermore, the second convolution layer performs convolution processing after splicing and dimensionality reduction processing on the convolution features, the first transfer features and the second transfer features. In the process of transfer learning, the exclusive attribute features of the image to be processed and the target attribute features are integrated to avoid the problem of stereotyped images obtained by transfer learning, highlight the exclusive features of the image to be processed, and improve the quality of the target image.

[0045] On the basis of the above embodiments, for the shape transfer model and the texture transfer model, a first parameter conversion module and a second parameter conversion module are respectively provided, the first parameter conversion module is used to convert the current attribute feature into a first style transfer parameter, and output it to the first style transfer network, and the second parameter conversion module is used to convert the target attribute feature into a second style transfer parameter, and input it to the second style transfer network. In some embodiments, the first parameter conversion module and the second parameter conversion module can be a fully connected layer respectively. Taking the first style transfer network and the second style transfer network as AdaIN networks as an example, the first style transfer parameter and the second style transfer parameter can be the affine transformation parameters β and γ required by the AdaIN network. In other embodiments, the first style transfer parameter and the second style transfer parameter can be set according to the network type of the first style transfer network and the second style transfer network.

[0046] On the basis of the above embodiment, the training method of the shape transfer model is to obtain a sample image and a target image corresponding to the target attribute of the sample image, respectively extract the shape features of the sample image and the target image based on the shape feature extraction model, input the shape features, current attribute features and target attribute features of the sample image into the shape transfer model to be trained, obtain the training shape features, determine the loss function based on the training shape features and the shape features of the target image, and adjust the network parameters of the shape transfer model to be trained based on the loss function. The above training process is iteratively performed until the training conditions are met, such as the number of iterations or the accuracy requirements are met, and it is determined that the training of the shape transfer model is completed.

[0047] Similarly, the training method of the texture transfer model is to obtain a sample image and a target image corresponding to the target attribute of the sample image, extract the texture features of the sample image and the target image respectively based on the texture feature extraction model, input the texture feature current attribute feature and the target attribute feature of the sample image into the texture transfer model to be trained, obtain the training texture feature, determine the loss function based on the texture transfer model and the texture feature of the target image, and adjust the network parameters of the texture transfer model to be trained based on the loss function. The above training process is iterated until the training conditions are met, such as the number of iterations or the accuracy requirements are met, and the texture transfer model training is determined to be completed.

[0048] The texture transfer model and shape transfer model provided in this embodiment perform transfer learning on texture features and shape features respectively. The current attribute features and target attribute features of the image to be processed are input during transfer learning, and the exclusive attribute features and target attribute features of the image to be processed are integrated, thereby avoiding the problem of stereotyped images obtained by transfer learning, highlighting the exclusive features of the image to be processed, and improving the quality of the target image.

[0049] Embodiment 2

[0050] Figure 4 : is a flow chart of an image processing method provided in Embodiment 2 of the present invention, which is refined on the basis of the above embodiment. Optionally, the image to be processed is a face image, and the attribute includes an age attribute. The method specifically includes:

[0051] S210, acquiring an initial image, performing face segmentation on the initial image, and obtaining a face image and a background image in the initial image.

[0052] S220: Extract current attribute features, shape features, texture features, and target attribute features of the face image.

[0053] S230: Input the current attribute feature, the shape feature and the target attribute feature into a pre-trained shape transfer model to generate a shape feature corresponding to the target attribute.

[0054] S240: Input the current attribute feature, the texture feature and the target attribute feature into a pre-trained texture migration model to generate a texture feature corresponding to the target attribute.

[0055] S250: Input the current attribute feature, the texture feature and the target attribute feature into a pre-trained texture migration model to generate a texture feature corresponding to the target attribute.

[0056] S260: Synthesize the target image and the background image.

[0057] In this embodiment, a transfer learning operation is performed on the age attribute of the facial image. After obtaining the initial image, the facial area in the initial image is identified, and the identified facial area is segmented to obtain a facial image. A transfer learning operation of the age attribute is performed on the facial image to avoid interference and invalid processing of the background image during the transfer learning operation.

[0058] For the age attribute, correspondingly, the current attribute feature is the current age feature, and the target attribute feature is the target age feature, wherein the current age feature is obtained by processing the face image to be processed based on the age estimation model. The target age feature is set according to the target age of transfer learning. For example, the target age is 70, and the age range of the target age is 70-79. Accordingly, in the one-hot vector of length n, the data of the age range 70-79 is set to 1, and the others are set to 0. The target age can be any required age, for example, it can be the age of a person younger than the face image, or it can be the age of a person older than the face image. It should be noted that the age range in the target age feature can be set according to demand, and there is no limitation on this.

[0059] Shape transfer learning is performed on face images based on a preset shape transfer model to obtain shape features corresponding to the target age. Texture transfer learning is performed on face images based on a preset texture transfer model to obtain texture features corresponding to the target age. Feature fusion is performed on the texture features and shape features corresponding to the target age to obtain a target face image of the target age.

[0060] The target face image and the background image are synthesized to obtain a composite image including the background image and the target face image. In some embodiments, the background and the target face image may be synthesized based on a refine model. It should be noted that, since the shape features of the face image change during the transfer learning process, for example, the morphological features may become larger or smaller, correspondingly, there is a synthesis difference between the target face image and the background image, resulting in a problem of poor quality of the synthesized image.

[0061] Optionally, synthesizing the target image and the background image includes: adjusting the background image based on the target image; and merging the target image with the adjusted background image. Specifically, according to the shape features of the target image (i.e., the target face image), the shape of the background image is adjusted so that the shape of the adjusted background image is adapted to the target face image, and the target image and the adjusted background image are merged to obtain a high-quality synthesized image.

[0062] The technical solution of this embodiment extracts the current age features, shape features, and texture features from the facial image, and performs feature transfer learning in the two dimensions of shape features and texture features through a pre-set shape transfer model and texture transfer model, thereby improving the accuracy of transfer learning of the facial image in the two dimensions of shape features and texture features. At the same time, the current age features and the target age features are used to assist the transfer learning of the shape features and texture features. In the process of transfer learning, the humanized features of the facial image are taken into account, the differentiation of the target facial image is improved, the stereotyped transfer learning is avoided, and the quality of the target facial image is improved.

[0063] Based on the above embodiment, a preferred example of an image processing method is also provided. For example, see Figure 5 and Figure 6 , Figure 5 is a schematic diagram of the structure of a texture migration model provided by an embodiment of the present invention, Figure 6 It is a structural diagram of an attribute migration module provided in an embodiment of the present invention.

[0064] Figure 5 and Figure 6 The input image is a face image to be processed. The input image is processed based on the age estimation model to obtain the current age feature. The shape feature uvmap of the input image is extracted based on the preset shape feature extraction model. The texture feature uv texture map of the input image is extracted based on the preset texture feature extraction model. Figure 5The texture transfer model in the example has the input information of the current age feature, the texture feature uv texture map and the target age feature. The target age feature is a one-hot vector of length n. The output information of the texture transfer model is the texture feature uv texture map of the output image. Similarly, the structure of the shape transfer model can be the same as that of the texture transfer model. The input information of the shape transfer model is the shape feature uv map of the face image, the current age feature and the target age feature. The output information is the shape feature uv map of the output image. Through the render method, the converted face image is synthesized based on the shape feature uv map of the output image and the texture feature uv texture map of the output image. The refine model is used to synthesize the background and other information, and the face image generated in the third step is optimized to obtain a more realistic face image.

[0065] The texture migration model includes three attribute migration modules. The structure of the attribute migration module is shown in Figure 6 . Figure 6 Each attribute transfer module includes two convolutional layers (i.e., the first convolutional layer and the second convolutional layer), and two AdaIN networks (i.e., the first style transfer network and the second style transfer network). The input features of the attribute transfer module can be the output features of the feature extraction module, or the output features of the previous attribute transfer module.

[0066] Embodiment 3

[0067] Figure 7 : is a schematic diagram of the structure of an image processing device provided by an embodiment of the present invention, the device comprising:

[0068] The image acquisition module 310 is used to acquire an image to be processed.

[0069] The feature extraction module 320 is used to extract the current attribute features, shape features, texture features of the image to be processed and the target attribute features of the image to be processed;

[0070] A shape feature transfer module 330, configured to input the current attribute feature, the shape feature and the target attribute feature into a pre-trained shape transfer model to generate a shape feature corresponding to the target attribute;

[0071] The texture feature migration module 340 is used to input the current attribute feature, the texture feature and the target attribute feature into a pre-trained texture migration model to generate a texture feature corresponding to the target attribute;

[0072] The target image generation module 350 is used to perform feature fusion based on the texture feature corresponding to the target attribute and the shape feature to obtain a target image corresponding to the target attribute.

[0073] Based on the above embodiments, the shape migration model and the texture migration model respectively include a feature extraction module and at least one attribute migration module. The attribute migration module generates a current output feature based on the output feature of the feature extraction module or the output feature of the previous attribute migration module, the current attribute feature, and the target attribute feature, and inputs it to the next attribute migration module.

[0074] On the basis of the above embodiment, the attribute transfer module includes: a first convolutional layer, a first style transfer network, a second style transfer network and a second convolutional layer;

[0075] The first convolution layer is used to perform a convolution operation on the input features, and input the convolution features into the first style migration network and the second style migration network respectively;

[0076] The first style transfer network performs a transfer operation on the current attribute feature and the convolution feature to generate a first transfer feature, and the second style transfer network performs a transfer operation on the target attribute feature and the convolution feature to generate a second transfer feature;

[0077] The second convolution layer performs dimensionality reduction processing on the concatenated convolution features, the first migration features, and the second migration features to obtain current output features.

[0078] On the basis of the above embodiment, the shape transfer model and the texture transfer model respectively have a first parameter conversion module and a second parameter conversion module. The first parameter conversion module is used to convert the current attribute features into first style transfer parameters and output them to the first style transfer network. The second parameter conversion module is used to convert the target attribute features into second style transfer parameters and input them to the second style transfer network.

[0079] Based on the above embodiment, the image to be processed is a face image, and the attributes include an age attribute.

[0080] Based on the above embodiment, the image acquisition module 310 is used to:

[0081] Acquire an initial image, perform face segmentation on the initial image, and obtain a face image and a background image in the initial image;

[0082] Based on the above embodiment, the device further includes:

[0083] An image synthesis module is used to synthesize the target image and the background image.

[0084] Based on the above embodiment, the image synthesis module is used for:

[0085] adjusting the background image based on the target image;

[0086] The target image and the adjusted background image are merged.

[0087] The image processing device provided by the embodiment of the present invention can execute the image processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0088] Embodiment 4

[0089] Figure 8 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Figure 8 A block diagram of an electronic device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 8 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The device 12 is typically an electronic device that undertakes the image classification function.

[0090] like Figure 8 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors 16, a storage device 28, and a bus 18 connecting various system components (including the storage device 28 and the processor 16).

[0091] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0092] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0093] The storage device 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 not shown, usually called a "hard drive"). Although Figure 8 Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a read-only optical disk (Compact Disc-Read Only Memory, CD-ROM), a digital video disk (Digital Video Disc-Read Only Memory, DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The storage device 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.

[0094] A program 36 having a set (at least one) of program modules 26 may be stored, for example, in a storage device 28, such program modules 26 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may include an implementation of a gateway environment. The program modules 26 generally perform the functions and / or methods of the embodiments described herein.

[0095] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, cameras, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. In addition, the electronic device 12 may also communicate with one or more gateways (e.g., a local area network (LAN), a wide area network (WAN), and / or a public gateway, such as the Internet) via a gateway adapter 20. As shown, the gateway adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) systems, tape drives, and data backup storage systems.

[0096] The processor 16 executes various functional applications and data processing by running the programs stored in the storage device 28, such as implementing the image processing method provided by the above-mentioned embodiment of the present invention.

[0097] Embodiment 5

[0098] Embodiment 5 of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the image processing method provided in the embodiment of the present invention is implemented.

[0099] Of course, the computer-readable storage medium provided by the embodiment of the present invention stores a computer program which is not limited to the method operation described above, but can also execute the image processing method provided by any embodiment of the present invention.

[0100] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0101] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable source code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0102] Source code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0103] Computer source code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The source code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of gateway, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An image processing method, characterized in that: include: Acquire an image to be processed, and extract current attribute features, shape features, texture features of the image to be processed, and target attribute features of the image to be processed; Inputting the current attribute feature, the shape feature and the target attribute feature into a pre-trained shape transfer model to generate a shape feature corresponding to the target attribute; Inputting the current attribute feature, the texture feature and the target attribute feature into a pre-trained texture migration model to generate a texture feature corresponding to the target attribute; Perform feature fusion based on the texture feature corresponding to the target attribute and the shape feature to obtain a target image corresponding to the target attribute; Among them, the shape migration model and the texture migration model respectively include a feature extraction module and at least one attribute migration module, and the attribute migration module generates a current output feature based on the output feature of the feature extraction module or the output feature of the previous attribute migration module, the current attribute feature, and the target attribute feature, and inputs it to the next attribute migration module.

2. The method according to claim 1, characterized in that The attribute transfer module includes: a first convolutional layer, a first style transfer network, a second style transfer network and a second convolutional layer; The first convolution layer is used to perform a convolution operation on the input features, and input the convolution features into the first style transfer network and the second style transfer network respectively; The first style transfer network performs a transfer operation on the current attribute feature and the convolution feature to generate a first transfer feature, and the second style transfer network performs a transfer operation on the target attribute feature and the convolution feature to generate a second transfer feature; The second convolution layer performs dimensionality reduction processing on the concatenated convolution features, the first migration features, and the second migration features to obtain current output features.

3. The method according to claim 2, characterized in that The shape transfer model and the texture transfer model are respectively provided with a first parameter conversion module and a second parameter conversion module. The first parameter conversion module is used to convert the current attribute feature into a first style transfer parameter and output it to the first style transfer network. The second parameter conversion module is used to convert the target attribute feature into a second style transfer parameter and input it to the second style transfer network.

4. The method according to any one of claims 1 to 3, characterized in that: The image to be processed is a face image, and the attributes include an age attribute.

5. The method according to claim 4, characterized in that The obtaining of the image to be processed comprises: Acquire an initial image, perform face segmentation on the initial image, and obtain a face image and a background image in the initial image; After obtaining the target image corresponding to the target attribute, the method further includes: The target image and the background image are synthesized.

6. The method according to claim 5, characterized in that The synthesizing the target image and the background image comprises: adjusting the background image based on the target image; The target image and the adjusted background image are merged.

7. An image processing device, characterized in that: include: A feature extraction module is used to obtain an image to be processed, and extract current attribute features, shape features, texture features of the image to be processed, and target attribute features of the image to be processed; A shape feature migration module, used for inputting the current attribute feature, the shape feature and the target attribute feature into a pre-trained shape migration model to generate a shape feature corresponding to the target attribute; A texture feature migration module, used for inputting the current attribute feature, the texture feature and the target attribute feature into a pre-trained texture migration model to generate a texture feature corresponding to the target attribute; A target image generation module, used for performing feature fusion based on the texture feature corresponding to the target attribute and the shape feature to obtain a target image corresponding to the target attribute; Among them, the shape migration model and the texture migration model respectively include a feature extraction module and at least one attribute migration module, and the attribute migration module generates a current output feature based on the output feature of the feature extraction module or the output feature of the previous attribute migration module, the current attribute feature, and the target attribute feature, and inputs it to the next attribute migration module.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the image processing method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image processing method as described in any one of claims 1 to 6 is implemented.

Images