Image generation method, device, equipment and storage medium

Abstract

The present invention relates to artificial intelligence and provides an image generation method, apparatus, device, and storage medium. The method transforms facial features of a source image to obtain transformed features, identifies first adaptive keypoint coordinates from the source image based on an adaptive keypoint recognition network, identifies first frontal facial keypoints of the source image based on the first adaptive keypoint coordinates and first facial motion features of the source image, and identifies second frontal facial keypoints of a driving image based on the first adaptive keypoint coordinates and second facial motion features of a driving image. Optical flow features derived from the transformed features, the second frontal facial keypoints, and the first frontal facial keypoints are input into an image generation network to obtain an output image, and image loss is calculated. Based on the image loss and a facial driving model, a natural and realistic target facial image can be generated. Furthermore, the present invention relates to blockchain technology, and the target facial image can be stored in a blockchain.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an image generation method, device, equipment and storage medium. Background Art

[0002] In the field of artificial intelligence, facial driving of virtual humans is a crucial technical direction. The realism and naturalness of facial movements are closely related to the overall driving effect.

[0003] In current face-driven solutions, facial movements are usually searched based on a trained feature space. Since there is no guarantee that the facial movements of the same person in the current feature space will be continuous, it is difficult to find the corresponding feature areas and movement content when encountering faces outside the training set domain. This results in facial details, especially mouth movements, not being realistic and natural enough. Summary of the Invention

[0004] In view of the above, it is necessary to provide an image generation method, apparatus, device and storage medium that can solve the technical problem of how to improve the authenticity and naturalness of the movements of virtual human facial images.

[0005] In one aspect, the present invention provides an image generation method, comprising:

[0006] Acquire a source image and a driving image, and acquire a face driving model, wherein the face driving model includes a face feature extraction network, an adaptive key point recognition network, a face posture detection network, and an image generation network;

[0007] Inputting the source image into the facial feature extraction network, and performing transformation processing on the obtained facial features to obtain transformed features;

[0008] Based on the adaptive key point recognition network, identifying first adaptive key point coordinates from the source image;

[0009] detecting a first facial action feature from the source image based on the facial pose detection network, identifying a first frontal facial key point of the source image based on the first adaptive key point coordinates and the first facial action feature, detecting a second facial action feature from the drive image based on the facial pose detection network, and identifying a second frontal facial key point of the drive image based on the first adaptive key point coordinates and the second facial action feature;

[0010] generating an optical flow feature based on the second frontal facial key point and the first frontal facial key point, and inputting the transformation feature and the optical flow feature into the image generation network to obtain an output image;

[0011] Generate an image loss for the output image based on an overall feature loss between the output image and the driving image, a face shape generation loss and a mouth shape generation loss of the output image, a local feature loss between the output image and the driving image, a consistency loss between the first frontal face key point and the second frontal face key point, a priori loss of coordinates of the second adaptive key point of the driving image, a posture loss of the driving image, a facial surface motion feature of the source image, and a difference in mouth shape between the output image and the driving image;

[0012] A target facial image is generated according to the image loss and the face-driven model.

[0013] According to a preferred embodiment of the present invention, the step of inputting the source image into the facial feature extraction network and performing transformation processing on the obtained facial features to obtain the transformed features comprises:

[0014] Acquiring voice data corresponding to the source image;

[0015] extracting speech features of the speech data;

[0016] Acquire the transformation coefficient that successfully matches the speech feature from the configuration library;

[0017] The facial features are transformed based on the transformation coefficients to obtain the transformed features.

[0018] According to a preferred embodiment of the present invention, the first facial motion feature includes the rotation matrix and offset of the source image and the facial surface motion feature, and the recognition formula of the first frontal facial key point is:

[0019] KP c =R -1 (KP s -t-exp);

[0020] Among them, KP c represents the key point matrix composed of the first frontal key points, R -1 represents the inverse matrix of the rotation matrix of the source image, KP s represents the key point matrix formed by the coordinates of the first adaptive key points, t represents the offset, and exp represents the facial surface motion feature.

[0021] According to a preferred embodiment of the present invention, generating optical flow features based on the second frontal face key points and the first frontal face key points, and inputting the transformation features and the optical flow features into the image generation network to obtain the output image includes:

[0022] Calculating a matrix difference between a key point matrix formed by the second frontal face key points and a key point matrix formed by the first frontal face key points to obtain the optical flow feature;

[0023] splicing the transformation features and the optical flow features to obtain input features;

[0024] Performing iterative convolution processing on the input features to obtain mapping features;

[0025] Performing a deconvolution operation on the mapped feature map to obtain a deconvolution feature;

[0026] The deconvolution feature is activated based on a preset activation function to obtain the output image.

[0027] According to a preferred embodiment of the present invention, the calculation formula for the image loss is:

[0028] L G =λ1L p +λ2L g +λ3L q +λ4L f +λ5L e +λ6L l +λ7L h +λ8L d +λ9L s ;

[0029]

[0030]

[0031]

[0032]

[0033]

[0034] L d =||exp||1;

[0035] L m =||J(G(z))-J(d)||2;

[0036] Among them, L G represents the image loss, L p Denotes the overall feature loss, L g represents the face shape generation loss, L q Represents the mouth shape generation loss, L f Represents the local feature loss, L e Denotes the consistency loss, L lDenotes the prior loss, L h represents the posture loss, L d represents the constraint loss of the face surface motion feature, L m represents the mouth shape difference, λ1, λ2, λ3, λ4, λ5, λ6, λ7, λ8, and λ9 are preset weights respectively, d represents the driving image, G(z) represents the output image, and D 1f represents the intermediate feature layer of the pre-trained face shape discrimination model, m represents the number of feature pixels of the driving image or the output image extracted by the intermediate feature layer, N represents the number of key points of the first frontal face key point or the second frontal face key point, and xi Jiliang yi represents the coordinate value of the first frontal face key point, and represents the coordinate value of the second frontal key point, K represents the number of key points corresponding to the coordinates of the second adaptive key point, kp i and kp j Respectively represent any two key points corresponding to the coordinates of the second adaptive key point, kp represents any key point corresponding to the coordinates of the second adaptive key point, M() represents the average value calculation formula, D t and z t Represents the preset hyperparameters, R d represents the rotation matrix identified by the pre-trained facial posture model for the driving image, represents the rotation matrix identified by the face posture detection network for the driving image, exp represents the face surface motion feature, and J represents the pre-trained mouth feature extraction model.

[0037] According to a preferred embodiment of the present invention, generating a target facial image according to the image loss and the face-driven model includes:

[0038] When the image loss is greater than a minimum loss threshold of the face driving model, adjusting the face driving model based on the image loss until the image loss meets a preset condition, thereby obtaining a target driving model;

[0039] An output image generated by the target driving model for the source image and the driving image is determined as the target facial image.

[0040] According to a preferred embodiment of the present invention, after adjusting the face driven model based on the image loss, the method further includes:

[0041] Processing the source image and the driving image based on the adjusted face driving model to obtain an updated image;

[0042] Calculating an update loss of the updated image based on the preset weights;

[0043] If the loss difference between the image loss and the update loss is less than a preset difference threshold, the preset weight is adjusted to obtain an updated weight.

[0044] On the other hand, the present invention further provides an image generating device, comprising:

[0045] An acquisition unit, configured to acquire a source image and a driving image, and acquire a face driving model, wherein the face driving model includes a face feature extraction network, an adaptive key point recognition network, a face posture detection network, and an image generation network;

[0046] a transformation unit, configured to input the source image into the facial feature extraction network, and transform the obtained facial features to obtain transformed features;

[0047] an identification unit, configured to identify first adaptive key point coordinates from the source image based on the adaptive key point identification network;

[0048] The recognition unit is further configured to detect a first facial action feature from the source image based on the facial pose detection network, identify a first frontal facial key point of the source image based on the first adaptive key point coordinates and the first facial action feature, detect a second facial action feature from the drive image based on the facial pose detection network, and identify a second frontal facial key point of the drive image based on the first adaptive key point coordinates and the second facial action feature;

[0049] an input unit, configured to generate an optical flow feature based on the second frontal facial key points and the first frontal facial key points, and input the transformation feature and the optical flow feature into the image generation network to obtain an output image;

[0050] a generating unit, configured to generate an image loss of the output image based on an overall feature loss of the output image and the driving image, a face shape generation loss and a mouth shape generation loss of the output image, a local feature loss of the output image and the driving image, a consistency loss of the first frontal face key point and the second frontal face key point, a priori loss of the coordinates of the second adaptive key point of the driving image, a posture loss of the driving image, a facial surface motion feature of the source image, and a difference in mouth shape between the output image and the driving image;

[0051] The generating unit is further configured to generate a target facial image according to the image loss and the face driving model.

[0052] On the other hand, the present invention further provides an electronic device, comprising:

[0053] a memory storing computer-readable instructions; and

[0054] A processor executes the computer-readable instructions stored in the memory to implement the image generating method.

[0055] On the other hand, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the image generation method.

[0056] It can be seen from the above technical solutions that this application can avoid directly predicting the frontal face key points by predicting the positions of the 3D facial key points, thereby reducing the fitting difficulty of the adaptive key point recognition network and enabling the face drive model to perform better under drastic posture changes. On the other hand, this application adds the calculation of the image loss caused by the mouth shape difference, realizes the constraint of the mouth movement on the face drive model, further optimizes the performance of the mouth movement, and enables the decoupling of the face drive model to carry more mouth features, thereby being able to express more complex mouth movements, thereby improving the movement authenticity and naturalness of the target facial image. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flow chart of a preferred embodiment of the image generating method of the present invention.

[0058] Figure 2 It is a network structure diagram of the mouth feature extraction model in the image generation method of the present invention.

[0059] Figure 3 It is a schematic diagram of the face driving model and the effect of mouth movements on the face driving model in the image generation method of the present invention.

[0060] Figure 4 It is a functional module diagram of a preferred embodiment of the image generating device of the present invention.

[0061] Figure 5 It is a structural diagram of an electronic device according to a preferred embodiment of the present invention for implementing the image generation method. DETAILED DESCRIPTION

[0062] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] like Figure 1FIG. 1 is a flow chart of a preferred embodiment of the image generation method of the present invention. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0064] The image generation method can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0065] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0066] The image generation method is applied to one or more electronic devices, which are devices that can automatically perform numerical calculations and / or information processing according to pre-set or stored computer-readable instructions. Their hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0067] The electronic device can be any electronic product that can interact with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0068] The electronic device may include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network electronic device, a group of electronic devices consisting of multiple network electronic devices, or a cloud based on cloud computing consisting of a large number of hosts or network electronic devices.

[0069] The network where the electronic device is located includes, but is not limited to: the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0070] 101. Obtain a source image and a driving image, and obtain a face driving model, wherein the face driving model includes a face feature extraction network, an adaptive key point recognition network, a face posture detection network, and an image generation network.

[0071] In at least one embodiment of the present invention, the image object in the source image is different from the image object in the driving image. The source image may be a facial image of a real person, and the driving image may be a facial image of any virtual person.

[0072] In at least one embodiment of the present invention, the face-driven model may be a model generated by training based on historical source images and historical driving images.

[0073] 102. Input the source image into the facial feature extraction network, and transform the obtained facial features to obtain transformed features.

[0074] In at least one embodiment of the present invention, the facial feature extraction network includes multiple convolutional network layers.

[0075] The facial features refer to features obtained by convolving the source image with multiple convolutional network layers in the facial feature extraction network.

[0076] The transformed features refer to features generated after twisting the facial features.

[0077] In at least one embodiment of the present invention, the electronic device inputs the source image into the facial feature extraction network, and the obtained facial features are transformed, where the obtained transformed features include:

[0078] Acquiring voice data corresponding to the source image;

[0079] Extracting speech features of the speech data;

[0080] Acquire the transformation coefficient that successfully matches the speech feature from the configuration library;

[0081] The facial features are transformed based on the transformation coefficients to obtain the transformed features.

[0082] The voice data refers to a voice segment extracted from the voice video containing the source image.

[0083] The configuration library stores a plurality of mapping relationships between configuration features and configuration coefficients.

[0084] Through the voice features, suitable transformation coefficients can be matched, and then the facial features can be transformed using the transformation coefficients to improve the authenticity and naturalness of the transformed features.

[0085] Specifically, the electronic device obtaining, from a configuration library, a transformation coefficient that successfully matches the voice feature includes:

[0086] Calculating feature similarity between the configuration features in the configuration library and the speech features;

[0087] The configuration coefficient corresponding to the configuration feature with the highest feature similarity is determined as the transformation coefficient.

[0088] Specifically, the electronic device performs transformation processing on the facial features based on the transformation coefficients, and the obtained transformation features include:

[0089] The product of the facial feature and the transformation coefficient is calculated to obtain the transformation feature.

[0090] 103. Identify first adaptive key point coordinates from the source image based on the adaptive key point recognition network.

[0091] In at least one embodiment of the present invention, the adaptive keypoint recognition network stores image pixel values of multiple reference images at each configured keypoint.

[0092] The first adaptive key point coordinates include coordinate values of a plurality of adaptive key points in the source image on the x-axis, the y-axis, and the z-axis.

[0093] In at least one embodiment of the present invention, the electronic device identifying the first adaptive key point coordinates from the source image based on the adaptive key point recognition network includes:

[0094] Obtaining a source pixel value of each pixel in the source image;

[0095] Calculating a pixel difference between the source pixel value and the image pixel value of the plurality of reference images at each configuration key point;

[0096] identifying adaptive source key points of the source image according to the pixel difference;

[0097] The first adaptive key point coordinates are generated based on the image position and depth information of the adaptive source key point in the source image.

[0098] The adaptive source key point refers to a pixel point that has the smallest pixel difference with the corresponding configuration key point and is located in the source image.

[0099] The image position includes the row position and the column position of the adaptive source key point in the source image.

[0100] The depth information may be information obtained by a binocular camera device photographing the image object of the source image.

[0101] The coordinate value of the first adaptive key point coordinate on the x-axis may be the row position, the coordinate value of the first adaptive key point coordinate on the y-axis may be the column position, and the coordinate value of the first adaptive key point coordinate on the z-axis may be the depth information.

[0102] By comparing the pixel values of the source image with those of multiple reference images stored in the adaptive key point recognition network, the recognition accuracy of the adaptive source key points can be improved, and then the image position and the depth information can be combined as the first adaptive key point coordinates to improve the comprehensiveness of the first adaptive key point coordinates.

[0103] 104. Detect a first facial action feature from the source image based on the facial pose detection network, identify a first frontal facial key point of the source image according to the first adaptive key point coordinates and the first facial action feature, and detect a second facial action feature from the drive image based on the facial pose detection network, identify a second frontal facial key point of the drive image according to the first adaptive key point coordinates and the second facial action feature.

[0104] In at least one embodiment of the present invention, the face posture detection network includes multiple convolutional layers and an activation layer, a global average pooling layer, etc. corresponding to each convolutional layer.

[0105] The first facial motion feature includes a rotation matrix and an offset of the source image and the facial surface motion feature. The first frontal facial key point is an adaptive source key point corresponding to when the pitch degree of freedom, the yaw degree of freedom, and the roll degree of freedom in the source image are all zero.

[0106] The second facial motion features include a rotation matrix of the driving image, an offset of the driving image, and facial surface motion features of the driving image. The second frontal facial keypoints are adaptive source keypoints corresponding to zero pitch, yaw, and roll degrees of freedom in the driving image.

[0107] In at least one embodiment of the present invention, the first facial action feature is generated in the following manner:

[0108] Based on the multiple convolutional layers in the face posture detection network and the activation layers corresponding to each convolutional layer, extracting motion features from the source image to obtain initial features;

[0109] Based on the global average pooling layer in the face pose detection network, the initial features are pooled to obtain the face surface motion features corresponding to the source image;

[0110] The rotation matrix of the source image and the offset of the source image are calculated according to the face surface motion features and configuration features.

[0111] The configuration features refer to pre-set facial motion features.

[0112] In this embodiment, the configuration matrix is obtained by multiplying the facial surface motion feature by the rotation matrix and summing the matrix product by the offset.

[0113] In at least one embodiment of the present invention, the identification formula of the first frontal facial key point is:

[0114] KP c =R -1 (KP s -t-exp);

[0115] Among them, KP c represents the key point matrix composed of the first frontal key points, R -1 represents the inverse matrix of the rotation matrix of the source image, KP s represents the key point matrix formed by the coordinates of the first adaptive key points, t represents the offset, and exp represents the facial surface motion feature.

[0116] In this embodiment, the key point matrix formed by the first frontal facial key points is an n*3 matrix, the offset is an n*3 matrix, and the facial surface motion features are an n*3 matrix.

[0117] By combining the first facial action feature, the coordinates of the first adaptive key points can be accurately adjusted to improve the recognition accuracy of the first frontal facial key points.

[0118] In other embodiments, the method for identifying the second frontal facial key points is similar to the method for identifying the first frontal facial key points, and this application will not elaborate on this.

[0119] 105. Generate optical flow features based on the second frontal facial key points and the first frontal facial key points, and input the transformation features and the optical flow features into the image generation network to obtain an output image.

[0120] In at least one embodiment of the present invention, the output image refers to an image generated after the face driving model drives the driving image based on the source image.

[0121] In at least one embodiment of the present invention, the electronic device generates optical flow features based on the second frontal facial key points and the first frontal facial key points, and inputs the transformation features and the optical flow features into the image generation network to obtain an output image including:

[0122] Calculating a matrix difference between a key point matrix formed by the second frontal face key points and a key point matrix formed by the first frontal face key points to obtain the optical flow feature;

[0123] splicing the transformation features and the optical flow features to obtain input features;

[0124] Performing iterative convolution processing on the input features to obtain mapping features;

[0125] Performing a deconvolution operation on the mapped feature map to obtain a deconvolution feature;

[0126] The deconvolution feature is activated based on a preset activation function to obtain the output image.

[0127] Among them, the preset activation function can be a tanh function, the preset activation function can also be a relu function, etc. This application does not limit the preset activation function.

[0128] By combining the transformation features and the optical flow features, the comprehensiveness of the input features can be improved. By activating the deconvolution features through the preset activation function, the convergence efficiency of the deconvolution features can be improved, thereby improving the generation efficiency of the output image.

[0129] Specifically, the electronic device performs iterative convolution processing on the input features to obtain mapping features including:

[0130] Performing deconvolution processing on the input features to obtain initial features;

[0131] Performing batch normalization on the initial features to obtain intermediate features;

[0132] The intermediate features are activated to obtain output features, and the output features are used as input features for the next iterative convolution process until the number of iterations of the iterative convolution process reaches a preset number, thereby obtaining the mapping features.

[0133] The preset number of times is equal to the number of iterative convolutional layers in the image generation network.

[0134] By performing batch normalization on the initial features, the output intermediate features can be made to obey the distribution, thereby accelerating the generation speed of the mapping features.

[0135] 106. Generate the image loss of the output image based on the overall feature loss of the output image and the driving image, the face shape generation loss and the mouth shape generation loss of the output image, the local feature loss of the output image and the driving image, the consistency loss of the first frontal face key point and the second frontal face key point, the prior loss of the second adaptive key point coordinates of the driving image, the posture loss of the driving image, the facial surface action features of the source image, and the difference in mouth shape between the output image and the driving image.

[0136] In at least one embodiment of the present invention, the overall feature loss is generated by:

[0137] Performing feature extraction on the output image based on a pre-trained image feature extraction model to obtain a first feature;

[0138] Performing feature extraction on the driving image based on the image feature extraction model to obtain a second feature;

[0139] The overall feature loss is generated according to a feature difference between the first feature and the second feature.

[0140] Among them, the image feature extraction model can be a model generated based on VGG19 network training.

[0141] Through the above implementation, the difference between the output image and the driving image can be analyzed from the same dimension based on the image feature extraction model, thereby improving the quantification accuracy of the overall feature loss.

[0142] In at least one embodiment of the present invention, the face shape generation loss is generated in the following manner:

[0143] Inputting the output image into the pre-trained face shape discrimination model to obtain the predicted probability corresponding to the output image on the configuration label;

[0144] The face shape generation loss is calculated based on the predicted probability and the labeling probability of the output image on the configuration label. The calculation formula of the face shape generation loss is:

[0145]

[0146] Among them, L g represents the face shape generation loss, y represents the labeling probability, represents the predicted probability.

[0147] The pre-trained face shape discrimination model and the image generation network may constitute a generative adversarial network.

[0148] The configuration tag can be set according to actual needs.

[0149] The accuracy of the generation loss can be improved by predicting the output image using a face recognition model that is adversarial to the image generation network, and by labeling the output image in the dimension corresponding to the configuration label.

[0150] In other embodiments, the method for generating the mouth shape generation loss is similar to the method for generating the face shape generation loss, and this application will not elaborate on this.

[0151] In at least one embodiment of the present invention, the image loss is calculated as follows:

[0152] L G =λ1L p +λ2L g +λ3L q +λ4L f +λ5L e +λ6L l +λ7L h +λ8L d +λ9L s ;

[0153]

[0154]

[0155]

[0156]

[0157]

[0158] L d =||exp||1;

[0159] L m =||J(G(z))-J(d)||2;

[0160] Among them, L G represents the image loss, L p Denotes the overall feature loss, L g represents the face shape generation loss, L q Represents the mouth shape generation loss, L f Represents the local feature loss, L e Denotes the consistency loss, L lDenotes the prior loss, L h represents the posture loss, L d represents the constraint loss of the face surface motion feature, L m represents the mouth shape difference, λ1, λ2, λ3, λ4, λ5, λ6, λ7, λ8, and λ9 are preset weights respectively, d represents the driving image, G(z) represents the output image, and D 1f represents the intermediate feature layer of the pre-trained face shape discrimination model, m represents the number of feature pixels of the driving image or the output image extracted by the intermediate feature layer, N represents the number of key points of the first frontal face key point or the second frontal face key point, k xi and k yi represents the coordinate value of the first frontal face key point, and represents the coordinate value of the second frontal key point, K represents the number of key points corresponding to the coordinates of the second adaptive key point, kp i and kp j Respectively represent any two key points corresponding to the coordinates of the second adaptive key point, kp represents any key point corresponding to the coordinates of the second adaptive key point, M() represents the average value calculation formula, D t and z t Represents the preset hyperparameters, R d represents the rotation matrix identified by the pre-trained facial posture model for the driving image, represents the rotation matrix identified by the face posture detection network for the driving image, exp represents the face surface motion feature, and J represents the pre-trained mouth feature extraction model.

[0161] In this embodiment, since the numerical value changes of the facial surface motion features are too large, which will cause the stability of the key point matrix composed of the first frontal facial key points to deteriorate, regularization of the facial surface motion features is added to ensure the stability of the key point matrix composed of the first frontal facial key points. The mouth feature extraction model can add the constraint loss of the mouth shape to the image loss based on the difference in mouth shape between the output image and the driving image, ensuring that the image loss can reflect the mouth movement.

[0162] In this embodiment, the mouth feature extraction model includes multiple convolutional networks. Figure 2 As shown, Figure 2 This is the network structure diagram of the mouth feature extraction model in the image generation method of the present invention. Figure 2In the figure, Input represents the input layer of the mouth feature extraction model, Conv represents the convolutional network of the mouth feature extraction model, LSTM (Long Short-Term Memory) represents the long short-term memory network of the mouth feature extraction model, and the CTC (Connectionist temporal classification) network is used to process the alignment problem of the input and output labels of the mouth feature extraction model.

[0163] In this embodiment, Figure 3 As shown, Figure 3 Schematic diagram of the face driving model and the effect of mouth movement on the face driving model in the image generation method of the present invention. Figure 3 In the example, Source represents the source image, Driving represents the driving image, FaceNet represents the face feature extraction network, Fs represents the face feature, Fs_warped represents the transformation feature, LmkNet represents the adaptive key point recognition network, KP s represents the key point matrix formed by the coordinates of the first adaptive key points, PoseNet represents the face pose detection network, Rs represents the rotation matrix in the first facial action feature, ts represents the offset in the first facial action feature, exp_s represents the face surface action feature in the first facial action feature, Rd represents the rotation matrix in the second facial action feature, td represents the offset in the second facial action feature, exp_d represents the face surface action feature in the second facial action feature, KP c represents the key point matrix composed of the first frontal key points, KP d represents the key point matrix formed by the second frontal facial key points, Flow represents the optical flow features, G represents the image generation network, Output represents the output image, MouthNet represents the mouth feature extraction model, D1 represents the face shape discrimination model, D2 represents the pre-trained mouth shape discrimination model, and D1 and D2 respectively constitute a generative adversarial network with G.

[0164] 107 : Generate a target facial image according to the image loss and the face driving model.

[0165] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned target facial image, the above-mentioned target facial image can also be stored in a node of a blockchain.

[0166] In at least one embodiment of the present invention, the target facial image is the output image corresponding to when the image loss satisfies a preset condition. The preset condition may be that the image loss is less than or equal to a minimum loss threshold of the face-driven model, or that the image loss no longer decreases.

[0167] In at least one embodiment of the present invention, the electronic device generating a target facial image according to the image loss and the face driving model includes:

[0168] When the image loss is greater than a minimum loss threshold of the face driving model, adjusting the face driving model based on the image loss until the image loss meets a preset condition, thereby obtaining a target driving model;

[0169] An output image generated by the target driving model for the source image and the driving image is determined as the target facial image.

[0170] The minimum loss threshold can be set according to actual needs.

[0171] By adjusting the parameters of the face driving model through the image loss, the driving ability of the target driving model on the source image and the driving image can be improved, thereby improving the naturalness and movement authenticity of the target facial image.

[0172] In at least one embodiment of the present invention, after adjusting the face-driven model based on the image loss, the method further includes:

[0173] Processing the source image and the driving image based on the adjusted face driving model to obtain an updated image;

[0174] Calculating an update loss of the updated image based on the preset weights;

[0175] If the loss difference between the image loss and the update loss is less than a preset difference threshold, the preset weight is adjusted to obtain an updated weight.

[0176] The preset difference threshold can be set according to actual needs.

[0177] By adjusting the preset weight when the loss difference between the image loss and the update loss is less than a preset difference threshold, it is possible to avoid the situation where the change between the image loss and the update loss is too small due to a certain weight being too small, thereby improving the generation accuracy of the image loss.

[0178] It can be seen from the above technical solutions that this application can avoid directly predicting the frontal face key points by predicting the positions of the 3D facial key points, thereby reducing the fitting difficulty of the adaptive key point recognition network and enabling the face drive model to perform better under drastic posture changes. On the other hand, this application adds the calculation of the image loss caused by the mouth shape difference, realizes the constraint of the mouth movement on the face drive model, further optimizes the performance of the mouth movement, and enables the decoupling of the face drive model to carry more mouth features, thereby being able to express more complex mouth movements, thereby improving the movement authenticity and naturalness of the target facial image.

[0179] like Figure 4 , which is a functional block diagram of a preferred embodiment of an image generation device according to the present invention. The image generation device 11 includes an acquisition unit 110, a transformation unit 111, a recognition unit 112, an input unit 113, a generation unit 114, a processing unit 115, a calculation unit 116, and an adjustment unit 117. As used herein, a module / unit refers to a series of computer-readable instruction segments that can be acquired by the processor 13 and perform a fixed function, and is stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0180] An acquisition unit 110 is configured to acquire a source image and a driving image, and to acquire a face driving model, wherein the face driving model includes a face feature extraction network, an adaptive key point recognition network, a face posture detection network, and an image generation network;

[0181] The transformation unit 111 is used to input the source image into the facial feature extraction network, and transform the obtained facial features to obtain transformed features;

[0182] an identification unit 112, configured to identify first adaptive key point coordinates from the source image based on the adaptive key point identification network;

[0183] The recognition unit 112 is further configured to detect a first facial action feature from the source image based on the facial pose detection network, identify a first frontal facial key point of the source image based on the first adaptive key point coordinates and the first facial action feature, detect a second facial action feature from the drive image based on the facial pose detection network, and identify a second frontal facial key point of the drive image based on the first adaptive key point coordinates and the second facial action feature;

[0184] An input unit 113 is configured to generate an optical flow feature based on the second frontal facial key points and the first frontal facial key points, and input the transformation feature and the optical flow feature into the image generation network to obtain an output image;

[0185] a generating unit 114 configured to generate an image loss of the output image based on an overall feature loss of the output image and the driving image, a face shape generation loss and a mouth shape generation loss of the output image, a local feature loss of the output image and the driving image, a consistency loss of the first frontal facial key point and the second frontal facial key point, a priori loss of the coordinates of the second adaptive key point of the driving image, a posture loss of the driving image, a facial surface motion feature of the source image, and a difference in mouth shape between the output image and the driving image;

[0186] The generating unit 114 is further configured to generate a target facial image according to the image loss and the face driving model.

[0187] In at least one embodiment of the present invention, the transforming unit 111 is further configured to obtain speech data corresponding to the source image;

[0188] extracting speech features of the speech data;

[0189] Acquire the transformation coefficient that successfully matches the speech feature from the configuration library;

[0190] The facial features are transformed based on the transformation coefficients to obtain the transformed features.

[0191] In at least one embodiment of the present invention, the first facial motion feature includes the rotation matrix and offset of the source image and the facial surface motion feature, and the identification formula of the first frontal facial key point is:

[0192] KP c =R -1 (KP s -t-exp);

[0193] Among them, KP c represents the key point matrix composed of the first frontal key points, R -1 represents the inverse matrix of the rotation matrix of the source image, KP s represents the key point matrix formed by the coordinates of the first adaptive key points, t represents the offset, and exp represents the facial surface motion feature.

[0194] In at least one embodiment of the present invention, the input unit 113 is further configured to calculate a matrix difference between a key point matrix formed by the second frontal facial key points and a key point matrix formed by the first frontal facial key points to obtain the optical flow feature;

[0195] splicing the transformation features and the optical flow features to obtain input features;

[0196] Performing iterative convolution processing on the input features to obtain mapping features;

[0197] Performing a deconvolution operation on the mapped feature map to obtain a deconvolution feature;

[0198] The deconvolution feature is activated based on a preset activation function to obtain the output image.

[0199] In at least one embodiment of the present invention, the image loss is calculated as follows:

[0200] L G =λ1L p +λ2L g +λ3L q +λ4L f +λ5L e +λ6L l +λ7L h +λ8L d +λ9L s ;

[0201]

[0202]

[0203]

[0204]

[0205]

[0206] L d =||exp||1;

[0207] L m =||J(G(z))-J(d)||2;

[0208] Among them, L G represents the image loss, L p Denotes the overall feature loss, L g represents the face shape generation loss, L q Represents the mouth shape generation loss, L f Represents the local feature loss, L e Denotes the consistency loss, L l Denotes the prior loss, L h represents the posture loss, L d represents the constraint loss of the face surface motion feature, L mrepresents the mouth shape difference, λ1, λ2, λ3, λ4, λ5, λ6, λ7, λ8, and λ9 are preset weights respectively, d represents the driving image, G(z) represents the output image, and D 1f represents the intermediate feature layer of the pre-trained face shape discrimination model, m represents the number of feature pixels of the driving image or the output image extracted by the intermediate feature layer, N represents the number of key points of the first frontal face key point or the second frontal face key point, k xi and k yi represents the coordinate value of the first frontal face key point, and represents the coordinate value of the second frontal key point, K represents the number of key points corresponding to the coordinates of the second adaptive key point, kp i and kp j Respectively represent any two key points corresponding to the coordinates of the second adaptive key point, kp represents any key point corresponding to the coordinates of the second adaptive key point, M() represents the average value calculation formula, D t and z t Represents the preset hyperparameters, R d represents the rotation matrix identified by the pre-trained facial posture model for the driving image, represents the rotation matrix identified by the face posture detection network for the driving image, exp represents the face surface motion feature, and J represents the pre-completed mouth feature extraction model.

[0209] In at least one embodiment of the present invention, the generating unit 114 is further configured to, when the image loss is greater than a minimum loss threshold of the face driving model, adjust the face driving model based on the image loss until the image loss satisfies a preset condition, thereby obtaining a target driving model;

[0210] An output image generated by the target driving model for the source image and the driving image is determined as the target facial image.

[0211] In at least one embodiment of the present invention, after adjusting the face driving model based on the image loss, the processing unit 115 is configured to process the source image and the driving image based on the adjusted face driving model to obtain an updated image;

[0212] a calculation unit 116, configured to calculate an update loss of the updated image based on the preset weight;

[0213] The adjusting unit 117 is configured to adjust the preset weight to obtain an updated weight if the loss difference between the image loss and the update loss is less than a preset difference threshold.

[0214] It can be seen from the above technical solutions that this application can avoid directly predicting the frontal face key points by predicting the positions of the 3D facial key points, thereby reducing the fitting difficulty of the adaptive key point recognition network and enabling the face drive model to perform better under drastic posture changes. On the other hand, this application adds the calculation of the image loss caused by the mouth shape difference, realizes the constraint of the mouth movement on the face drive model, further optimizes the performance of the mouth movement, and enables the decoupling of the face drive model to carry more mouth features, thereby being able to express more complex mouth movements, thereby improving the movement authenticity and naturalness of the target facial image.

[0215] like Figure 5 FIG. 1 is a schematic diagram of the structure of an electronic device according to a preferred embodiment of the present invention for implementing the image generation method.

[0216] In one embodiment of the present invention, the electronic device 1 includes, but is not limited to, a memory 12 , a processor 13 , and computer-readable instructions stored in the memory 12 and executable on the processor 13 , such as an image generation program.

[0217] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. The electronic device 1 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 1 may also include input and output devices, network access devices, buses, etc.

[0218] The processor 13 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 13 is the computing core and control center of the electronic device 1, connecting various parts of the entire electronic device 1 using various interfaces and lines, and executing the operating system of the electronic device 1 as well as various installed applications, program codes, etc.

[0219] Exemplarily, the computer-readable instructions may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to implement the present invention. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which describe the execution process of the computer-readable instructions in the electronic device 1. For example, the computer-readable instructions may be divided into an acquisition unit 110, a transformation unit 111, a recognition unit 112, an input unit 113, a generation unit 114, a processing unit 115, a calculation unit 116, and an adjustment unit 117.

[0220] The memory 12 can be used to store the computer-readable instructions and / or modules. The processor 13 implements the various functions of the electronic device 1 by running or executing the computer-readable instructions and / or modules stored in the memory 12 and calling the data stored in the memory 12. The memory 12 can mainly include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the electronic device. The memory 12 can include non-volatile and volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, at least one disk storage device, a flash memory device, or other storage devices.

[0221] The memory 12 may be an external memory and / or an internal memory of the electronic device 1. Furthermore, the memory 12 may be a physical memory, such as a memory stick, a TF card (Trans-flash Card), and the like.

[0222] If the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention can also implement all or part of the processes in the above-mentioned embodiment methods by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed by a processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0223] The computer-readable instructions include computer-readable instruction codes, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer-readable instruction codes, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), or a random access memory (RAM).

[0224] The blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed image generation, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0225] Combine Figure 1 The memory 12 in the electronic device 1 stores computer-readable instructions to implement an image generation method, and the processor 13 can execute the computer-readable instructions to implement:

[0226] Acquire a source image and a driving image, and acquire a face driving model, wherein the face driving model includes a face feature extraction network, an adaptive key point recognition network, a face posture detection network, and an image generation network;

[0227] Inputting the source image into the facial feature extraction network to perform transformation processing on the facial features obtained to obtain transformed features;

[0228] Identifying first adaptive keypoint coordinates from the source image based on the adaptive keypoint recognition network;

[0229] Identifying first frontal facial key points of the source image based on the first adaptive key point coordinates and first facial motion features generated by the face pose detection network for the source image, and identifying second frontal facial key points of the drive image based on the first adaptive key point coordinates and second facial motion features generated by the face pose detection network for the drive image;

[0230] Inputting the transformed features, the second frontal facial key points, and the optical flow features obtained by the first frontal facial key points into the image generation network to obtain an output image;

[0231] The image loss of the output image is generated based on the overall feature loss of the output image and the driving image, the face shape generation loss and mouth shape generation loss of the output image by the image generation network, the local feature loss of the output image and the driving image, the consistency loss of the first frontal face key point and the second frontal face key point, the priori loss of the coordinates of the second adaptive key point of the driving image, the posture loss of the driving image by the face posture detection network, the facial surface action features generated by the face posture detection network for the source image, and the difference in mouth shape between the output image and the driving image by a pre-trained mouth feature extraction model;

[0232] A target facial image is generated according to the image loss and the face-driven model.

[0233] Specifically, the specific implementation method of the processor 13 for the above computer readable instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0234] In the several embodiments provided herein, it should be understood that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical functional division, and actual implementation may employ other division methods.

[0235] The computer-readable storage medium stores computer-readable instructions, wherein the computer-readable instructions, when executed by the processor 13, are used to implement the following steps:

[0236] Acquire a source image and a driving image, and acquire a face driving model, wherein the face driving model includes a face feature extraction network, an adaptive key point recognition network, a face posture detection network, and an image generation network;

[0237] Inputting the source image into the facial feature extraction network to perform transformation processing on the facial features obtained to obtain transformed features;

[0238] Identifying first adaptive keypoint coordinates from the source image based on the adaptive keypoint recognition network;

[0239] Identifying first frontal facial key points of the source image based on the first adaptive key point coordinates and first facial motion features generated by the face pose detection network for the source image, and identifying second frontal facial key points of the drive image based on the first adaptive key point coordinates and second facial motion features generated by the face pose detection network for the drive image;

[0240] Inputting the transformed features, the second frontal facial key points, and the optical flow features obtained by the first frontal facial key points into the image generation network to obtain an output image;

[0241] The image loss of the output image is generated based on the overall feature loss of the output image and the driving image, the face shape generation loss and mouth shape generation loss of the output image by the image generation network, the local feature loss of the output image and the driving image, the consistency loss of the first frontal face key point and the second frontal face key point, the priori loss of the coordinates of the second adaptive key point of the driving image, the posture loss of the driving image by the face posture detection network, the facial surface action features generated by the face posture detection network for the source image, and the difference in mouth shape between the output image and the driving image by a pre-trained mouth feature extraction model;

[0242] The face-driven model generates a target facial image according to the image loss.

[0243] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0244] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0245] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0246] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices described may also be implemented by a single unit or device through software or hardware. Terms such as first and second are used to indicate names and do not imply any particular order.

[0247] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An image generation method, characterized in that: The image generation method comprises: Acquire a source image and a driving image, and acquire a face driving model, wherein the face driving model includes a face feature extraction network, an adaptive key point recognition network, a face posture detection network, and an image generation network; Inputting the source image into the facial feature extraction network, and transforming the obtained facial features to obtain transformed features, including: obtaining voice data corresponding to the source image; extracting voice features of the voice data; obtaining transformation coefficients that successfully match the voice features from a configuration library; and transforming the facial features based on the transformation coefficients to obtain the transformed features; Based on the adaptive key point recognition network, identifying first adaptive key point coordinates from the source image; detecting a first facial action feature from the source image based on the facial pose detection network, identifying a first frontal facial key point of the source image based on the first adaptive key point coordinates and the first facial action feature, detecting a second facial action feature from the drive image based on the facial pose detection network, and identifying a second frontal facial key point of the drive image based on the first adaptive key point coordinates and the second facial action feature; generating an optical flow feature based on the second frontal facial key point and the first frontal facial key point, and inputting the transformation feature and the optical flow feature into the image generation network to obtain an output image; Generate an image loss for the output image based on an overall feature loss between the output image and the driving image, a face shape generation loss and a mouth shape generation loss of the output image, a local feature loss between the output image and the driving image, a consistency loss between the first frontal face key point and the second frontal face key point, a priori loss of coordinates of the second adaptive key point of the driving image, a posture loss of the driving image, a facial surface motion feature of the source image, and a difference in mouth shape between the output image and the driving image; A target facial image is generated according to the image loss and the face-driven model.

2. The image generation method according to claim 1, wherein: The first facial motion feature includes the rotation matrix and offset of the source image and the facial surface motion feature. The identification formula of the first frontal facial key point is: ； in, represents the key point matrix formed by the first frontal key points, represents the inverse matrix of the rotation matrix of the source image, represents the key point matrix composed of the coordinates of the first adaptive key points, represents the offset, Indicates the surface motion features of the face.

3. The image generation method according to claim 1, wherein: Generating optical flow features based on the second frontal face key points and the first frontal face key points, and inputting the transformation features and the optical flow features into the image generation network to obtain an output image includes: Calculating a matrix difference between a key point matrix formed by the second frontal face key points and a key point matrix formed by the first frontal face key points to obtain the optical flow feature; splicing the transformation features and the optical flow features to obtain input features; Performing iterative convolution processing on the input features to obtain mapping features; Performing a deconvolution operation on the mapped feature map to obtain a deconvolution feature; The deconvolution feature is activated based on a preset activation function to obtain the output image.

4. The image generation method according to claim 1, wherein: The calculation formula of the image loss is: ； ； ； ； ； ； ； ； in, represents the image loss, represents the overall feature loss, represents the face shape generation loss, represents the mouth shape generation loss, represents the local feature loss, represents the consistency loss, represents the prior loss, represents the posture loss, , Indicates the mouth shape difference, 、 are preset weights, represents the driving image, represents the output image, Represents the intermediate feature layer of the pre-trained face recognition model, represents the number of feature pixels of the driving image or the output image extracted by the intermediate feature layer, represents the number of key points of the first frontal face key points or the second frontal face key points, and represents the coordinate value of the first frontal face key point, and represents the coordinate value of the second frontal face key point, represents the number of key points corresponding to the coordinates of the second adaptive key point, Respectively represent any two key points corresponding to the coordinates of the second adaptive key point, represents any key point corresponding to the coordinates of the second adaptive key point, The formula for calculating the average value is: and represents the preset hyperparameters, represents the rotation matrix identified by the pre-trained facial posture model for the driving image, represents the rotation matrix identified by the face pose detection network for the driving image, represents the face surface motion feature, Represents the pre-trained mouth feature extraction model.

5. The image generation method according to claim 4, wherein: Generating a target facial image according to the image loss and the face driving model includes: When the image loss is greater than a minimum loss threshold of the face driving model, adjusting the face driving model based on the image loss until the image loss meets a preset condition, thereby obtaining a target driving model; An output image generated by the target driving model for the source image and the driving image is determined as the target facial image.

6. The image generation method according to claim 5, wherein: After adjusting the face-driven model based on the image loss, the method further includes: Processing the source image and the driving image based on the adjusted face driving model to obtain an updated image; Calculating an update loss of the updated image based on the preset weights; If the loss difference between the image loss and the update loss is less than a preset difference threshold, the preset weight is adjusted to obtain an updated weight.

7. An image generating device, characterized in that: The image generating device comprises: An acquisition unit, configured to acquire a source image and a driving image, and acquire a face driving model, wherein the face driving model includes a face feature extraction network, an adaptive key point recognition network, a face posture detection network, and an image generation network; a transformation unit, configured to input the source image into the facial feature extraction network, and transform the obtained facial features to obtain transformed features, comprising: obtaining voice data corresponding to the source image; extracting voice features of the voice data; obtaining transformation coefficients that successfully match the voice features from a configuration library; and transforming the facial features based on the transformation coefficients to obtain the transformed features; an identification unit, configured to identify first adaptive key point coordinates from the source image based on the adaptive key point identification network; The recognition unit is further configured to detect a first facial action feature from the source image based on the facial pose detection network, identify a first frontal facial key point of the source image based on the first adaptive key point coordinates and the first facial action feature, detect a second facial action feature from the drive image based on the facial pose detection network, and identify a second frontal facial key point of the drive image based on the first adaptive key point coordinates and the second facial action feature; an input unit, configured to generate an optical flow feature based on the second frontal facial key points and the first frontal facial key points, and input the transformation feature and the optical flow feature into the image generation network to obtain an output image; a generating unit, configured to generate an image loss of the output image based on an overall feature loss of the output image and the driving image, a face shape generation loss and a mouth shape generation loss of the output image, a local feature loss of the output image and the driving image, a consistency loss of the first frontal face key point and the second frontal face key point, a priori loss of the coordinates of the second adaptive key point of the driving image, a posture loss of the driving image, a facial surface motion feature of the source image, and a difference in mouth shape between the output image and the driving image; The generating unit is further configured to generate a target facial image according to the image loss and the face driving model.

8. An electronic device, characterized in that: The electronic device comprises: a memory storing computer-readable instructions; and A processor is configured to execute computer-readable instructions stored in the memory to implement the image generation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the image generation method according to any one of claims 1 to 6.

Images