Children long phase prediction method based on multi-modal gene selection

Abstract

The invention discloses a children long phase prediction method based on multi-modal gene selection, and relates to the technical field of artificial intelligence. The child longphase prediction method based on multi-modal gene method selection comprises the following steps: step 1, data preparation: collecting an open source data set and real portrait data mainly including adult male and female, fully covering race, then preprocessing the data, including image size adjustment, face detection, cutting and alignment correction of five sense organ points, and obtaining a face image; according to the child longphase prediction method based on multi-modal gene method selection, face features and gene information are accurately extracted and processed through a multi-module structure of a model, gene selection is guided by using text data, and finally, accurate child longphase prediction is realized through a proper loss function training model, so that after a parent photo is input, the child longphase prediction accuracy is improved. The predicted child portrait image is automatically generated, and the process is efficient and high in accuracy.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly relates to a method for predicting children's appearance based on multi-modal gene selection. Background Art

[0002] In recent years, the technology of artificial intelligence (AI) has developed rapidly and has been widely applied in various industries. From healthcare to financial services, from autonomous driving to smart homes, artificial intelligence is profoundly changing our way of life and work. Especially in the fields of computer vision and multi-modal learning, the application of artificial intelligence has greatly improved the efficiency and accuracy of image recognition and prediction;

[0003] Predicting children's appearance is an important application in the field of computer vision and is widely used in many fields such as healthcare, genetic research, and personal services, with significant scientific research and commercial value. It predicts the future facial features of children by analyzing the facial features of their parents. Traditional methods for predicting children's appearance mainly rely on image data, using the photos of parents for feature extraction and fusion. However, methods that solely rely on image data have limitations in terms of accuracy and diversity, unable to fully utilize other potentially useful information and unable to customize the generation of the desired effects for users. By introducing multi-modal gene learning technology, new breakthroughs can be brought to the solution for predicting children's appearance;

[0004] Traditional methods for predicting children's appearance generally include the following steps:

[0005] Data collection: Collect photos of parents;

[0006] Preprocessing: Preprocess the collected images, including operations such as face extraction, image denoising, and alignment, to ensure the quality and consistency of the images;

[0007] Feature extraction: Use manually designed feature extraction algorithms (such as LBP, HOG, etc.) or deep learning-based feature extraction methods (such as convolutional neural networks) to extract facial features;

[0008] Feature fusion: Fuse the extracted facial features of parents to generate predicted facial features of children;

[0009] Prediction and generation: Generate a facial image of a child based on the fused features;

[0010] Although traditional methods for predicting children's appearance meet the actual application requirements to a certain extent, there are still the following main problems:

[0011] Single data: Only rely on image data, ignoring other modal information (such as gene data, living habits, etc.) that may have an important impact on prediction;

[0012] Insufficient prediction accuracy: Due to the singularity of information, the accuracy and diversity of prediction results are limited;

[0013] Lack of personalization: It is impossible to customize the genetic pattern of a child's appearance (such as the eyes being more like the father's and the nose being more like the mother's);

[0014] To address the above problems, there is an urgent need for a method for predicting a child's appearance based on multi-modal gene selection, which should be able to:

[0015] Intelligent preprocessing and feature extraction: Utilize advanced technologies such as deep learning and multi-modal learning to achieve intelligent preprocessing of multi-modal data and efficient feature extraction, improving the efficiency and accuracy of the prediction model;

[0016] Simulate gene inheritance prediction: Extract the feature of each organ sub-region of the face as a simulated gene, and select specific genes from the simulated genes of the parents for genetic prediction to make the prediction results more accurate;

[0017] Text and image multi-modal guided gene selection: Dynamically guide the selection of genes through text input, thereby achieving customized prediction of a child's appearance;

[0018] In summary, a method for predicting a child's appearance based on multi-modal gene selection can not only solve many problems in traditional methods, but also significantly improve the overall accuracy and customization flexibility of the prediction system, and has broad application prospects and market demand Summary of the Invention

[0019] The purpose of the present invention is to solve at least one of the technical problems existing in the prior art, and provide a method for predicting a child's appearance based on multi-modal gene selection, which can solve the problem that traditional methods for predicting a child's appearance mainly rely on image data, use the photos of parents for feature extraction and fusion. However, the method that simply relies on image data has limitations in accuracy and diversity, cannot make full use of other potentially useful information, and cannot customize the generation of the desired effect of the user.

[0020] To achieve the above object, the present invention provides the following technical solution: A method for predicting a child's appearance based on multi-modal gene selection, including the following steps:

[0021] Step 1: Data preparation

[0022] Collect open-source datasets and real-life portrait data, mainly including adult men and women, covering a comprehensive range of ethnic groups, and then preprocess the data, including resizing images, face detection, cropping, and alignment correction of facial feature points;

[0023] Step 2: Model construction

[0024] The model includes the following sub-modules: a face region segmenter, an image encoder-1, an image encoder-2, a gene encoder, a gene decoder, a text encoder, and an image generator, where:

[0025] Face region segmenter: Receives a face image as input and outputs a pixel-level segmentation result (mask) of 34 regions on the face;

[0026] Image encoder-1: Receives a face image and the 34-region masks corresponding to the face as input, and outputs gene features corresponding to the 34 regions;

[0027] Image encoder-2: Receives a face image and outputs the corresponding features of the face image;

[0028] Gene encoder: Receives the portrait features output by the image encoder-2 and outputs the regional gene features corresponding to the portrait;

[0029] Gene decoder: Receives the regional gene features corresponding to the portrait and outputs the overall portrait features;

[0030] Image generator: Receives portrait features and outputs a generated portrait image;

[0031] Text encoder: Receives text data as input. This text is used to guide the gene selection for the child's appearance and outputs the proportion of gene selection;

[0032] Step 3: Model training

[0033] Design a suitable loss function, and use the training dataset constructed in Step 1 to train the model constructed in Step 2 to obtain a trained model;

[0034] Step Four: Model inference

[0035] Get the trained model in Step 3, input a photo of a man and a photo of a woman, and the model can output the predicted appearance of the child.

[0036] Preferably, the data preparation in Step 1 further includes:

[0037] 1) Download open-source datasets, including the CelebAHQ and MS-Celeb-1M datasets, which include rich face data of adult men and women and cover a comprehensive range of ethnic groups, ensuring the diversity and representativeness of the data;

[0038] 2) Actually collect hundreds of real face data and add them to the training set;

[0039] 3) Traverse the dataset. Use a face detection algorithm, input a face image, and output the rectangular box coordinates Rect of the face in the image and the coordinates landmarks of the facial feature points;

[0040] 4) Crop the face from the image according to Rect and landmarks, and align the face to a size of 128x128 based on landmarks using affine transformation.

[0041] Preferably, the model construction in step 2 further includes:

[0042] 1) Face region segmenter:

[0043] Input: Face image;

[0044] Output: Pixel-level segmentation results (Mask) of 34 regions on the face;

[0045] Structure: It includes a feature extraction sub-module and a segmentation sub-module. The feature extraction sub-module uses the StyleGAN2 model, receives the face image as input, and outputs face features. The segmentation sub-module contains an upsampling layer and a classification layer. The upsampling layer is used to upsample the face features to the same resolution as the face image, and the classification layer is used to classify each pixel point of the upsampled features. There are 34 categories in total, referring to 34 face regions;

[0046] 2) Image encoder-1:

[0047] Input: Face image + corresponding face segmentation Mask result;

[0048] Output: Gene features (Gene) of 34 regions on the face;

[0049] Structure: The main network structure is a convolutional neural network, which receives the face image as input and outputs face features;

[0050] 3) Image encoder-2:

[0051] Input: Face image;

[0052] Output: Face feature encoding;

[0053] Structure: The main network structure is a pre-trained convolutional neural network, which receives the face image as input and outputs face feature encoding;

[0054] 4) Gene encoder:

[0055] Input: Face feature encoding extracted by Image encoder-2;

[0056] Output: Predicted gene features of 34 regions on the face;

[0057] Structure: The network structure includes a custom mean encoding network and a standard deviation encoding network. The network receives face feature encodings as input and outputs the genetic features of 34 regions of the face, including mean features, standard deviation features, and fusion features.

[0058] 5) Gene decoder:

[0059] Input: The genetic features of 34 regions of the face;

[0060] Output: The predicted face feature encoding;

[0061] Structure: The main network structure is a convolutional neural network. The input is the fused gene features obtained after reparameterization of the genetic features of 34 regions, and the output is the predicted face feature encoding.

[0062] 6) Text encoder:

[0063] Input: The guiding text input by the user;

[0064] Output: The weights corresponding to 34 genes;

[0065] Structure: The network contains a text encoder and a classifier. The text encoder directly uses the text encoder in the CLIP model. The input is the text, and the output is the text feature vector. The classifier is a fully connected layer. The input is the text feature vector, and the output is 34 dimensions, respectively representing the weights corresponding to 34 genes.

[0066] 7) Image generator:

[0067] Input: The face features output by the image encoder;

[0068] Output: The restored face image;

[0069] Structure: The pre-trained StyleGAN2 generator model is used.

[0070] Preferably, the loss function in step three is designed as:

[0071] 1) Loss function design:

[0072] Define the face feature encoding similarity loss, and calculate the L2 distance between the face feature encoding w1 output by the image encoder - 2 and the face feature encoding output by the gene decoder as the similarity loss:

[0073] L embedding = ||w1 - w2||2

[0074] Define the gene feature similarity loss, and calculate the L2 distance between the gene feature g output by the image encoder - 1 and the gene feature between g and with mean μ and and variance σ, in two parts:

[0075] The final overall loss is the sum of the above two losses:

[0076] L total = L gene + L embedding

[0077] Preferably, the training steps of the training model in step three are:

[0078] In the above network structure, the modules we need to train are: Image Encoder - 1, Gene Encoder, Gene Decoder, and Text Encoder, and the modules with fixed parameters are: Face Region Extractor, Image Encoder - 2, and Image Generator;

[0079] The training is generally divided into three parts:

[0080] The first part is the joint training of Image Encoder - 1 and Gene Decoder;

[0081] A1. Input a face image I, and use the face region segmenter to extract the face contour M;

[0082] A2. Input the face contour M and the face image I into Image Encoder - 1 together to obtain the corresponding gene feature G;

[0083] A3. Input the gene feature G into the gene decoder to output the predicted face feature encoding F1;

[0084] A4. Input the face image I directly into Image Encoder - 2 to output the actual face feature encoding F2.

[0085] A5. Use the L embedding loss function to calculate the loss between F1 and F2, and use the ADAM optimizer to optimize the network parameters of Image Encoder - 1 and Gene Decoder.

[0086] Preferably, the second part is the training of Image Encoder - 1 and Gene Encoder;

[0087] B1. Input a face image I, and use the face region segmenter to extract the face contour M;

[0088] B2. Input the face contour M and the face image I into Image Encoder - 1 together to obtain the corresponding gene feature G1;

[0089] B3. Input the face image I directly into Image Encoder - 2 to output the actual face feature encoding F2;

[0090] B4. Input the feature F2 into the gene encoder and output the corresponding actual gene feature G2;

[0091] B5. Use L gene The loss function calculates the loss between G1 and G2, and uses the ADAM optimizer to optimize the network parameters of the image encoder-1 and the gene encoder.

[0092] Preferably, the third part is the training of the text encoder;

[0093] C1. Prepare a set of descriptive texts. Each text sentence is used to describe which parts are expected to look like the mother and which parts look like the father. For example: "The nose is more like the mother, and the forehead is more like the father". Each text sentence has a floating-point number array label with a length of 34, which is used to label the 34 weights corresponding to the text;

[0094] C2. For each descriptive text T and its corresponding label L, input the descriptive text T into the text encoder to obtain the text feature F T ;

[0095] C3. Input F T into a fully connected layer and output a 34-dimensional vector V T ;

[0096] C4. Calculate the MSE loss between V T and the label L, and use the ADAM optimizer to optimize the network parameters of the text encoder and the fully connected layer;

[0097] C5. Connect the fully connected layer to the end of the text encoder to merge into a complete text encoder.

[0098] Preferably, the specific steps of the model inference in step four are as follows:

[0099] 1) Input the father image I f and the mother image I m into the image encoder-2 respectively to obtain the image features W f and W m , and then input these two features into the gene encoder to obtain the gene features G f and G m ;

[0100] 2) Input a descriptive text T. The content format of the text refers to the training part of the text encoder in the model training section. Input T into the text encoder to obtain the weights V T of the 34 gene selections;

[0101] 3) Based on the preset gene library, use the weights VT For gene feature G f and G m perform weighted fusion to obtain the fused gene feature G fusion ;

[0102] 4) Input the fused gene feature G fusion into the gene decoder to obtain the decoded face feature encoding F fusion ;

[0103] 5) Input the face feature F fusion into the image generator to output the final predicted child image I child .

[0104] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0105] This method for predicting a child's appearance selected based on the multi-modal gene method can efficiently predict a child's appearance by collecting and preprocessing various portrait data and constructing a multi-module model including a face region segmenter, an image encoder, a gene encoder, a gene decoder, a text encoder, and an image generator. In the specific implementation process, first, data preparation is carried out to ensure that the data set comprehensively covers different ethnic groups and the images are standardized. Then, through the multi-module structure of the model, face features and gene information are accurately extracted and processed, and text data is used to guide gene selection. Finally, the model is trained through a suitable loss function to achieve accurate prediction of a child's appearance. In this way, after inputting the parents' photos, a predicted child portrait can be automatically generated, and the process is efficient and highly accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] The present invention will be further described below with reference to the drawings and embodiments:

[0107] Figure 1 is a schematic diagram of the outflow framework of the present invention;

[0108] Figure 2 is a detailed flowchart of model training in the present invention;

[0109] Figure 3 is a detailed flowchart of model testing in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0110] This part will describe in detail the specific embodiments of the present invention. The preferred embodiments of the present invention are shown in the drawings. The drawings are used to supplement the description of the text part of the specification, enabling people to intuitively and vividly understand each technical feature and the overall technical solution of the present invention, but it should not be construed as a limitation on the protection scope of the present invention.

[0111] In the description of the present invention, it should be understood that for the orientation description, such as the orientation or positional relationship indicated by up, down, front, back, left, right, etc., it is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present invention.

[0112] In the description of the present invention, "greater than", "less than", "exceeding", etc. are understood not to include the present number, and "above", "below", "within", etc. are understood to include the present number. If the first and second are described only for the purpose of distinguishing technical features, they should not be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence relationship of the indicated technical features.

[0113] In the description of the present invention, unless otherwise clearly defined, words such as "set", "installed", "connected", etc. should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0114] Please refer to Figures 1-3 , the present invention provides a technical solution: a method for predicting children's looks based on multi-modal gene selection, including the following steps:

[0115] Step 1: Data preparation

[0116] Collect open-source datasets and real-life portrait data, mainly including adult men and women, covering a comprehensive range of ethnic groups. Then, preprocess the data, including resizing the images, face detection, cropping, and aligning and correcting the five facial feature points. The specific steps are as follows:

[0117] 1) Download open-source datasets, including the CelebAHQ and MS-Celeb-1M datasets, which include rich face data of adult men and women, covering a comprehensive range of ethnic groups, and can ensure the diversity and representativeness of the data;

[0118] 2) Actually collect hundreds of real face data and add them to the training set; [[ID=2⑤]]

[0119] 3) Traverse the dataset. Use a face detection algorithm, input a face image, and output the rectangular box coordinates Rect of the face in the image and the coordinates landmarks of the five facial feature points;

[0120] 4) According to Rect and landmarks, crop the face from the image, and based on landmarks, use affine transformation to align the face to a size of 128x128;

[0121] Step 2: Model construction

[0122] The model includes the following sub - modules: face region segmenter, image encoder - 1, image encoder - 2, gene encoder, gene decoder, text encoder, and image generator, where:

[0123] Face region segmenter: Receives a face image as input and outputs a pixel - level segmentation result (mask) of 34 regions on the face;

[0124] Image encoder - 1: Receives a face image and the 34 - region mask corresponding to the face as input, and outputs gene features corresponding to the 34 regions;

[0125] Image encoder - 2: Receives a face image and outputs the corresponding features of the face image;

[0126] Gene encoder: Receives the portrait features output by image encoder - 2 and outputs the regional gene features corresponding to the portrait;

[0127] Gene decoder: Receives the regional gene features corresponding to the portrait and outputs the overall portrait features;

[0128] Image generator: Receives portrait features and outputs the generated portrait image;

[0129] Text encoder: Receives text data as input. This text is used to guide the gene selection for the child's appearance and outputs the proportion of gene selection;

[0130] 1) Face region segmenter:

[0131] Input: Face image;

[0132] Output: Pixel - level segmentation result (Mask) of 34 regions on the face;

[0133] Structure: It includes a feature extraction sub - module and a segmentation sub - module. The feature extraction sub - module uses the StyleGAN2 model, receives a face image as input, and outputs face features. The segmentation sub - module contains an up - sampling layer and a classification layer. The up - sampling layer is used to up - sample the face features to the same resolution as the face image, and the classification layer is used to classify each pixel point of the up - sampled features. There are 34 categories in total, referring to 34 face regions;

[0134] 2) Image encoder - 1:

[0135] Input: Face image + corresponding face segmentation Mask result;

[0136] Output: Gene features (Gene) of 34 regions of the face;

[0137] Structure: The main network structure is a convolutional neural network, which receives a face image as input and outputs face features;

[0138] 3) Image Encoder - 2:

[0139] Input: Facial image;

[0140] Output: Facial feature encoding;

[0141] Structure: The main network structure is a pre - trained convolutional neural network that receives a facial image as input and outputs a facial feature encoding;

[0142] 4) Gene Encoder:

[0143] Input: Facial feature encoding extracted by Image Encoder - 2;

[0144] Output: Gene features of 34 regions of the face predicted;

[0145] Structure: The network structure includes a custom mean encoding network and a standard deviation encoding network. The network receives the facial feature encoding as input and outputs the gene features of 34 regions of the face, including mean features, standard deviation features, and fusion features;

[0146] 5) Gene Decoder:

[0147] Input: Gene features of 34 regions of the face;

[0148] Output: Predicted facial feature encoding;

[0149] Structure: The main network structure is a convolutional neural network. The input is the fusion gene feature obtained after re - parameterization of the gene features of 34 regions, and the output is the predicted facial feature encoding;

[0150] 6) Text Encoder:

[0151] Input: Guiding text input by the user;

[0152] Output: Weights corresponding to 34 genes;

[0153] Structure: The network contains a text encoder and a classifier. The text encoder directly uses the text encoder in the CLIP model. It inputs the text and outputs a text feature vector. The classifier is a fully - connected layer. It inputs the text feature vector and the output is 34 - dimensional, representing the weights corresponding to 34 genes respectively;

[0154] 7) Image Generator:

[0155] Input: Facial features output by the image encoder;

[0156] Output: Restored facial image;

[0157] Structure: The pre-trained StyleGAN2 generator model is used;

[0158] Step 3: Model training

[0159] Design a suitable loss function, and use the training dataset constructed in Step 1 to train the model constructed in Step 2 to obtain a trained model;

[0160] 1) Loss function design:

[0161] Define the face feature encoding similarity loss, and calculate the L2 distance between the face feature encoding w1 output by the image encoder-2 and the face feature encoding output by the gene decoder as the similarity loss:

[0162] L embedding = ||w1 - w2||2

[0163] Define the gene feature similarity loss, and calculate the L2 distance between the gene feature g output by the image encoder-1 and the gene feature output by the gene encoder, and both g and have mean u, and variance σ, in two parts:

[0164] The final overall loss is the sum of the above two losses:

[0165] L total = L gene + L embedding

[0166] 2) Training steps:

[0167] In the above network structure, the modules we need to train are: image encoder-1, gene encoder, gene decoder, and text encoder, and the modules with fixed parameters are: face region extractor, image encoder-2, and image generator;

[0168] The training is generally divided into three parts:

[0169] The first part: Joint training of the image encoder-1 and the gene decoder;

[0170] A1. Input a face image I, and use the face region segmenter to extract the face contour M;

[0171] A2. Input the face contour M and the face image I into the image encoder-1 together to obtain the corresponding gene feature G;

[0172] A3. Input the gene feature G into the gene decoder and output the predicted face feature encoding F1;

[0173] A4. Directly input the face image I into the image encoder - 2 to output the actual face feature encoding F2.

[0174] A5. Use L embedding The loss function calculates the loss between F1 and F2, and uses the ADAM optimizer to optimize the network parameters of the image encoder - 1 and the gene decoder;

[0175] The second part: Training of the image encoder - 1 and the gene encoder;

[0176] B1. Input a face image I, and use the face region segmenter to extract the face contour M;

[0177] B2. Input the face contour M and the face image I together into the image encoder - 1 to obtain the corresponding gene feature G1;

[0178] B3. Directly input the face image I into the image encoder - 2 to output the actual face feature encoding F2;

[0179] B4. Input the feature F2 into the gene encoder to output the corresponding actual gene feature G2;

[0180] B5. Use L gene The loss function calculates the loss between G1 and G2, and uses the ADAM optimizer to optimize the network parameters of the image encoder - 1 and the gene encoder;

[0181] The third part: Training of the text encoder;

[0182] C1. Prepare a set of descriptive texts. Each text sentence is used to describe which parts are expected to look like the mother and which parts look like the father. For example: "The nose is more like the mother, and the forehead is more like the father". Each text sentence has a floating - point number array label of length 34, used to label the 34 weights corresponding to the text;

[0183] C2. For each descriptive text T and its corresponding label L, input the descriptive text T into the text encoder to obtain the text feature F T ;

[0184] C3. Input F T into a fully - connected layer to output a 34 - dimensional vector V T ;

[0185] C4. Calculate the MSE loss between V T and the label L, and use the ADAM optimizer to optimize the network parameters of the text encoder and the fully - connected layer;

[0186] C5. Connect the fully connected layer to the end of the text encoder and merge them into a complete text encoder;

[0187] Step 4: Model Inference

[0188] Take the model trained in Step 3, input a photo of a man and a photo of a woman, and the model can output the predicted appearance of the child. The specific steps are as follows:

[0189] 1) Input the father image I f and the mother image I m into the image encoder - 2 respectively to obtain the image features W f and W m , and then input these two features into the gene encoder to obtain the gene features G f and G m ;

[0190] 2) Input a descriptive text T. The content format of the text refers to the training part of the text encoder in the model training section. Input T into the text encoder to obtain the weights V T for 34 gene selections;

[0191] 3) Based on the pre - set gene library, use the weights V T to perform weighted fusion on the gene features G f and G m to obtain the fused gene feature G fusion ;

[0192] 4) Input the fused gene feature G fusion into the gene decoder to obtain the decoded face feature encoding F fusion ;

[0193] 5) Input the face feature F fusion into the image generator to output the final predicted child image I child .

[0194] Working principle: By collecting and pre - processing various portrait data, a multi - module model including a face region segmenter, an image encoder, a gene encoder, a gene decoder, a text encoder, and an image generator is constructed, which can efficiently predict the appearance of a child. In the specific implementation process, first, data preparation is carried out to ensure that the dataset comprehensively covers different ethnic groups and the images are standardized;

[0195] Then, through the multi - module structure of the model, face features and gene information are accurately extracted and processed, and text data is used to guide gene selection;

[0196] Finally, the model is trained with a suitable loss function to achieve accurate prediction of the child's appearance. In this way, after inputting the photos of the parents, the predicted portrait of the child can be automatically generated, with high efficiency and accuracy.

[0197] The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present invention within the scope of knowledge possessed by those of ordinary skill in the art.

Claims

1. A method for predicting a child's appearance based on multi-modal gene selection, comprising the following steps: Step 1: Data preparation Collect open-source datasets and real-life portrait data, mainly including adult men and women, covering a comprehensive range of ethnic groups. Then, preprocess the data, including resizing the images, face detection, cropping, and aligning and correcting the five facial feature points; Step 2: Model construction The model includes the following sub-modules: a face region segmenter, an image encoder-1, an image encoder-2, a gene encoder, a gene decoder, a text encoder, and an image generator, where: Face region segmenter: Receives a face image as input and outputs a pixel-level segmentation result (mask) of 34 regions on the face; Image encoder-1: Receives a face image and the 34-region mask corresponding to the face as input and outputs the gene features corresponding to the 34 regions; Image encoder-2: Receives a face image and outputs the corresponding features of the face image; Gene encoder: Receives the portrait features output by the image encoder-2 and outputs the regional gene features corresponding to the portrait; Gene decoder: Receives the regional gene features corresponding to the portrait and outputs the overall portrait features; Image generator: Receives portrait features and outputs the generated portrait image; Text encoder: Receives text data as input, where the text is used to guide gene selection for the child's appearance, and outputs the proportion of gene selection; Step 3: Model training Design a suitable loss function, and use the training dataset constructed in Step 1 to train the model constructed in Step 2 to obtain a trained model; Step 4: Model inference Take the model trained in Step 3, input a photo of a man and a photo of a woman, and the model can output the predicted appearance of the child.

2. The method for predicting the appearance of a child based on multi-modal gene selection according to claim 1, wherein: The data preparation described in Step 1 further includes: 1) Download open-source datasets, including the CelebAHQ and MS-Celeb-1M datasets, which include rich face data of adult men and women, covering a comprehensive range of ethnic groups, ensuring the diversity and representativeness of the data; 2) Actively collect hundreds of real face data and add them to the training set; 3) Traverse the dataset. Use a face detection algorithm, input a face image, and output the rectangular frame coordinates Rect of the face in the image and the coordinates landmarks of the five facial feature points; 4) According to Rect and landmarks, crop the face from the image, and based on landmarks, use affine transformation to align the face to a size of 128x128.

3. A method for predicting a child's appearance based on multi-modal gene selection according to claim 1, characterized in that: The model construction described in Step 2 further includes: 1) Face region segmenter: Input: Face image; Output: A pixel-level segmentation result (Mask) of 34 regions on the face; Structure: Includes a feature extraction sub-module and a segmentation sub-module. The feature extraction sub-module uses the StyleGAN2 model, receives a face image as input, and outputs face features. The segmentation sub-module includes an upsampling layer and a classification layer. The upsampling layer is used to upsample the face features to the same resolution as the face image, and the classification layer is used to classify each pixel point of the upsampled features. There are 34 categories, referring to 34 face regions; 2) Image encoder-1: Input: Facial image + corresponding facial segmentation Mask result; Output: Gene features (Gene) of 34 regions of the face; Structure: The main network structure is a convolutional neural network that receives a facial image as input and outputs facial features; 3) Image Encoder - 2: Input: Facial image; Output: Facial feature encoding; Structure: The main network structure is a pre - trained convolutional neural network that receives a facial image as input and outputs facial feature encoding; 4) Gene Encoder: Input: Facial feature encoding extracted by Image Encoder - 2; Output: Predicted gene features of 34 regions of the face; Structure: The network structure includes a custom mean - encoding network and a standard - deviation encoding network. The network receives facial feature encoding as input and outputs gene features of 34 regions of the face, including mean features, standard - deviation features, and fusion features; 5) Gene Decoder: Input: Gene features of 34 regions of the face; Output: Predicted facial feature encoding; Structure: The main network structure is a convolutional neural network. The input is the fusion gene feature obtained by re - parameterizing the gene features of 34 regions, and the output is the predicted facial feature encoding; 6) Text Encoder: Input: Guiding text input by the user; Output: Weights corresponding to 34 genes; Structure: The network contains a text encoder and a classifier. The text encoder directly uses the text encoder in the CLIP model. It inputs text and outputs a text feature vector. The classifier is a fully - connected layer. It inputs the text feature vector and the output is 34 - dimensional, representing the weights corresponding to 34 genes respectively; 7) Image Generator: Input: Facial features output by the Image Encoder; Output: Restored facial image; Structure: The pre - trained StyleGAN2 generator model is used.

4. A method for predicting a child's appearance based on multi-modal gene selection according to claim 1, characterized in that: The loss function described in step three is designed as: 1) Loss function design: Define the facial feature encoding similarity loss, calculate the L2 distance between the facial feature encoding w1 output by Image Encoder - 2 and the facial feature encoding output by the Gene Decoder as the similarity loss: L embedding = ||w1 - w2||2 Define the gene feature similarity loss, and calculate the L2 distance between the gene feature g output by the image encoder-1 and the gene feature output by the gene encoder. The g and both have a mean value u, and a variance σ, in two parts: ​ The final overall loss is the sum of the above two losses: L total = L gene + L embedding 5. A method for predicting a child's appearance based on multi-modal gene selection according to claim 1, characterized in that: The training steps of the training model described in step three are: In the above network structure, the modules we need to train are: Image Encoder - 1, Gene Encoder, Gene Decoder, and Text Encoder. The modules with fixed parameters are: Facial Region Extractor, Image Encoder - 2, and Image Generator; The training is generally divided into three parts: The first part is the joint training of Image Encoder - 1 and Gene Decoder; A1. Input a facial image I, and use the facial region segmenter to extract the facial contour M; A2. Input the facial contour M and the facial image I into Image Encoder - 1 together to obtain the corresponding gene feature G; A3. Input the gene feature G into the Gene Decoder to output the predicted facial feature encoding F1; A4. Input the facial image I directly into Image Encoder - 2 to output the actual facial feature encoding F2. A5. Use L embedding The loss function calculates the loss between F1 and F2, and uses the ADAM optimizer to optimize the network parameters of the image encoder-1 and the gene decoder.

6. The method for predicting the appearance of a child based on multi-modal gene selection according to claim 5, wherein: The second part is the training of Image Encoder - 1 and Gene Encoder; B1. Input a face image I, and use a face region segmenter to extract the face contour M; B2. Input the face contour M and the face image I into Image Encoder - 1 together to obtain the corresponding gene feature G1; B3. Input the face image I directly into Image Encoder - 2 to output the actual face feature encoding F2; B4. Input the feature F2 into the gene encoder to output the corresponding actual gene feature G2; B5. Use L gene The loss function calculates the loss between G1 and G2, and uses the ADAM optimizer to optimize the network parameters of the image encoder-1 and the gene encoder.

7. A method for predicting a child's appearance based on multi-modal gene selection according to claim 5, characterized in that: The third part is the training of the text encoder; C1. Prepare a set of descriptive texts. Each text sentence is used to describe which parts are expected to look like the mother and which parts look like the father. For example: "The nose is more like the mother, and the forehead is more like the father". Each text sentence has a floating - point number array label of length 34, which is used to label the 34 weights corresponding to the text; C2. For each description text T and its corresponding label L, input the description text T into the text encoder to obtain the text feature F T ; C3. Input F T into a fully connected layer and output a 34-dimensional vector V T ; C4. Calculate V T Calculate the MSE loss between it and label L, and use the ADAM optimizer to optimize the network parameters of the text encoder and the fully connected layer; C5. Connect the fully - connected layer to the end of the text encoder to combine it into a complete text encoder.

8. A method for predicting a child's appearance based on multi-modal gene selection according to claim 5, characterized in that: The specific steps of the model inference described in Step 4 are as follows: 1) Input the father image I f and the mother image I m into the image encoder - 2 respectively to obtain the image features W f and W m , and then input these two features into the gene encoder to obtain the gene features G f and G m ; 2) Input a descriptive text T, the content format of the text refers to the training part of the text encoder in the model training section, input T into the text encoder, and obtain the weights V for the selection of 34 genes T ; 3) Based on the preset gene library, use the weight V T to perform weighted fusion on the gene feature G f and G m to obtain the fused gene feature G fusion ; 4) Input the fusion gene feature G fusion into the gene decoder to obtain the decoded face feature encoding F fusion ; 5) Input the face feature F fusion into the image generator, and output the final predicted child image I child .

Images