Method for predicting whole-growth-period image based on crop genotypes of artificial intelligence

Abstract

The invention discloses a method for predicting images in a whole growth period based on crop genotypes of artificial intelligence. The method comprises the steps of collecting crop images, processing the crop images, constructing a multi-layer perceptron neural network and a condition control generative adversarial network for training so as to learn the mapping relation of crop genotypes, phenotypes and images in the whole growth period, obtaining a crop gene-phenotype prediction model and a phenotype-image prediction model, and finally obtaining the crop genotype-phenotype prediction model and the phenotype-image prediction model through the two models. Based on the genotype of a to-be-predicted crop, a visual result of the whole growth period of the crop in an ideal environment under the regulation of the gene is predicted. According to the method, based on an artificial intelligence technology, a multi-dimensional space multi-layer perception network and a condition control generative adversarial network are constructed, and vivid and accurate visual prediction of crops is realized by deeply utilizing mapping rules contained in crop genes, phenotypes and image big data.

Description

Technical Field

[0001] The present invention relates to the technical fields of smart agriculture, artificial intelligence, crop informatics and smart crop breeding, and in particular to a method for predicting crop genotypes throughout their growth period based on artificial intelligence. Background Art

[0002] Traditional crop breeding technology has developed slowly, and the growth rate of yield cannot meet future food needs. With the rapid development of high-throughput omics technology and artificial intelligence technology, a large amount of crop genomics data and high-throughput phenotypic omics data have been accumulated. At present, the effective use of genomics technology plays a vital role in solving problems related to rapid breeding and omics research and promoting sustainable agricultural development.

[0003] Genomic prediction can predict the field phenotype of crops based on genotypes before planting. Existing genomic prediction methods include classic statistical learning models such as RRBLUP and BayesA, as well as deep learning methods such as DNNGP. DNNGP performs PCA dimensionality reduction on the genotype data of the whole genome and uses the features after dimensionality reduction to achieve genomic prediction. Existing genomic prediction methods have high computational complexity, low genomic prediction accuracy, and are challenging to predict phenotypes throughout the growth period.

[0004] Furthermore, converting crop phenotypic information into intuitive and predictable growth images, that is, realizing crop profiling, has become a hot topic and difficulty in current research. This technology can simulate the growth of crops with different genotypes at different growth stages, provide farmers with scientific planting guidance and management suggestions, and significantly improve the level of intelligent and precise agricultural production.

[0005] The invention patent with publication number CN115880433A collects environmental data and crop appearance images, builds a crop environmental data prediction network and a crop growth image prediction network, and realizes crop simulation visualization. The invention patent with publication number CN118097372A proposes a crop growth visualization prediction method, which realizes the portrait of future crops through crop appearance images. These studies show that it is feasible to predict crop appearance images through phenotypic information through existing machine vision methods. Summary of the invention

[0006] At present, artificial intelligence technology has made significant progress in generating realistic images. However, in the field of agriculture, it is still difficult to directly convert crop genotype data into corresponding appearance images. This is because gene expression patterns show high specificity and dynamics in different tissues and different growth and development stages, and crop phenotypes are jointly shaped by multiple genes and their complex interaction networks, as well as external environmental factors. This makes it very difficult to build a model to learn nonlinear, multi-factor association mechanisms from static genotype data and accurately predict crop appearance images.

[0007] Currently, there is no mature dataset or algorithm that can directly establish an accurate mapping relationship between genotype data and crop appearance images. This is mainly due to the lack of a sufficient amount of high-quality sample data to train the model, as well as the lack of advanced algorithms that can process such highly complex and high-dimensional data.

[0008] The present invention discloses a method for predicting crop genotype full growth period images based on artificial intelligence, the method comprising: Step S1, preparing a plurality of crop seeds of known varieties, planting them in different pots under the same growth environment, and using a camera to regularly capture crop images during the complete growth period of the crops to obtain original images of the crops during the entire growth period; Step S2, training a crop semantic segmentation model through manual annotation, and using the model to process the original crop image obtained in step S1, removing irrelevant content in the image, retaining only the area where the crop is located, and obtaining a crop image; Step S3, performing image processing on the crop image obtained in step S2 to extract phenotypic information of the crop at different growth stages; Step S4, obtaining the genotype sequence of the crop by sequencing technology, sorting out the one-to-one corresponding genotype sequence and crop phenotype information, constructing a multidimensional space multilayer perceptron neural network MDMLP, training to obtain a genotype-phenotype prediction model, and establishing a mapping relationship between the genotype and phenotype of the crop throughout its growth period; Step S5, sorting out the crop phenotypes and corresponding image data of each growth period, constructing a condition-controlled crop image generator and a discriminator integrating condition information based on the improved StyleGAN high-resolution generative adversarial network model, and designing corresponding loss functions, training to obtain the phenotype-image prediction model of each growth period, and establishing a mapping relationship between the crop phenotypes and crop images of each growth period; Step S6, inputting the genotype sequence to be predicted into the genotype-phenotype prediction model obtained in step S4 to obtain the predicted crop phenotype; Step S7, inputting the crop phenotype predicted in step S6 and the corresponding growth period information into the phenotype-image prediction model obtained in step S5 to obtain a predicted crop image.

[0009] Furthermore, the method in step S2 of the present invention includes: Step S21, selecting different plants with large growth differences and selecting crop images at different time points as training sets; Step S22, performing pixel-level semantic annotation on the area where the crop is located in the training set image to obtain annotation information corresponding to the image; Step S23, inputting the training set images and corresponding annotation information into a semantic segmentation network for training to obtain a crop semantic segmentation model; Step S24, using the crop semantic segmentation model to segment the crop original image acquired in step S1 to obtain a crop binary image; Step S25, using the crop binary image obtained in step S24 to perform mask processing on the original crop image, to obtain a crop image that only retains the area where the crop is located.

[0010] Furthermore, the method in step S4 of the present invention includes: Step S41, sequencing crops to obtain single nucleotide variant site data SNP of all crop genomes, and randomly screening m SNP sites are selected to ensure that the whole genome SNP distribution density after screening is consistent with the whole genome SNP distribution density before screening, and the genotype sequence for predicting crop phenotypes is obtained; Step S42, constructing a multidimensional space multilayer perceptron neural network MDMLP to achieve the fusion of local and global features between the extracted SNP sites; Step S43, using the neural network designed in step S42 to learn the one-to-one corresponding genotype and phenotype data of each period, to obtain a genotype-phenotype prediction model for each growth period.

[0011] Furthermore, the method for constructing the MDMLP in step S42 of the present invention includes: Step S421, set the patch embedding layer, and use the patch embedding layer to embed the input m The SNP site features are converted into word embeddings of different dimensions including height, channel and width; Step S422, set the MDblock layer, respectively for the word embedding of different dimensions of height, channel and width, use the fully connected neural network to construct the height mixed MLP layer, channel mixed MLP layer and token width mixed MLP layer respectively, to further realize the local feature extraction of different dimensions; the specific implementation process is shown in formula (1), (1) In formula (1), yrepresents the feature vector extracted by the hybrid MLP layer, x represents the encoded word embedding features, W 1 and W 2 represents the weights of the two layers of linear mapping, b 1 and b 2 is the bias of the two-layer linear mapping, σ Represents the activation function Gelu , the captured linear features can be nonlinearized through the activation function. The activation function calculation is shown in formula (2). (2) In the above formula, X Indicates compliance N (0,1) Normally distributed random numbers, P ( X ≤ x ) represents a random variable X Less than or equal to x The probability that X~N (0,1), so P ( X ≤ x ) refers to the cumulative distribution function of the standard normal distribution; Step S423, setting a global average pooling layer, and using global average pooling to achieve global feature fusion based on features extracted from different dimensions; Step S424, setting a fully connected neural network layer, and using a fully connected neural network based on the globally fused features to further improve the effect of feature extraction; Step S425, adding a dropout regularization method and nonlinear activation functions such as ReLU to the network to prevent overfitting, and finally constructing an MDMLP network.

[0012] Furthermore, the method in step S5 of the present invention includes: Step S51, obtaining a high-resolution generative adversarial network based on StyleGAN with the conditional vector introduced; Step S52, in the discriminator, construct a multi-scale discriminator network, use a downsampling convolution layer to extract image features, and obtain a feature vector; Step S53, the phenotype data is input into the discriminator network as a conditional vector, the size of the conditional vector is 1×1×S, S is the number of phenotypes, the first two dimensions of the vector are expanded by replication, and a 4×4×S conditional vector is obtained. At this time, the expanded conditional vector is consistent with the image feature vector obtained after downsampling in the first two dimensions, and the image feature vector size is 4×4×C, C is the number of channels. The expanded conditional vector and the image feature vector are aligned in the first two dimensions, and the third dimension is superimposed to obtain a fused feature vector with a size of 4×4×(S+C), then the fused features are extracted by convolution, and finally the discriminator score is obtained using a fully connected layer; Step S54, calculate the cross entropy between the discriminator score and 0 or 1 as the loss function of the generative adversarial network, as shown in formula (3), (3) In the formula D is the discriminator, G For the generator, E is the mathematical expectation, I To input a real image, y S is a conditional vector of size 1×1×S, z is Gaussian distributed noise, V ( D , G ) represents the difference between the generated rice image and the real rice image, using the cross entropy loss function of the two categories of true and false, including and Two parts: Represents a fixed generator G Parameter training discriminator D parameters, by maximizing the cross entropy loss V ( D , G ) to update the discriminator D parameter; represents a fixed discriminator D Parameter training generator G Parameters, generator G In the discriminator D Maximize the cross entropy loss V ( D , G ), minimize the cross entropy loss.

[0013] The present invention constructs a complete process from crop planting to image acquisition, and also designs all necessary preprocessing steps in detail to ensure the quality and accuracy of the data. By introducing the MDMLP artificial intelligence algorithm, the key phenotypic values ​​of crops throughout their growth period are accurately predicted, covering all stages of crop growth, providing comprehensive data support for agricultural research and production.

[0014] Based on the accurately predicted phenotypic values ​​for each growth period, the present invention further innovatively applies the improved algorithm of StyleGAN to draw crop images. This method not only retains the advantages of StyleGAN in generating high-quality, high-resolution images, but also combines the crop-specific phenotypic information to make the generated images closer to reality, and can intuitively display the growth status of crops in different growth periods, providing an intuitive and reliable visual reference for agricultural decision makers.

[0015] The invention has greatly promoted the intelligent and automated process of agricultural production. By constructing an intelligent prediction model and image generation system, agricultural workers can obtain a full picture of crop growth without field observation, effectively reducing labor intensity and improving work efficiency. The invention also provides important technical support for the research of related disciplines such as crop genetics and plant physiology, helps to reveal the molecular mechanism of crop growth and development, and provides new ideas and methods for crop breeding and genetic improvement. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is the overall technical scheme diagram of the present invention.

[0017] Figure 2 This is the MDMLP network structure diagram constructed by the present invention.

[0018] Figure 3 This is a structural diagram of the improved StyleGAN network discriminator of the present invention.

[0019] Figure 4 This is an example diagram of the prediction effect of a specific embodiment of the present invention. DETAILED DESCRIPTION

[0020] An exemplary embodiment of the present invention is now described in detail, and this detailed description should not be considered as a limitation of the present invention, but should be understood as a more detailed description of certain aspects of the present invention. It should be understood that the terms described in the present invention are only for describing a particular embodiment, rather than being limitations of the present invention. For the parameter value range in the present invention, it should be understood that the upper and lower limits of the range and their intermediate values ​​are specifically disclosed.

[0021] Unless otherwise specified, all techniques and terms used herein have the same meanings as commonly understood by those skilled in the art in the art to which the invention pertains. Although only one specific exemplary embodiment has been described herein, methods and materials similar or equivalent to those described herein may also be used in the practice or testing of the invention.

[0022] like Figure 1The flowchart of the method for predicting crop genotypes throughout the growth period based on artificial intelligence provided by this embodiment is shown, including: Step S1, preparing seeds of 533 rice varieties with known genotype sequences, selecting 4 healthy seeds of each variety, planting them in different pots under the same growth environment, and using a camera to capture rice images every 6 days for 8 times two weeks after sowing to obtain original images of rice in the whole growth period; Step S2, training a rice semantic segmentation model through manual annotation, and using the model to process the original rice image obtained in step S1, removing irrelevant content in the image, retaining only the area where the rice is located, and obtaining a rice image; Step S3, using a high-throughput crop phenotyping device RAP to process the rice image obtained in step S2, extracting 13 phenotypic information of rice, including 6 grayscale histogram features, plant projection area, height, width, projection area ratio to circumscribed rectangle area, perimeter ratio to projection area, fractal dimension and green part projection area, and normalizing all data; Step S4, obtaining the genotype sequence of rice by sequencing technology, sorting out the one-to-one correspondence between the genotype sequence and phenotype information of rice, constructing a multidimensional space multilayer perceptron neural network MDMLP, training to obtain a genotype-phenotype prediction model, and establishing a mapping relationship between the genotype and phenotype of rice throughout its growth period; Step S5, sorting out the crop phenotypes and corresponding image data of each growth period, constructing a condition-controlled crop image generator and a discriminator integrating condition information based on the improved StyleGAN high-resolution generative adversarial network model, and designing corresponding loss functions, training to obtain the phenotype-image prediction model of each growth period, and establishing a mapping relationship between the crop phenotypes and crop images of each growth period; Step S6, inputting the genotype sequence to be predicted and the expected rice growth period information into the genotype-phenotype prediction model obtained in step S4 to obtain the predicted rice phenotype of the period; Step S7, inputting the rice phenotype predicted in step S6 and the corresponding growth period information into the phenotype-image prediction model obtained in step S5 to obtain a predicted rice image. The prediction effect is shown in the following example: Figure 4 shown.

[0023] Furthermore, the method in step S2 of the present invention includes: Step S21, selecting plants with large growth differences at different time points, a total of 150 original rice images as a training set; Step S22, marking the pixels outside the rice area in the training set image as black, and saving the marked image, which is the marking information corresponding to the original image; Step S23, inputting the training set images and corresponding annotation information into the Segformer semantic segmentation network for training to obtain a rice semantic segmentation model; Step S24, using the rice semantic segmentation model to segment the original rice image acquired in step S1, after segmentation, the model sets the rice plant area to white and other areas to black, to obtain a rice binary image; Step S25, using the rice binary image obtained in step S24 to perform mask processing on the original rice image, setting the pixels outside the rice area to black, and obtaining an image that only retains the rice area.

[0024] Furthermore, the method in step S4 of the present invention includes: Step S41, obtaining single nucleotide variant site data (SNP) of the whole genome of 533 rice varieties by sequencing, ensuring that the whole genome SNP distribution density after screening is consistent with the whole genome SNP distribution density before screening, and randomly screening to obtain 150,000 SNP sites on the whole genome for predictive phenotypic analysis; Step S42, constructing a multidimensional space multilayer perceptron neural network MDMLP, inputting the 150,000 SNP sites screened in step S41 into the MDMLP, and realizing the fusion of local and global features between the extracted SNP sites; Step S43, using the neural network designed in step S42 to learn the one-to-one corresponding genotype and phenotype data of each period, to obtain a genotype-phenotype prediction model for each growth period.

[0025] Furthermore, the method for constructing MDMLP in step S42 of the present invention is as follows: Figure 2 As shown, including: Step S421, setting a patch embedding layer, using the patch embedding layer to convert the input 150,000 SNP site features into word embeddings of different dimensions including height, channel and width; Step S422, set the MDblock layer, respectively for the word embedding of different dimensions of height, channel and width, use the fully connected neural network to construct the height mixed MLP layer, channel mixed MLP layer and token width mixed MLP layer respectively, to further realize the local feature extraction of different dimensions; the specific implementation process is shown in formula (4), (4) In formula (4), y represents the feature vector extracted by the hybrid MLP layer, x represents the encoded word embedding features, W 1 and W2 represents the weights of the two layers of linear mapping, b 1 and b 2 is the bias of the two-layer linear mapping, σ Represents the activation function Gelu , the captured linear features can be nonlinearized through the activation function. The activation function calculation is shown in formula (5). (5) In the above formula, X Indicates compliance N (0,1) Normally distributed random numbers, P ( X ≤ x ) represents a random variable X Less than or equal to x The probability that X~N (0,1), so P ( X ≤ x ) refers to the cumulative distribution function of the standard normal distribution; Step S423, setting a global average pooling layer, and using global average pooling to achieve global feature fusion based on features extracted from different dimensions; Step S424, setting a fully connected neural network layer, and using a fully connected neural network based on the globally fused features to further improve the effect of feature extraction; Step S425, adding a dropout regularization method and nonlinear activation functions such as ReLU to the network to prevent overfitting, and finally constructing an MDMLP network.

[0026] Furthermore, the method in step S5 of the present invention includes: Step S51, removing data of plant lodging, withering and image blur from the rice image obtained in step S2 and the rice phenotype information obtained in step S3, sorting them into one-to-one corresponding phenotype-image pairs, and dividing them into two groups of data, before heading and after heading; Step S52, 533 rice varieties are randomly divided into a training set and a test set at a ratio of 9:1, 16321 images and corresponding phenotypic data before heading are used as the training set, 1830 images and corresponding phenotypic data are used as the test set, 12352 images and corresponding phenotypic data after heading are used as the training set, and 1790 images and corresponding phenotypic data are used as the test set; Step S53, based on the conditional control high-resolution generative adversarial network ConditionalStyleGAN of the StyleGAN series, improve its discriminator network structure and corresponding loss function, such as Figure 3 As shown; Step S54, using the two training sets obtained in step S52, training the improved high-resolution generative adversarial network in step S53, and obtaining two phenotype-image prediction models corresponding to the pre-heading period and the post-heading period, respectively.

[0027] Furthermore, the method for improving the high-resolution generative adversarial network ConditionalStyleGAN in step S53 of the present invention includes: Step S531, in the discriminator of Conditional StyleGAN, a multi-scale discriminator network is constructed, and a downsampling convolution layer is used to extract image features to obtain a feature vector; Step S532, the phenotypic data is input into the discriminator network as a conditional vector, the size of the conditional vector is 1×1×13, the first two dimensions of the vector are expanded by copying, and a 4×4×13 conditional vector is obtained. At this time, the expanded conditional vector and the image feature vector obtained after downsampling are consistent in the first two dimensions, and the image feature vector size is 4×4×3. The expanded conditional vector and the image feature vector are aligned in the first two dimensions, and the third dimension is superimposed to obtain a fused feature vector with a size of 4×4×(13+3), then the fused features are extracted by convolution, and finally the discriminator score is obtained using a fully connected layer; Step S533, calculate the cross entropy between the discriminator score and 0 or 1 as the loss function of the generative adversarial network, as shown in formula (6), (6) In the formula D is the discriminator, G For the generator, E is the mathematical expectation, I To input a real image, y S is a conditional vector of size 1×1×13, z is Gaussian distributed noise, V ( D , G ) represents the difference between the generated rice image and the real rice image, using the cross entropy loss function of the two categories of true and false, including and Two parts: Represents a fixed generator G Parameter training discriminator D parameters, by maximizing the cross entropy loss V ( D , G ) to update the discriminator D parameter; represents a fixed discriminator D Parameter training generator G Parameters, generatorG In the discriminator D Maximize the cross entropy loss V ( D , G ), minimize the cross entropy loss.

[0028] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the exemplary embodiments described in the present invention are merely illustrative of the spirit of the present invention. Those skilled in the art of the present invention may make various modifications, supplements or replace the described exemplary embodiments in a similar manner, but will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A method for predicting crop genotypes throughout their growth period based on artificial intelligence, the method comprising: Step S1, preparing a plurality of crop seeds of known varieties, planting them in different pots under the same growth environment, and using a camera to regularly capture crop images during the complete growth period of the crops to obtain original images of the crops during the entire growth period; Step S2, training a crop semantic segmentation model through manual annotation, and using the model to process the original crop image obtained in step S1, removing irrelevant content in the image, retaining only the area where the crop is located, and obtaining a crop image; Step S3, performing image processing on the crop image obtained in step S2 to extract phenotypic information of the crop at different growth stages; Step S4, obtaining the genotype sequence of the crop by sequencing technology, sorting out the one-to-one corresponding genotype sequence and crop phenotype information, constructing a multidimensional space multilayer perceptron neural network MDMLP, training to obtain a genotype-phenotype prediction model, and establishing a mapping relationship between the genotype and phenotype of the crop throughout its growth period; Step S5, sorting out the crop phenotypes and corresponding image data of each growth period, constructing a condition-controlled crop image generator and a discriminator integrating condition information based on the improved StyleGAN high-resolution generative adversarial network model, and designing corresponding loss functions, training to obtain the phenotype-image prediction model of each growth period, and establishing a mapping relationship between the crop phenotypes and crop images of each growth period; Step S6, inputting the genotype sequence to be predicted into the genotype-phenotype prediction model obtained in step S4 to obtain the predicted crop phenotype; Step S7, inputting the crop phenotype predicted in step S6 and the corresponding growth period information into the phenotype-image prediction model obtained in step S5 to obtain a predicted crop image.

2. The method for predicting crop genotypes throughout the growth period based on artificial intelligence according to claim 1, characterized in that: The method in step S2 comprises: Step S21, selecting different plants with large growth differences and selecting crop images at different time points as training sets; Step S22, performing pixel-level semantic annotation on the area where the crop is located in the training set image to obtain annotation information corresponding to the image; Step S23, inputting the training set images and corresponding annotation information into a semantic segmentation network for training to obtain a crop semantic segmentation model; Step S24, using the crop semantic segmentation model to segment the crop original image acquired in step S1 to obtain a crop binary image; Step S25, using the crop binary image obtained in step S24 to perform mask processing on the original crop image, to obtain a crop image that only retains the area where the crop is located.

3. The method for predicting crop genotypes throughout the growth period based on artificial intelligence according to claim 1, characterized in that: The method in step S4 includes: Step S41, sequencing crops to obtain single nucleotide variant site data SNP of all crop genomes, and randomly screening m SNP sites are selected to ensure that the whole genome SNP distribution density after screening is consistent with the whole genome SNP distribution density before screening, and the genotype sequence for predicting crop phenotypes is obtained; Step S42, constructing a multidimensional space multilayer perceptron neural network MDMLP to achieve the fusion of local and global features between the extracted SNP sites; Step S43, using the neural network designed in step S42 to learn the one-to-one corresponding genotype and phenotype data of each period, to obtain a genotype-phenotype prediction model for each growth period.

4. The method for predicting crop genotypes throughout the growth period based on artificial intelligence according to claim 3, characterized in that: The method for constructing the MDMLP in step S42 includes: Step S421, set the patch embedding layer, and use the patch embedding layer to embed the input m The SNP site features are converted into word embeddings of different dimensions including height, channel and width; Step S422, set the MDblock layer, respectively for the word embedding of different dimensions of height, channel and width, use the fully connected neural network to construct the height mixed MLP layer, channel mixed MLP layer and token width mixed MLP layer respectively, to further realize the local feature extraction of different dimensions; the specific implementation process is shown in formula (1), （1） In formula (1), y represents the feature vector extracted by the hybrid MLP layer, x represents the encoded word embedding features, W 1 and W 2 represents the weights of the two layers of linear mapping, b 1 and b 2 is the bias of the two-layer linear mapping, σ Represents the activation function Gelu , the captured linear features can be nonlinearized through the activation function. The activation function calculation is shown in formula (2). （2） In the above formula, X Indicates compliance N (0,1) Normally distributed random numbers, P ( X ≤ x ) represents a random variable X Less than or equal to x The probability of X~N (0,1), so P ( X ≤ x ) refers to the cumulative distribution function of the standard normal distribution; Step S423, setting a global average pooling layer, and using global average pooling to achieve global feature fusion based on features extracted from different dimensions; Step S424, setting a fully connected neural network layer, and using a fully connected neural network based on the globally fused features to further improve the effect of feature extraction; Step S425, adding a dropout regularization method and nonlinear activation functions such as ReLU to the network to prevent overfitting, and finally constructing an MDMLP network.

5. The method for predicting crop genotypes throughout the growth period based on artificial intelligence according to claim 1, characterized in that: The method in step S5 comprises: Step S51, obtaining a high-resolution generative adversarial network based on StyleGAN with the conditional vector introduced; Step S52, in the discriminator, construct a multi-scale discriminator network, use a downsampling convolution layer to extract image features, and obtain a feature vector; Step S53, the phenotype data is input into the discriminator network as a conditional vector, the size of the conditional vector is 1×1×S, S is the number of phenotypes, the first two dimensions of the vector are expanded by replication, and a 4×4×S conditional vector is obtained. At this time, the expanded conditional vector is consistent with the image feature vector obtained after downsampling in the first two dimensions, and the image feature vector size is 4×4×C, C is the number of channels. The expanded conditional vector and the image feature vector are aligned in the first two dimensions, and the third dimension is superimposed to obtain a fused feature vector with a size of 4×4×(S+C), then the fused features are extracted by convolution, and finally the discriminator score is obtained using a fully connected layer; Step S54, calculate the cross entropy between the discriminator score and 0 or 1 as the loss function of the generative adversarial network, as shown in formula (3), m i n G m a x D V D G = m i n G m a x D E l o g D I y S + E l o g 1 - D G z y S y S （3） In the formula D is the discriminator, G For the generator, E is the mathematical expectation, I To input a real image, y S is a conditional vector of size 1×1×S, z is Gaussian distributed noise, V ( D , G ) represents the difference between the generated image and the real image, and adopts the cross entropy loss function of two categories, namely true and false, including and Two parts; Represents a fixed generator G Training the Discriminator D , by maximizing the cross entropy loss V ( D , G ) to update the discriminator D Parameters; represents a fixed discriminator D Training the Generator G , the generator needs to maximize the cross entropy loss between real and fake images in the discriminator V ( D , G ), minimize the cross entropy loss.

Images