Soybean multi-mode phenotype prediction method and device and electronic equipment

Abstract

The invention provides a soybean multi-modal phenotype prediction method and device and electronic equipment, and the method comprises the steps: obtaining the multi-modal information of the soybean phenotype and the environment information of a soybean growth region, inputting the multi-modal information and the environment information into a soybean phenotype prediction model, and obtaining a soybean phenotype prediction result. The soybean phenotype prediction model is obtained by training a pre-constructed phenotype prediction model through sample data, and the pre-constructed phenotype prediction model comprises a soybean phenotype feature extraction module and a phenotype environment mechanism module; the soybean phenotype is predicted by considering the multi-modal information of the soybean phenotype and the environment information of the soybean growth area and combining the soybean phenotype feature extraction module and the phenotype environment mechanism module, the accuracy of the soybean phenotype prediction result is improved, the soybean phenotype prediction method can be suitable for different growth environments, and the soybean phenotype prediction efficiency is improved. And the applicability of the soybean phenotype prediction model is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent agriculture, and in particular to a soybean multi-modal phenotype prediction method and device and electronic equipment. BACKGROUND

[0002] Crop phenotype refers to the sum of observable morphological, structural, physiological, biochemical and yield characteristics and traits exhibited by crops under the joint action of genotype and environmental factors. At present, crop phenotype information is mainly collected through unmanned aerial vehicle platforms. However, in the study of the phenotypes of important crops such as soybeans, due to the long growth cycle and complex and variable phenotypic characteristics, how to achieve multi-modal and high-precision phenotype prediction has become an important problem to be solved in the field of intelligent agriculture.

[0003] In related technologies, data of soybean phenotypes collected by unmanned aerial vehicles are used as sample data for model prediction to obtain prediction results of soybean phenotypes. However, since the soybean phenotype data collected by unmanned aerial vehicles is single-modal data, or the multi-modal data is simply spliced, the dynamic changes of different modal phenotypes in different growth periods are not fully considered, resulting in inaccurate prediction data of soybean phenotypes. Moreover, only relying on sample data to drive the model (for example, a deep learning model) lacks the physiological and ecological mechanisms of crops, the prediction results may violate biological laws, and the generalization ability sharply decreases when encountering unseen environmental conditions, and the interpretability is poor, making it difficult to provide reliable decision-making basis for breeders and agronomists. In addition, the model trained based on single-region or annual data is difficult to adapt to the needs of phenotype prediction under different soil types, climate zones and management measures, limiting its large-scale industrial application.

[0004] Therefore, it is necessary to improve the soybean multi-modal phenotype prediction method in the related art. SUMMARY

[0005] In view of the above-mentioned shortcomings of the prior art, the present application provides a soybean multi-modal phenotype prediction method, device and electronic equipment to solve the above technical problems.

[0006] According to an aspect of an embodiment of the present application, a soybean multi-modal phenotype prediction method is provided, the method comprising: acquiring multi-modal information of soybean phenotypes and environmental information of a soybean growth area; inputting the multi-modal information and the environmental information into a soybean phenotype prediction model to obtain a soybean phenotype prediction result; the soybean phenotype prediction model is obtained by training a pre-constructed phenotype prediction model using sample data; and the pre-constructed phenotype prediction model comprises a soybean phenotype feature extraction module and a phenotype environment mechanism module.

[0007] In an embodiment of the present application, if the sample data comprises historical multi-modal information of soybeans in multiple growth regions, historical environmental information, and historical planting management information, the process of training the pre-constructed phenotype prediction model to obtain the soybean phenotype prediction model through the sample data comprises: training the pre-constructed phenotype prediction model through the historical multi-modal information to obtain a trained phenotype prediction model; and training the trained phenotype prediction model through the historical multi-modal information, the historical environmental information, and the historical planting management information to obtain the soybean phenotype prediction model.

[0008] In an embodiment of the present application, the process of training the pre-constructed phenotype prediction model through the historical multi-modal information to obtain a trained phenotype prediction model comprises: pre-processing and spatio-temporal alignment of the historical multi-modal information to obtain multi-modal fusion information; the pre-processing comprises a denoising operation; inputting the multi-modal fusion information into the soybean phenotype feature extraction module to obtain soybean phenotype feature data; inputting the soybean phenotype feature data into the phenotype environmental mechanism module to obtain first soybean physiological parameter data; the phenotype environmental mechanism module comprises a biomass accumulation mechanism model, a yield formation mechanism model, and an anti-lodging mechanism model; the first soybean physiological parameter data comprises a biomass theoretical value, a yield theoretical value, and an anti-lodging index theoretical value; inputting the first soybean physiological parameter data into the soybean phenotype feature extraction module to obtain first soybean phenotype feature data and second soybean physiological parameter data; adjusting parameters in the pre-constructed phenotype prediction model to minimize the sum of differences between the first soybean physiological parameter data and the second soybean physiological parameter data, and between the first soybean phenotype feature data and feature data corresponding to the multi-modal fusion information, to obtain the trained phenotype prediction model.

[0009] In an embodiment of the present application, if the historical multi-modal information comprises soybean leaf and canopy spectral data, soybean plant image data, and soybean plant point cloud data, the process of pre-processing and spatio-temporal alignment of the historical multi-modal information to obtain multi-modal fusion information comprises: performing radiation calibration, atmospheric correction, reflectance inversion, and spectral resampling on the soybean leaf and canopy spectral data to obtain denoised spectral data; performing photometric correction and color equalization on the soybean plant image data to obtain orthographic image data; performing point cloud denoising and point cloud classification on the soybean plant point cloud data to obtain classified point cloud data, and generating a digital surface model and a canopy height model based on the classified point cloud data; performing data point matching on the denoised spectral data and the orthographic image data to obtain image matching data; and matching the image matching data with the digital surface model and the canopy height model to obtain the multi-modal fusion information.

[0010] In an embodiment of the present application, if the soybean phenotype feature extraction module includes a spectral feature extraction module, an image feature extraction module, and a point cloud feature extraction module, a dynamic weight generation module, and a weighted cross-attention module, the process of inputting the multi-modal fusion information into the soybean phenotype feature extraction module to obtain soybean phenotype feature data includes: inputting the multi-modal fusion information into the spectral feature extraction module to obtain spectral feature data; inputting the multi-modal fusion information into the image feature extraction module to obtain image feature data; inputting the multi-modal fusion information into the point cloud feature extraction module to obtain point cloud feature data; inputting the spectral feature data, the image feature data, and the point cloud feature data into the dynamic weight generation module to output the weight of the spectral feature data, the weight of the image feature data, and the weight of the point cloud feature data; the dynamic weight generation module is used to determine the weight of the spectral feature data, the weight of the image feature data, and the weight of the point cloud feature data according to the growth period indication features extracted from the spectral feature data, the growth period indication features extracted from the image feature data, and the growth period indication features extracted from the point cloud feature data; inputting the spectral feature data, the image feature data, the point cloud feature data, the weight of the spectral feature data, the weight of the image feature data, and the weight of the point cloud feature data into the weighted cross-attention module to obtain the soybean phenotype feature data.

[0011] In an embodiment of the present application, the calculation formula of the weighted cross-attention module is as follows: wherein, represents soybean phenotype feature data, represents a query matrix, which is a matrix of current modal data, represents a key matrix, represents a value matrix, represents an activation function, represents the weight of the key matrix, represents taking the matrix of the first modal data as the key matrix, represents a scaling factor, represents the weight of the value matrix, represents taking the matrix of the first modal data as the value matrix, and the current modal data is different from the first modal data and the second modal data.

[0012] In one embodiment of this application, the difference between the first soybean physiological parameter data and the second soybean physiological parameter data, and the sum of the differences between the first soybean phenotypic feature data and the feature data corresponding to the multimodal fusion information, are characterized by a loss function, the expression of which is as follows: ,in, Represents the loss function. This represents the mean squared error loss function. The mechanism consistency loss function is represented. Indicates the first Feature data corresponding to modality fusion information obtained by preprocessing and spatiotemporal alignment of sample data. This indicates that the soybean phenotypic feature extraction module extracts the first... The first soybean phenotypic characteristic data was obtained by predicting from a sample of data. Represents the regularization coefficient. This indicates that the soybean phenotypic feature extraction module extracts the first... The second soybean physiological parameter data was obtained by predicting from a sample of data. This indicates that the phenotypic environment mechanism module is responsible for the first... The first soybean physiological parameter data was obtained by predicting from a sample of data.

[0013] In one embodiment of this application, the process of training the trained phenotypic prediction model using the historical multimodal information, the historical environmental information, and the historical planting management information to obtain the soybean phenotypic prediction model includes: performing feature annotation on the historical multimodal information to obtain historical multimodal information with feature annotation information; establishing a correspondence between the historical multimodal information with feature annotation information and the historical environmental information and the historical planting management information, denoted as correspondence data; and training the trained phenotypic prediction model using the correspondence data to obtain the soybean phenotypic prediction model.

[0014] According to one aspect of the embodiments of this application, a soybean multimodal phenotypic prediction device is provided, comprising: an information acquisition module for acquiring multimodal information of soybean phenotypes and environmental information of soybean growing areas; a phenotypic prediction module for inputting the multimodal information and the environmental information into a soybean phenotypic prediction model to obtain soybean phenotypic prediction results; the soybean phenotypic prediction model is obtained by training a pre-constructed phenotypic prediction model with sample data; the pre-constructed phenotypic prediction model includes: a soybean phenotypic feature extraction module and a phenotypic environmental mechanism module.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the soybean multimodal phenotypic prediction method as described above.

[0016] The beneficial effects of this application are as follows: This application obtains multimodal information of soybean phenotype and environmental information of soybean growing area, inputs the multimodal information and environmental information into soybean phenotype prediction model, and obtains soybean phenotype prediction results. The soybean phenotype prediction model is obtained by training a pre-constructed phenotype prediction model with sample data. The pre-constructed phenotype prediction model includes a soybean phenotype feature extraction module and a phenotype environment mechanism module. The above process, by considering the multimodal information of soybean phenotype and the environmental information of soybean growing area, and by combining the soybean phenotype feature extraction module and the phenotype environment mechanism module, improves the accuracy of soybean phenotype prediction results and makes it applicable to different growing environments, thus improving the applicability of the soybean phenotype prediction model.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application; Figure 2 This is a flowchart illustrating a soybean multimodal phenotypic prediction method in an exemplary embodiment of this application; Figure 3 This is a block diagram illustrating a soybean multimodal phenotypic prediction system as an exemplary embodiment of this application; Figure 4 This is a block diagram illustrating a soybean multimodal phenotypic prediction device according to an exemplary embodiment of this application; Figure 5 This is a schematic diagram of the structure of a computer system for an electronic device, as illustrated in an exemplary embodiment of this application. Detailed Implementation

[0019] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0021] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0022] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application.

[0023] Reference Figure 1 As shown, the system architecture may include a data acquisition device 101 and an electronic device 102. The electronic device 102 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, or a neural network computer. Those skilled in the art can use the electronic device 102 to acquire multimodal information about soybean phenotypes and environmental information about the soybean growing area. This multimodal information and environmental information are then input into a soybean phenotype prediction model to obtain soybean phenotype prediction results. The soybean phenotype prediction model is trained using sample data on a pre-built phenotype prediction model, which includes a soybean phenotype feature extraction module and a phenotype environmental mechanism module. The data acquisition device 101 is used to collect multimodal information about soybean phenotypes and environmental information about the soybean growing area, and provides this information to the electronic device 102 for processing.

[0024] Indicatively, after acquiring multimodal information on soybean phenotypes and environmental information of the soybean growing area from the acquisition device 101, the electronic device 102 inputs the multimodal information and environmental information into the soybean phenotype prediction model to obtain soybean phenotype prediction results. The soybean phenotype prediction model is obtained by training a pre-constructed phenotype prediction model with sample data. The pre-constructed phenotype prediction model includes a soybean phenotype feature extraction module and a phenotype environment mechanism module. The above process, by considering the multimodal information on soybean phenotypes and the environmental information of the soybean growing area, and by combining the soybean phenotype feature extraction module and the phenotype environment mechanism module, predicts soybean phenotypes, improves the accuracy of soybean phenotype prediction results, and makes the model applicable to different growing environments, thus improving the applicability of the soybean phenotype prediction model.

[0025] The implementation details of the technical solutions in the embodiments of this application are described in detail below: Figure 2 This is a flowchart illustrating a soybean multimodal phenotypic prediction method according to an exemplary embodiment of this application. (Refer to...) Figure 2 As shown, the soybean multimodal phenotypic prediction method includes at least steps S210 to S220, which are described in detail below: In step S210, multimodal information of soybean phenotype and environmental information of the soybean growing area are acquired. In one embodiment of this application, the multimodal information of soybean phenotype includes: soybean leaf information, soybean canopy information, soybean plant information, biomass information, yield information, etc., and the environmental information includes meteorological environmental information and soil environmental information, etc. Meteorological environmental information includes: air temperature, air humidity, precipitation, wind force, air pressure, light intensity, carbon dioxide concentration, etc., and soil environmental information includes: soil temperature, soil moisture, soil pH, soil texture, soil density and porosity, soil hydraulic conductivity, organic matter content, nutrient content, microbial activity, microbial diversity, etc.

[0026] In step S220, multimodal information and environmental information are input into the soybean phenotypic prediction model to obtain the soybean phenotypic prediction result. In one embodiment of this application, the soybean phenotypic prediction model is obtained by training a pre-constructed phenotypic prediction model with sample data. The pre-constructed phenotypic prediction model includes a soybean phenotypic feature extraction module and a phenotypic environment mechanism module. The above process, by considering the multimodal information of soybean phenotype and the environmental information of soybean growth area, and by combining the soybean phenotypic feature extraction module and the phenotypic environment mechanism module, predicts the soybean phenotype, improves the accuracy of the soybean phenotypic prediction result, and is applicable to different growth environments, thus improving the applicability of the soybean phenotypic prediction model.

[0027] In one embodiment of this application, if the sample data includes: historical multimodal information of soybeans in multiple growing regions, historical environmental information, and historical planting management information, then the process of training a pre-constructed phenotypic prediction model using the sample data to obtain a soybean phenotypic prediction model includes: A pre-built phenotypic prediction model is trained using historical multimodal information to obtain a trained phenotypic prediction model. In one embodiment of this application, the pre-built phenotypic prediction model includes a soybean phenotypic feature extraction module and a phenotypic environment mechanism module. Training the pre-built phenotypic prediction model using historical multimodal information improves its ability to recognize soybean multimodal features.

[0028] A soybean phenotypic prediction model is obtained by training a trained phenotypic prediction model using historical multimodal information, historical environmental information, and historical planting management information. In one embodiment of this application, the historical multimodal information includes: soybean leaf and canopy spectral data, soybean plant image data, soybean plant point cloud data, historical biomass information, and historical yield information, etc. The historical environmental information includes: historical meteorological environmental information and historical soil environmental information, etc. The historical planting management information includes: planting time arrangement, spatial layout design, irrigation and drainage plan, etc. By training the trained phenotypic prediction model using historical multimodal information, historical environmental information, and historical planting management information, the ability of the soybean phenotypic prediction model to identify soybean multimodal characteristics in different growth environments and geographical regions is improved, exhibiting strong universality. Moreover, the combination of the soybean phenotypic feature extraction module and the phenotypic environment mechanism module makes the prediction of soybean multimodal characteristics by the soybean phenotypic prediction model more consistent with the physiological and ecological mechanisms of soybean, avoiding prediction results of soybean multimodal characteristics that violate biological laws.

[0029] In one embodiment of this application, the process of training a pre-built phenotypic prediction model using historical multimodal information to obtain a trained phenotypic prediction model includes: Historical multimodal information is preprocessed and spatiotemporally aligned to obtain multimodal fusion information. In one embodiment of this application, the preprocessing includes a denoising operation. By preprocessing and spatiotemporally aligning the historical multimodal information, pixel-level or point-level alignment of the multimodal fusion information under the same geographic coordinate system is achieved, providing a high-quality data foundation for subsequent model training.

[0030] Multimodal fusion information is input into a soybean phenotypic feature extraction module to obtain soybean phenotypic feature data. In one embodiment of this application, if the soybean phenotypic feature extraction module includes: a spectral feature extraction module, an image feature extraction module, a point cloud feature extraction module, a dynamic weight generation module, and a weighted cross-attention module, then the process of inputting multimodal fusion information into the soybean phenotypic feature extraction module to obtain soybean phenotypic feature data includes: inputting multimodal fusion information into a spectral feature extraction module to obtain spectral feature data; inputting multimodal fusion information into an image feature extraction module to obtain image feature data; inputting multimodal fusion information into a point cloud feature extraction module to obtain point cloud feature data; inputting spectral feature data, image feature data, and point cloud feature data into a dynamic weight generation module, and outputting the weights of the spectral feature data, the image feature data, and the point cloud feature data; the dynamic weight generation module is used to calculate the weights based on the fertility period indicator features extracted from the spectral feature data, the fertility period indicator features extracted from the image feature data, and the point cloud feature data. The extracted growth stage indicator features are used to determine the weights of spectral feature data, image feature data, and point cloud feature data. These weights are then input into a weighted cross-attention module to obtain soybean phenotypic feature data. This process not only preserves the original information of each modality but also highlights key features at different growth stages through a dynamic weighting mechanism, making the extracted soybean phenotypic feature data more biologically plausible. The weighted cross-attention module calculates the correlation between different modal feature data based on their weights, automatically learning complementary relationships and further enhancing the robustness of the soybean phenotypic feature data representation, providing a more reliable feature foundation for subsequent model training.

[0031] Soybean phenotypic data are input into the phenotypic environment mechanism module to obtain the first soybean physiological parameter data. In one embodiment of this application, the phenotypic environment mechanism module includes: a biomass accumulation mechanism model, a yield formation mechanism model, and a lodging resistance mechanism model; the first soybean physiological parameter data includes: theoretical biomass value, theoretical yield value, and theoretical lodging resistance index value. The biomass accumulation mechanism model is the Farquhar (Farquhar-von Caemmerer-Berry, FvCB) photosynthetic model. The Farquhar photosynthetic model is used to calculate the instantaneous photosynthetic rate of soybean based on the PAR (Photosynthetically Active Radiation) and CO2 concentration data of ground nodes. Based on the instantaneous photosynthetic rate of soybean, the number of days in the growth period, and the cumulative amount of photosynthetically active radiation, the theoretical biomass value is calculated. The formula for calculating the instantaneous photosynthetic rate of soybean is as follows: Equation (1) in, This indicates the instantaneous photosynthetic rate of soybeans. This represents the electron transport efficiency driven by photosynthetically active radiation (μmol·m⁻²·s⁻¹). The intercellular CO2 concentration (μmol·mol⁻¹) is measured using a portable photosynthesis meter. This represents the CO2 compensation point (μmol·mol⁻¹). The dark respiration rate (μmol·m⁻²·s⁻¹) is measured using a closed system method, an open system method, or an isotope tracing method.

[0032] The formula for calculating electron transfer efficiency is shown below: Equation (2) in, This represents the electron transport efficiency driven by photosynthetically active radiation (μmol·m⁻²·s⁻¹). Apparent quantum efficiency, This represents photosynthetically active radiation, measured using a quantum sensor. Indicates the curvature parameter, This represents the maximum electron transport rate (μmol·m⁻²·s⁻¹).

[0033] The formula for calculating the theoretical value of biomass is as follows: Equation (3) in, This represents the theoretical value of biomass. This indicates the instantaneous photosynthetic rate of soybeans. This represents the cumulative amount of photosynthetically active radiation. Indicates the number of days in the reproductive period. This indicates biomass conversion efficiency (e.g., 0.35 for seedling stage, 0.55 for flowering stage, 0.45 for pod-setting stage, and 0.30 for grain-filling stage).

[0034] In one embodiment of this application, the yield formation mechanism model is the Michaelis-Menten nutrient uptake model. The Michaelis-Menten nutrient uptake model calculates the soybean nitrogen uptake rate based on soil available nitrogen content data, and calculates the theoretical yield value based on soybean phenotypic characteristics such as plant height, stem diameter, and pod number. The formula for calculating the soybean nitrogen uptake rate is as follows: Equation (4) in, Indicates the rate of nitrogen absorption in soybeans. The maximum nitrogen uptake rate is measured using a hydroponic experiment and is expressed in kg / hm². 2 / d, or, determined by the field root bag method, in μmol / g root / h. The available nitrogen content in the soil is indicated by the Kjeldahl method, measured in μg / g, or by the ion-selective electrode method, measured in mg / kg. denoted as Michaelis constant, which reflects the root's affinity for nitrogen. The smaller the value, the higher the affinity. It is obtained through a concentration gradient experiment.

[0035] The formula for calculating the theoretical output value is as follows: Equation (5) in, This represents the theoretical output value. Indicates the rate of nitrogen absorption in soybeans. Indicates plant height, Indicates stem thickness. Indicates the number of pods. This represents the production coefficient (with a value range of 0.6-0.8).

[0036] In one embodiment of this application, the lodging resistance mechanism model is a stem mechanics model, which is used to determine the plant height extracted from soybean plant point cloud data. Stem thick Based on the stem toughness T measured at ground nodes, the theoretical value of the lodging resistance index is calculated. The formula for calculating the theoretical value of the lodging resistance index is as follows: Equation (6) in, This represents the theoretical value of the lodging resistance index. This represents the stem thickness extracted from soybean plant point cloud data. This represents the plant height extracted from soybean plant point cloud data. It indicates the toughness of the stem.

[0037] The first soybean physiological parameter data is input into the soybean phenotypic feature extraction module to obtain first soybean phenotypic feature data and second soybean physiological parameter data. In one embodiment of this application, the soybean phenotypic feature extraction module has the function of extracting the first soybean physiological parameter data when the first soybean physiological parameter data is input, thereby obtaining the second soybean physiological parameter data, and outputting the first soybean phenotypic feature data based on the second soybean physiological parameter data.

[0038] The goal is to minimize the difference between the first soybean physiological parameter data and the second soybean physiological parameter data, as well as the sum of the differences between the first soybean phenotypic feature data and the feature data corresponding to the multimodal fusion information. The parameters in a pre-constructed phenotypic prediction model are adjusted to obtain a trained phenotypic prediction model. In one embodiment of this application, the parameters in the pre-constructed phenotypic prediction model are adjusted to obtain a parameter-adjusted phenotypic prediction model. If the difference between the physiological parameter data output by the parameter-adjusted phenotypic prediction model and the physiological parameter data output by the parameter-adjusted soybean phenotypic feature extraction module, and the sum of the differences between the soybean phenotypic feature data output by the parameter-adjusted phenotypic prediction model and the feature data corresponding to the multimodal fusion information, is less than a preset difference threshold, or if the number of parameter adjustments in the pre-constructed phenotypic prediction model is greater than or equal to a preset number of adjustments threshold, then the adjustment of the parameters in the pre-constructed phenotypic prediction model is stopped, and a trained phenotypic prediction model is obtained.

[0039] In one embodiment of this application, the difference between the first soybean physiological parameter data and the second soybean physiological parameter data, and the sum of the differences between the first soybean phenotypic feature data and the feature data corresponding to the multimodal fusion information, are characterized by a loss function, the expression of which is as follows: Equation (7) in, Represents the loss function. This represents the mean squared error loss function. The mechanism consistency loss function is represented. Indicates the first Feature data (e.g., real feature data) corresponding to modality fusion information obtained by preprocessing and spatiotemporal alignment of sample data. This indicates that the soybean phenotypic feature extraction module extracts the first... The first soybean phenotypic characteristic data was obtained by predicting from a sample of data. Represents the regularization coefficient. This indicates that the soybean phenotypic feature extraction module extracts the first... The second soybean physiological parameter data was obtained by predicting from a sample of data. This indicates that the phenotypic environment mechanism module is responsible for the first... The first soybean physiological parameter data was obtained by predicting from a sample of data.

[0040] In one embodiment of this application, by dynamically adjusting the parameters in a pre-built phenotypic prediction model through a bidirectional feedback mechanism, the prediction error of the pre-built phenotypic prediction model can be gradually reduced, thereby improving the prediction accuracy and robustness of the trained phenotypic prediction model. Simultaneously, by combining quantitative evaluation of the loss function, the training process of the pre-built phenotypic prediction model becomes more transparent and controllable, providing reliable technical support for soybean multimodal phenotypic prediction.

[0041] In one embodiment of this application, if the historical multimodal information includes: soybean leaf and canopy spectral data, soybean plant image data, and soybean plant point cloud data, then the process of preprocessing and spatiotemporally aligning the historical multimodal information to obtain multimodal fusion information includes: Radiometric calibration, atmospheric correction, reflectance inversion, and spectral resampling are performed on soybean leaf and canopy spectral data to obtain denoised spectral data. In one embodiment of this application, soybean leaf and canopy spectral data are obtained by scanning soybeans using a hyperspectral camera mounted on a UAV. The hyperspectral camera has a wavelength range of 400-1700 nm and a spectral resolution ≤3 nm. By performing radiometric calibration, atmospheric correction, reflectance inversion, and spectral resampling on the soybean leaf and canopy spectral data, noise and interference factors in the soybean leaf and canopy spectral data are effectively removed, improving the quality and accuracy of the spectral data.

[0042] Photometric correction and color equalization are performed on soybean plant image data to obtain orthophoto data. In one embodiment of this application, soybean plant image data is acquired by a drone equipped with an RGB sensor. By performing photometric correction and color equalization on the soybean plant image data, the problems of uneven lighting and color deviation in the soybean plant image data are effectively eliminated, so that the orthophoto data can more realistically reflect the actual shape and color characteristics of the soybean plant.

[0043] Point cloud data of soybean plants is denoised and classified to obtain classified point cloud data. Based on the classified point cloud data, a digital surface model and a canopy height model are generated. In one embodiment of this application, soybean plant point cloud data is obtained by scanning soybean plants using a LiDAR (Light Detection and Ranging) sensor mounted on a UAV. By denoising and classifying the soybean plant point cloud data, noise and outliers in the soybean plant point cloud data are effectively removed, and the point cloud data is divided into different categories, such as ground points and vegetation points. Based on the classified point cloud data, a digital surface model and a canopy height model can be further generated. The digital surface model can accurately depict the surface morphology of the soybean plant and its surrounding terrain; the generated canopy height model can accurately reflect the height distribution of the soybean canopy. The process of generating the digital surface model and canopy height model based on the classified point cloud data can be implemented with reference to the process of generating digital surface models and canopy height models based on point cloud data in related technologies, and is not specifically limited here.

[0044] The denoised spectral data is matched with orthophoto data to obtain image matching data; then, the image matching data is matched with a digital surface model and a canopy height model to obtain multimodal fusion information. In one embodiment of this application, the process of matching denoised spectral data with orthophoto data to obtain image matching data, and then matching the image matching data with a digital surface model and a canopy height model to obtain multimodal fusion information, is implemented based on a concatenated alignment algorithm for SIFT (Scale-Invariant Feature Transform) features and point cloud registration.

[0045] In one embodiment of this application, if the soybean phenotypic feature extraction module includes: a spectral feature extraction module, an image feature extraction module, a point cloud feature extraction module, a dynamic weight generation module, and a weighted cross-attention module, then the process of inputting multimodal fusion information into the soybean phenotypic feature extraction module to obtain soybean phenotypic feature data includes: The multimodal fusion information is input into the spectral feature extraction module to obtain spectral feature data. In one embodiment of this application, the spectral feature extraction module includes a one-dimensional convolutional neural network (1D Convolutional Neural Network) module and a shifted window transformer (Shifted Window Transformer) module. The 1D Convolutional Neural Network is used to extract local spectral features from the multimodal fusion information, and the Shifted Window Transformer module is used to extract long-range inter-band dependencies in the local spectral features. Through the synergistic effect of the 1D Convolutional Neural Network module and the Shifted Window Transformer module, the spectral feature extraction module can comprehensively and accurately capture key feature information in the spectral data, providing rich spectral dimension data support for subsequent soybean phenotypic feature analysis.

[0046] The multimodal fusion information is input into the image feature extraction module to obtain image feature data. In one embodiment of this application, the image feature extraction module is a pre-trained Vision Transformer module. The pre-trained Vision Transformer module extracts spatial texture features and color features from the multimodal fusion information and uses the spatial texture features and color features as image feature data.

[0047] The multimodal fusion information is input into the point cloud feature extraction module to obtain point cloud feature data. In one embodiment of this application, the point cloud feature extraction module is a PointNet++ network model, which extracts the three-dimensional structural features from the multimodal fusion information.

[0048] Spectral feature data, image feature data, and point cloud feature data are input into a dynamic weight generation module, which outputs the weights of the spectral feature data, image feature data, and point cloud feature data. In one embodiment of this application, the dynamic weight generation module is used to determine the weights of the spectral feature data, image feature data, and point cloud feature data based on the fertility period indicator features extracted from the spectral feature data, the image feature data, and the point cloud feature data. For example, if the fertility period indicator is in the vegetative growth stage, then the spectral feature data (e.g., chlorophyll content) is assigned a higher weight; if the fertility period indicator is in the reproductive growth stage, then the point cloud feature data (e.g., ear structure) is assigned a higher weight.

[0049] The spectral feature data, image feature data, point cloud feature data, and the weights of the spectral feature data, image feature data, and point cloud feature data are input into a weighted cross-attention module to obtain soybean phenotypic feature data. In one embodiment of this application, the calculation formula of the weighted cross-attention module is as follows: Equation (8) in, This represents soybean phenotypic characteristic data. This represents the query matrix, which is a matrix of data for the current modality. Represents the key matrix. Represents a value matrix, This represents the activation function. The weights of the key matrix are represented. Indicates the first The matrix of modal data is used as the key matrix. Indicates the scaling factor. The weights represent the values ​​in the matrix. Indicates the first The matrix of modal data is used as the value matrix.

[0050] In one embodiment of this application, the current modal data is any one of spectral feature data, image feature data, and point cloud feature data. Modal data and the first The modal data consists of two other types of feature data besides the current modal data. For example, when the current modal data is spectral feature data, the second modal data... If the modal data is image feature data, then the weights of the key matrix are the weights of the image feature data. If the modal data is point cloud feature data, then the weights of the value matrix are the weights of the point cloud feature data.

[0051] In one embodiment of this application, the process of training a soybean phenotypic prediction model using historical multimodal information, historical environmental information, and historical planting management information to obtain a soybean phenotypic prediction model includes: Historical multimodal information is annotated with features to obtain historical multimodal information with feature annotation information. In one embodiment of this application, the historical multimodal information is annotated with features using ENVI (The Environment for Visualizing Images) or QGIS (Quantum Geographic Information System).

[0052] A correspondence is established between historical multimodal information with feature annotations and historical environmental and planting management information, denoted as correspondence data. In one embodiment of this application, the correspondence data is used to characterize the intrinsic relationship between historical multimodal information with feature annotations and historical environmental and planting management information. This correspondence data can clearly demonstrate the correlation between different information, providing richer and more accurate input for subsequent model training.

[0053] A soybean phenotypic prediction model is obtained by training the trained phenotypic prediction model using correspondence data. In one embodiment of this application, the process of training the trained phenotypic prediction model using correspondence data includes: inputting the correspondence data into the trained phenotypic prediction model to obtain second soybean phenotypic feature data; adjusting the parameters in the trained phenotypic prediction model with the goal of minimizing the difference between the second soybean phenotypic feature data and the feature data corresponding to historical multimodal information in the correspondence data, until the difference between the phenotypic feature data output by the re-adjusted phenotypic prediction model and the feature data corresponding to historical multimodal information in the correspondence data is less than a preset difference threshold, or the number of times the parameters in the trained phenotypic prediction model are adjusted is greater than or equal to a preset number of times threshold, then the adjustment of the parameters in the trained phenotypic prediction model is stopped, and the soybean phenotypic prediction model is obtained.

[0054] Figure 3 This is a block diagram illustrating a soybean multimodal phenotypic prediction system as shown in an exemplary embodiment of this application, such as... Figure 3 As shown, the soybean multimodal phenotypic prediction system includes: a multimodal data synchronous acquisition subsystem → a dynamic preprocessing and spatiotemporal alignment subsystem → a feature extraction and dynamic attention fusion subsystem → a phenotypic environment mechanism feedback subsystem → a cross-scene adaptive learning and prediction output subsystem. Among them, the multimodal data synchronous acquisition subsystem includes: a multi-rotor drone equipped with a hyperspectral camera with a wavelength of (400-1700) nm and a spectral resolution of ≤3 nm, an RGB industrial camera with a resolution of ≥20 million pixels, and a miniature LiDAR sensor with a point density of ≥300 points / m², as well as a ground IoT node integrating soil temperature and humidity sensors and photosynthetically active radiation sensors. The drone and the ground IoT node achieve data synchronization through a 5G edge computing module (time error ≤0.5s). The drone and the ground IoT node achieve millisecond-level synchronous acquisition of all sensors through the second pulse signal in GPS (Global Positioning System), ensuring strict spatiotemporal consistency of different modal data. Moreover, the latitude and longitude, time, solar altitude angle and other metadata of each drone flight are automatically recorded, providing a foundation for subsequent spatiotemporal alignment and cross-scene learning.

[0055] In one embodiment of this application, soybean leaf and canopy spectral data are acquired by a hyperspectral camera, soybean plant image data are acquired by an RGB industrial camera, soybean plant point cloud data are acquired by a miniature LiDAR sensor, soil temperature and humidity are acquired by a soil temperature and humidity sensor, and photosynthetically active radiation is acquired by a photosynthetically active radiation sensor.

[0056] The dynamic preprocessing and spatiotemporal alignment subsystem includes a preprocessing module and a spatiotemporal alignment module. The preprocessing module performs radiometric calibration, atmospheric correction, reflectance inversion, and spectral resampling on soybean leaf and canopy spectral data to obtain denoised spectral data. This data is then used to perform photometric correction and color equalization on soybean plant image data to obtain orthophoto data. Additionally, it performs point cloud denoising and point cloud classification on soybean plant point cloud data to obtain classified point cloud data. Based on the classified point cloud data, a digital surface model and a canopy height model are generated. The spatiotemporal alignment module uses a cascaded alignment algorithm that matches SIFT features with point cloud registration to match data points between the denoised spectral data and the orthophoto data, obtaining image matching data. Finally, it matches the image matching data with the digital surface model and the canopy height model to obtain multimodal fusion information.

[0057] The feature extraction and dynamic attention fusion subsystem includes: a spectral feature extraction module, an image feature extraction module, a point cloud feature extraction module, a dynamic weight generation module, and a weighted cross-attention module. The spectral feature extraction module is used to extract spectral feature data from the multimodal fusion information; the image feature extraction module is used to extract image feature data from the multimodal fusion information; the point cloud feature extraction module is used to extract point cloud feature data from the multimodal fusion information; the dynamic weight generation module is used to determine the weights of the spectral feature data, image feature data, and point cloud feature data based on the fertility period indicator features extracted from the spectral feature data, image feature data, and point cloud feature data; the weighted cross-attention module is used to perform feature fusion on the spectral feature data, image feature data, and point cloud feature data based on the weights of the spectral feature data, image feature data, and point cloud feature data to obtain soybean phenotypic feature data.

[0058] The phenotypic environmental mechanism feedback subsystem includes a phenotypic mechanism model, which incorporates a parameterized soybean growth process model (including a biomass accumulation mechanism model, a yield formation mechanism model, and a lodging resistance mechanism model). This model takes daily meteorological data (temperature, precipitation, radiation) and initial soil conditions as input and can simulate processes such as soybean biomass accumulation and phenological development. By inputting multimodal fusion information into the soybean phenotypic feature extraction module, soybean phenotypic feature data is obtained. This data is then input into the phenotypic environment mechanism module to obtain first soybean physiological parameter data. Finally, this first soybean physiological parameter data is input into the soybean phenotypic feature extraction module to obtain first soybean phenotypic feature data and second soybean physiological parameter data. The goal is to minimize the difference between the first and second soybean physiological parameter data, as well as the sum of the differences between the first soybean phenotypic feature data and the feature data corresponding to the multimodal fusion information. This process adjusts the parameters in the pre-constructed phenotypic prediction model to obtain the trained phenotypic prediction model. Through this process, the soybean phenotypic feature extraction module and the phenotypic environment mechanism module are optimized using a bidirectional feedback mechanism and a loss function. This constrains the prediction direction of the soybean phenotypic feature extraction module and ensures that it conforms to basic physiological laws.

[0059] In one embodiment of this application, when there is a large difference between the output of the soybean phenotypic feature extraction module and the output of the phenotypic environment mechanism module, the soybean multimodal phenotypic prediction system will trigger an alarm to prompt the operator to perform manual verification, thereby enhancing the robustness of the system and giving priority to the output of the phenotypic environment mechanism module that incorporates real-time data.

[0060] The cross-scenario adaptive learning and prediction output subsystem includes: training the trained phenotypic prediction model using the Model-Agnostic Meta-Learning (MAML) framework; using historical multimodal information, historical environmental information, and historical planting and management information from different soybean growth regions as the meta-training set to train the model, enabling it to adapt to different growth environments. The training process requires only a few gradient updates, allowing for rapid adaptation of the trained phenotypic prediction model to different growth regions, significantly reducing the need for large amounts of data for new scenarios. The soybean phenotypic prediction results output by the model include: predicted values ​​of soybean biomass, soybean yield, soybean plant height, soybean lodging resistance index, and soybean grain protein content, along with confidence intervals for these predictions and a visualized spatial distribution map.

[0061] In one embodiment of this application, a representative soybean variety (e.g., "Zhengdou 196") and an experimental field (e.g., the Kaifeng experimental field) are selected. Multimodal information is collected by drones according to the key growth stages V3 (three-leaf stage), R1 (initial flowering stage), R3 (connecting stage), R5 (grain filling stage), and R7 (maturity stage). The multimodal information collected historically and the corresponding feature data (measured phenotypic data) are used as a sample dataset. The sample dataset is divided into a training set, a validation set, and a test set in an 8:1:1 ratio. The Adam optimizer is used to minimize the loss function to train a pre-established phenotypic prediction model end-to-end, resulting in a soybean phenotypic prediction model. This model is then encapsulated as a cloud service or edge computing module. Users can upload drone data via a web interface or mobile app and receive a phenotypic prediction report with the soybean phenotypic prediction results as the main content after the model outputs the prediction.

[0062] The soybean multimodal phenotypic prediction system described in this application was tested in field trials in different ecological zones (e.g., the Northeast spring soybean region and the Huang-Huai-Hai summer soybean region). The prediction accuracy of the soybean multimodal phenotypic prediction system is shown in Table 1. Table 1 As shown in Table 1, the soybean multimodal phenotypic prediction system in this application significantly outperforms the comparative models (single hyperspectral modality model, pure data-driven model (static fusion), and traditional static Cross-Attention model) in multiple phenotypic prediction aspects. Specifically, in terms of lodging resistance, the prediction accuracy is improved by 11 percentage points compared to the comparative models, and in terms of yield, biomass, and chlorophyll content, the prediction accuracy is improved by 9 percentage points compared to the comparative models. This fully demonstrates the synergistic advantages of the feature extraction and dynamic attention fusion subsystem and the phenotypic environment mechanism feedback subsystem in the soybean multimodal phenotypic prediction system of this application.

[0063] The prediction accuracy of the soybean multimodal phenotypic prediction system in this application is characterized by the root mean square error (RMSE), as shown in Table 2: Table 2 As can be seen from Table 2, the cross-regional prediction error of the soybean multimodal phenotypic prediction system in this application is significantly lower than that of the comparative models (single hyperspectral modality model, pure data-driven model (static fusion), and traditional static Cross-Attention model). Among them, the lodging resistance error is reduced by 33% and the yield error is reduced by 35%, which fully verifies the effectiveness of the cross-scenario adaptive learning unit.

[0064] This application quantifies and integrates the contributions of historical multimodal data to different phenotypic features at different growth stages through a soybean phenotypic feature extraction module (introducing a hierarchical dynamic attention mechanism), avoiding information redundancy and conflict. Experiments show that the prediction accuracy R² for soybean phenotypic features exceeds 0.92. By introducing a meta-learning framework (MAML framework) and a phenotypic environment mechanism model, the soybean phenotypic prediction model can quickly adapt to new environments. Combining the soybean phenotypic feature extraction module with the phenotypic environment mechanism model effectively prevents prediction results that violate biological laws under extreme or unseen conditions, reducing cross-regional generalization error by more than 35%. The phenotypic environment mechanism model constrains the decision-making process of the "black box" deep learning model with physical laws, making the prediction results more biologically meaningful and enhancing the trust of breeders and agronomists in the model. It supports multimodal information input from the seedling stage to maturity, can dynamically track phenotypic change trajectories, and provides continuous and accurate information support for cultivation regulation and final yield prediction.

[0065] The following describes an embodiment of the apparatus described in this application, which can be used to execute a soybean multimodal phenotypic prediction method as described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the above embodiments of a soybean multimodal phenotypic prediction method described in this application.

[0066] Figure 4 This is a block diagram illustrating a soybean multimodal phenotypic prediction device according to an exemplary embodiment of this application.

[0067] like Figure 4 As shown, this exemplary soybean multimodal phenotypic prediction device 400 includes: The information acquisition module 401 is used to acquire multimodal information of soybean phenotype and environmental information of soybean growing area; The phenotypic prediction module 402 is used to input multimodal information and environmental information into the soybean phenotypic prediction model to obtain soybean phenotypic prediction results.

[0068] In one embodiment of this application, the multimodal information of soybean phenotype includes: soybean leaf information, soybean canopy information, soybean plant information, biomass information, yield information, etc., and environmental information includes meteorological environmental information and soil environmental information, etc. Meteorological environmental information includes: air temperature, air humidity, precipitation, wind force, air pressure, light intensity, carbon dioxide concentration, etc., and soil environmental information includes: soil temperature, soil moisture, soil pH, soil texture, soil density and porosity, soil hydraulic conductivity, organic matter content, nutrient content, microbial activity, microbial diversity, etc.

[0069] In one embodiment of this application, the soybean phenotype prediction model is obtained by training a pre-constructed phenotype prediction model with sample data. The pre-constructed phenotype prediction model includes a soybean phenotype feature extraction module and a phenotype environment mechanism module. The above process, by considering the multimodal information of soybean phenotype and the environmental information of soybean growth area, and by combining the soybean phenotype feature extraction module and the phenotype environment mechanism module, predicts soybean phenotype, improves the accuracy of soybean phenotype prediction results, and is applicable to different growth environments, thus improving the applicability of the soybean phenotype prediction model.

[0070] It should be noted that the soybean multimodal phenotypic prediction device and the soybean multimodal phenotypic prediction method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the soybean multimodal phenotypic prediction device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0071] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, enable the electronic device to implement the soybean multimodal phenotypic prediction method provided in the above embodiments.

[0072] Figure 5 This is a schematic diagram illustrating the structure of a computer system for an electronic device, as shown in an exemplary embodiment of this application. It should be noted that... Figure 5 The computer system 600 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0073] like Figure 5 As shown, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on programs stored in Read-Only Memory (ROM) 502 or programs loaded from storage portion 508 into Random Access Memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0074] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0075] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs various functions defined in the system of this application.

[0076] Another aspect of this application provides a computer-readable storage medium storing computer-readable instructions that, when executed by a computer's processor, cause the computer to perform the soybean multimodal phenotypic prediction method provided in the above embodiments. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0077] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0078] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A soybean multi-modal phenotype prediction method, characterized in that, The method comprises: obtaining multi-modal information of soybean phenotypes and environmental information of soybean growth areas; inputting the multi-modal information and the environmental information into a soybean phenotype prediction model to obtain a soybean phenotype prediction result; the soybean phenotype prediction model is obtained by training a pre-constructed phenotype prediction model based on sample data; the pre-constructed phenotype prediction model comprises a soybean phenotype feature extraction module and a phenotype environment mechanism module.

2. The soybean multi-modal phenotype prediction method of claim 1, wherein, If the sample data comprises historical multi-modal information, historical environmental information and historical planting management information of soybeans in multiple growth areas, the process of training the pre-constructed phenotype prediction model based on the sample data to obtain the soybean phenotype prediction model comprises: training the pre-constructed phenotype prediction model based on the historical multi-modal information to obtain a trained phenotype prediction model; training the trained phenotype prediction model based on the historical multi-modal information, the historical environmental information and the historical planting management information to obtain the soybean phenotype prediction model.

3. The soybean multi-modal phenotype prediction method of claim 2, wherein, The process of training the pre-constructed phenotype prediction model based on the historical multi-modal information to obtain a trained phenotype prediction model comprises: preprocessing and spatio-temporal alignment of the historical multi-modal information to obtain multi-modal fusion information; the preprocessing comprises a denoising operation; inputting the multi-modal fusion information into the soybean phenotype feature extraction module to obtain soybean phenotype feature data; inputting the soybean phenotype feature data into the phenotype environment mechanism module to obtain first soybean physiological parameter data; the phenotype environment mechanism module comprises a biomass accumulation mechanism model, a yield formation mechanism model and a lodging resistance mechanism model; the first soybean physiological parameter data comprises a biomass theoretical value, a yield theoretical value and a lodging resistance index theoretical value; inputting the first soybean physiological parameter data into the soybean phenotype feature extraction module to obtain first soybean phenotype feature data and second soybean physiological parameter data; adjusting parameters in the pre-constructed phenotype prediction model to obtain the trained phenotype prediction model, with the objective of minimizing the difference between the first soybean physiological parameter data and the second soybean physiological parameter data, and the sum of the difference between the first soybean phenotype feature data and the corresponding feature data of the multi-modal fusion information.

4. The soybean multi-modal phenotype prediction method of claim 3, wherein, If the historical multi-modal information comprises soybean leaf and canopy spectral data, soybean plant image data and soybean plant point cloud data, the process of preprocessing and spatio-temporal alignment of the historical multi-modal information to obtain multi-modal fusion information comprises: performing radiation calibration, atmospheric correction, reflectivity inversion and spectral resampling on the soybean leaf and canopy spectral data to obtain denoised spectral data; performing photometric correction and color balance on the soybean plant image data to obtain orthographic image data; performing point cloud denoising and point cloud classification on the soybean plant point cloud data to obtain classified point cloud data, and generating a digital surface model and a canopy height model based on the classified point cloud data; The denoised spectral data is matched with the data points of the orthographic image data to obtain image matching data; and the image matching data is matched with the digital surface model and the canopy height model to obtain the multi-modal fusion information.

5. The soybean multi-modal phenotype prediction method of claim 3, wherein, If the soybean phenotype feature extraction module includes a spectral feature extraction module, an image feature extraction module, and a point cloud feature extraction module, a dynamic weight generation module, and a weighted cross-attention module, the process of inputting the multi-modal fusion information into the soybean phenotype feature extraction module to obtain soybean phenotype feature data includes: inputting the multi-modal fusion information into the spectral feature extraction module to obtain spectral feature data; inputting the multi-modal fusion information into the image feature extraction module to obtain image feature data; inputting the multi-modal fusion information into the point cloud feature extraction module to obtain point cloud feature data; inputting the spectral feature data, the image feature data, and the point cloud feature data into the dynamic weight generation module to output the weight of the spectral feature data, the weight of the image feature data, and the weight of the point cloud feature data; the dynamic weight generation module is configured to determine the weight of the spectral feature data, the weight of the image feature data, and the weight of the point cloud feature data according to the growth period indicating features extracted from the spectral feature data, the growth period indicating features extracted from the image feature data, and the growth period indicating features extracted from the point cloud feature data; inputting the spectral feature data, the image feature data, the point cloud feature data, the weight of the spectral feature data, the weight of the image feature data, and the weight of the point cloud feature data into the weighted cross-attention module to obtain the soybean phenotype feature data.

6. The soybean multi-modal phenotype prediction method of claim 5, wherein, The calculation formula of the weighted cross-attention module is as follows: ， wherein, denotes soybean phenotypic trait data, denotes a query matrix, a matrix of the current modality data, denotes a key matrix, denotes a value matrix, denotes an activation function, denotes a weight of the key matrix, denotes a matrix of the first modality data as a key matrix, denotes a matrix of the second modality data as a key matrix, denotes a scaling factor, denotes a weight of the value matrix, denotes a matrix of the first modality data as a value matrix, denotes a matrix of the second modality data as a value matrix.

7. The soybean multi-modal phenotype prediction method of claim 3, wherein, The difference between the first soybean physiological parameter data and the second soybean physiological parameter data, and the sum of the differences between the first soybean phenotype feature data and the corresponding feature data of the multi-modal fusion information are represented by a loss function, and the expression of the loss function is as follows: ， in, Represents the loss function. This represents the mean squared error loss function. The mechanism consistency loss function is represented. Indicates the first Feature data corresponding to modality fusion information obtained by preprocessing and spatiotemporal alignment of sample data. This indicates that the soybean phenotypic feature extraction module extracts the first... The first soybean phenotypic characteristic data was obtained by predicting from a sample of data. Represents the regularization coefficient. This indicates that the soybean phenotypic feature extraction module extracts the first... The second soybean physiological parameter data was obtained by predicting from a sample of data. This indicates that the phenotypic environment mechanism module is responsible for the first... The first soybean physiological parameter data was obtained by predicting from a sample of data.

8. The soybean multi-modal phenotyping prediction method of any one of claims 2-5, wherein, The process of training the trained phenotype prediction model by the historical multi-modal information, the historical environmental information, and the historical planting management information to obtain the soybean phenotype prediction model includes: annotating the features of the historical multi-modal information to obtain historical multi-modal information with feature annotation information; establishing a corresponding relationship between the historical multi-modal information with feature annotation information and the historical environmental information and the historical planting management information, denoted as corresponding relationship data; training the trained phenotype prediction model by the corresponding relationship data to obtain the soybean phenotype prediction model.

9. A soybean multi-modal phenotype prediction apparatus, characterized by, It includes: an information acquisition module configured to acquire multi-modal information of soybean phenotypes and environmental information of soybean growth areas; a phenotype prediction module configured to input the multi-modal information and the environmental information into a soybean phenotype prediction model to obtain a soybean phenotype prediction result; and a phenotype prediction module configured to input the multi-modal information and the environmental information into a soybean phenotype prediction model to obtain a soybean phenotype prediction result. The soybean phenotype prediction model is obtained by training a pre-constructed phenotype prediction model through sample data. The pre-constructed phenotype prediction model comprises a soybean phenotype feature extraction module and a phenotype environment mechanism module.

10. An electronic device, comprising: Comprise: One or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, so that the electronic device realizes the soybean multi-modal phenotype prediction method as claimed in any one of claims 1 to 8.

Images