Soybean germplasm resource viability small sample classification prediction method, system and equipment

Abstract

The invention discloses a soybean germplasm resource viability small sample classification prediction method, system and device, and relates to the technical field of soybean germplasm resource viability classification, and the method comprises the steps: obtaining a multi-modal data set of sample soybean germplasm resources; the multi-modal data set is preprocessed; based on the preprocessed multi-modal data set, training a soybean germplasm small sample viability detection model to obtain a soybean seed viability detection model; hyperspectral data and volatilization data of the soybean seeds to be detected are obtained, the soybean seed viability detection model is used for carrying out viability detection on the soybean seeds to be detected, and the viability detection result of the soybean seeds to be detected is obtained. According to the method, the hyperspectral data and the volatilization data of the soybean seeds to be detected are integrated, and the internal chemical characteristics and the volatile substances released from the outside of the soybean seeds are comprehensively considered, so that the vitality of the soybean seeds is more comprehensively and accurately evaluated.

Description

Technical Field

[0001] The present invention relates to the technical field of soybean germplasm resource vitality classification, and in particular to a small sample classification prediction method, system and equipment for soybean germplasm resource vitality. Background Art

[0002] Soybean seeds gradually lose viability during the natural aging process. Accurately measuring the viability of naturally aged soybean seeds is crucial for seed quality control, germplasm conservation, and agricultural production. However, the slow aging of soybeans and the extremely limited sample size present challenges in developing models for detecting the viability of naturally aged soybean germplasm. Small sample sizes lead to an uneven distribution of data categories, and traditional detection techniques struggle to fully capture seed characteristics, resulting in a severe lack of detection capabilities for a few categories and insufficient accuracy to meet practical requirements. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to address the deficiencies of the existing technology and specifically provide a method, system and device for classifying and predicting the vitality of soybean germplasm resources in a small sample, as follows:

[0004] 1) In the first aspect, the present invention provides a method for classifying and predicting the vitality of soybean germplasm resources using a small sample size. The specific technical solution is as follows:

[0005] Obtain hyperspectral data and volatile data of each sample soybean seed in the sample soybean germplasm resources, and obtain the vitality label of each sample soybean seed in the sample soybean germplasm resources to form a multimodal dataset of the sample soybean germplasm resources;

[0006] Preprocessing of multimodal datasets;

[0007] Based on the preprocessed multimodal dataset, a soybean germplasm small sample vitality detection model was trained to obtain a soybean seed vitality detection model.

[0008] Hyperspectral data and volatile data of soybean seeds to be tested are obtained, and a soybean seed vitality detection model is used to perform vitality detection on the soybean seeds to obtain vitality detection results of the soybean seeds to be tested.

[0009] The beneficial effects of the small sample classification prediction method for soybean germplasm vitality provided by the present invention are as follows:

[0010] Soybean seeds release different volatile organic compounds (VOCs) at different viability levels. The types and concentrations of VOCs can provide additional information about the physiological state and viability of soybean seeds. A soybean seed viability detection model integrates hyperspectral and volatile data from the tested soybean seeds, comprehensively considering both the internal chemical properties of the seeds and the volatile compounds released externally. This allows for a more comprehensive and accurate assessment of seed viability. Furthermore, a small amount of sample data (hyperspectral, volatile, and viability labels) can be used to train a viability detection model with high classification accuracy. This is due to the following reasons: 1) Hyperspectral data contains rich spectral information. Hyperspectral imaging technology can acquire large amounts of continuous spectral data across the visible and near-infrared bands of the electromagnetic spectrum. The spectral reflectance and absorption characteristics of soybean seeds in specific wavelengths are closely related to seed viability. For example, the integrity of the seed's cellular structure and physiological metabolic activity can affect its absorption and reflectance. These spectral features can reflect the seed's viability at the molecular level, providing a detailed identification basis for model training. 2) Volatile data also has a high degree of specificity. Under different states of viability, soybean seeds produce different volatile organic compounds (VOCs) through their internal biochemical processes. These VOCs can serve as a chemical fingerprint of seed viability. For example, seeds with high viability may have active respiration and metabolism, producing specific volatile compounds such as aldehydes, alcohols, and esters, while seeds with low viability may exhibit significant variations in the types and content of these compounds. This rich feature information allows even a small sample size to contain sufficient patterns for model learning. (3) Vitality labels are generally obtained through relatively accurate experimental methods, such as tetrazolium staining and germination experiments. These labels clearly indicate whether a seed has high, medium, or low viability. During model training, the model can learn the correspondence between the sample's hyperspectral and volatile data and the clear viability labels. Even with a small number of samples, as long as the labels are accurate, the model can construct effective classification or prediction rules by mining the label-related features in the data. (4) The proposed soybean germplasm small sample viability detection model has powerful feature extraction and pattern recognition capabilities and can also continuously adjust parameters through a backpropagation algorithm to minimize the error between the predicted value and the actual viability label. Even with only a small number of samples, as long as the data features of these samples can be effectively extracted and utilized by the model, the model can learn the key discriminant features of seed viability, thereby constructing a high-quality soybean seed viability detection model. ⑤ If a small number of samples can well represent the main types of soybean seeds with different viability states, then the trained model is likely to be of higher quality.For example, by designing a rational experiment, selecting seeds with varying viability levels (e.g., high, medium, and low) as samples, and ensuring that these samples have a certain degree of diversity in other factors such as variety and origin, these samples can cover the key feature space for soybean seed viability testing. After the model is trained on these representative samples, it can generalize to a wider range of seed samples to a certain extent.

[0011] On the basis of the above scheme, the small sample classification prediction method of soybean germplasm resource vitality of the present invention can also be improved as follows.

[0012] Furthermore, the multimodal dataset is preprocessed, including:

[0013] The hyperspectral data in the multimodal dataset are subjected to multivariate scattering correction, SG filtering, normalization and second-order derivative processing channel by channel; the volatile data in the multimodal dataset are subjected to SG filtering.

[0014] The beneficial effect of adopting the above further solution is that the accuracy of the vitality test results of the soybean seeds to be tested can be further improved.

[0015] Furthermore, the soybean germplasm small sample vitality detection model includes a feature extraction part, a feature fusion part and a decision fusion part, or the soybean germplasm small sample vitality detection model includes a feature extraction part and a feature fusion part;

[0016] The feature extraction part includes a volatile feature extraction module and a hyperspectral feature extraction module. The volatile feature extraction module includes time series convolution and uses continuous residual connections to obtain information between multiple time dimensions from the preprocessed volatile data to obtain time series data features. The hyperspectral feature extraction module includes a one-dimensional convolutional neural network and uses continuous residual connections to obtain spectral data features from the preprocessed hyperspectral data.

[0017] The feature fusion part includes a shallow feature fusion module and a deep feature fusion module. The shallow feature fusion module performs local weighted fusion of time series data features and spectral data features through one-dimensional convolution and Sigmoid activation function to obtain the first fusion result; the deep feature fusion module processes the first fusion result through a multi-head hybrid self-attention module to obtain the second fusion result;

[0018] In the decision fusion part, one-dimensional convolution is used to map the time series data features and spectral data features into two equal-dimensional classification scores, and the two classification scores are fused with the classification results of the feature fusion module obtained based on the second fusion result to obtain the vitality detection results of the tested soybeans.

[0019] Furthermore, hyperspectral data and volatile data of the sample soybean germplasm resources are obtained, and the vitality label of each sample soybean seed in the sample soybean germplasm resources is obtained, including:

[0020] Hyperspectral data of sample soybean germplasm resources were obtained using hyperspectral equipment, volatile data of sample soybean germplasm resources were obtained through olfactory detection equipment, and the vitality label of each sample soybean seed in the sample soybean germplasm resources was obtained through standard germination test. The vitality label was germination or no germination.

[0021] Furthermore, it also includes:

[0022] Through the visual interactive interface, the user's operation instructions are received to complete the vitality detection of the soybeans to be tested.

[0023] The beneficial effect of adopting the above further solution is: receiving user operation instructions through a visual interactive interface can make operations more intuitive and convenient, improve efficiency, reduce user learning costs, and enhance user experience.

[0024] 2) In a second aspect, the present invention further provides a small sample classification prediction system for soybean germplasm vitality, the specific technical solution of which is as follows:

[0025] It includes data acquisition module, data preprocessing module, data training module and prediction module;

[0026] The data acquisition module is used to obtain the hyperspectral data and volatile data of each sample soybean seed in the sample soybean germplasm resources, and obtain the vitality label of each sample soybean seed in the sample soybean germplasm resources to form a multimodal dataset of the sample soybean germplasm resources;

[0027] The data preprocessing module is used to: preprocess the multimodal dataset;

[0028] The data training module is used to: train a soybean germplasm small sample vitality detection model based on the preprocessed multimodal data set to obtain a soybean seed vitality detection model;

[0029] The prediction module is used to obtain the hyperspectral data and volatile data of the soybean seeds to be tested, and use the soybean seed vitality detection model to perform vitality detection on the soybean seeds to be tested, and obtain the vitality detection results of the soybean seeds to be tested.

[0030] Based on the above scheme, the soybean germplasm resource vitality small sample classification prediction system of the present invention can also be improved as follows.

[0031] Furthermore, the data preprocessing module is specifically used to: perform multivariate scattering correction processing, SG filtering processing, normalization processing and second-order derivative processing on the hyperspectral data in the multimodal data set channel by channel; and perform SG filtering processing on the volatile data in the multimodal data set.

[0032] Furthermore, the soybean germplasm small sample vitality detection model includes a feature extraction part, a feature fusion part and a decision fusion part, or the soybean germplasm small sample vitality detection model includes a feature extraction part and a feature fusion part;

[0033] The feature extraction part includes a volatile feature extraction module and a hyperspectral feature extraction module. The volatile feature extraction module includes time series convolution and uses continuous residual connections to obtain information between multiple time dimensions from the preprocessed volatile data to obtain time series data features. The hyperspectral feature extraction module includes a one-dimensional convolutional neural network and uses continuous residual connections to obtain spectral data features from the preprocessed hyperspectral data.

[0034] The feature fusion part includes a shallow feature fusion module and a deep feature fusion module. The shallow feature fusion module performs local weighted fusion of time series data features and spectral data features through one-dimensional convolution and Sigmoid activation function to obtain the first fusion result; the deep feature fusion module processes the first fusion result through a multi-head hybrid self-attention module to obtain the second fusion result;

[0035] In the decision fusion part, one-dimensional convolution is used to map the time series data features and spectral data features into two equal-dimensional classification scores, and the two classification scores are fused with the classification results of the feature fusion module obtained based on the second fusion result to obtain the vitality detection results of the tested soybeans.

[0036] Furthermore, the data acquisition module is specifically used to: obtain hyperspectral data of sample soybean germplasm resources using hyperspectral equipment, obtain volatile data of sample soybean germplasm resources through olfactory detection equipment, and obtain the vitality label of each sample soybean seed in the sample soybean germplasm resources through a standard germination test, and the vitality label is germination or no germination.

[0037] Furthermore, the system also includes an interactive module, which is used to receive user operation instructions through a visual interactive interface to complete the vitality detection of the soybeans to be tested.

[0038] 3) In a third aspect, the present invention further provides an electronic device, comprising a processor coupled to a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor, so that the electronic device implements any of the above-mentioned methods for classifying and predicting small samples of soybean germplasm resource vitality.

[0039] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned methods for classifying and predicting small samples of soybean germplasm resource vitality.

[0040] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments of the present invention:

[0042] Figure 1 This is a flow chart of a method for classifying and predicting the vitality of soybean germplasm resources using a small sample size according to an embodiment of the present invention;

[0043] Figure 2 This is one of the structural diagrams of the soybean germplasm small sample vitality detection model;

[0044] Figure 3 This is the second structural diagram of the soybean germplasm small sample vitality detection model;

[0045] Figure 4 This is the principle block diagram of the data fusion model for soybean germplasm small sample vitality detection;

[0046] Figure 5 This is the interface diagram of Transformer soybean germplasm small sample vitality detection software;

[0047] Figure 6 This is a schematic structural diagram of a small sample classification prediction system for soybean germplasm vitality according to an embodiment of the present invention;

[0048] Figure 7 The figure is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0050] The following describes in detail the technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems using specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following embodiments of the present invention are described in conjunction with the accompanying drawings.

[0051] like Figure 1 As shown, a small sample classification prediction method for soybean germplasm vitality according to an embodiment of the present invention includes the following steps:

[0052] S1. Obtain hyperspectral data and volatile data of each sample soybean seed in the sample soybean germplasm resources, and obtain the vitality label of each sample soybean seed in the sample soybean germplasm resources to form a multimodal dataset of the sample soybean germplasm resources;

[0053] Among them, hyperspectral data of sample soybean germplasm resources are obtained using hyperspectral equipment, volatile data of sample soybean germplasm resources are obtained through olfactory detection equipment, and the vitality label of each sample soybean seed in the sample soybean germplasm resources is obtained through standard germination test, and the vitality label is germination or no germination.

[0054] Among them, based on the small sample conditions of soybeans, intact seeds are selected and the test process is ensured to be correct to ensure the validity of the sample.

[0055] S2. Preprocess the multimodal dataset, specifically:

[0056] Multivariate scattering correction, SG filtering, normalization, and second-order derivative processing were performed on the hyperspectral data in the multimodal dataset channel by channel. SG filtering was also performed on the volatile data in the multimodal dataset to further improve the accuracy of the soybean seed viability test results. The details are further explained as follows:

[0057] The analysis is performed during a period of stable 3D volatility data acquisition to reduce environmental interference and more accurately reflect the true state of the measured gas. SG filtering is then performed on the 3D volatility data sensor by sensor.

[0058] To address potential issues with hyperspectral data collection, including spectral scattering caused by seed sample size and shape, instrument noise, differences in spectral reflectance scales between different seed samples, and the identification of overlapping and weak peaks in near-infrared spectra, multivariate scatter correction and SG filtering, as well as normalization and second-order derivative processing, are performed on each channel of the hyperspectral data. This makes the hyperspectral preprocessing of soybean seeds more targeted and flexible. All characteristic bands of the hyperspectral preprocessed data are selected for modeling.

[0059] S3. Based on the preprocessed multimodal dataset, a soybean germplasm small sample vitality detection model is trained to obtain a soybean seed vitality detection model. Specifically:

[0060] The preprocessed multimodal dataset was divided into training set and test set, and the soybean germplasm small sample vitality detection model was trained based on the training set and test set.

[0061] Among them, the soybean germplasm small sample vitality detection model can also be called: feature decision fusion multimodal Transformer soybean germplasm small sample vitality detection algorithm.

[0062] S4. Acquire hyperspectral data and volatile data of the soybean seeds to be tested, and use a soybean seed vitality detection model to perform vitality detection on the soybean seeds to obtain vitality detection results of the soybean seeds to be tested.

[0063] Soybean seeds release different volatile organic compounds (VOCs) at different viability levels. The types and concentrations of VOCs provide additional information about the physiological state and viability of the seeds. The soybean seed viability detection model integrates hyperspectral and volatile data from the tested soybean seeds, comprehensively considering the internal chemical properties of the seeds and the volatile compounds released externally, to provide a more comprehensive and accurate assessment of seed viability.

[0064] Optionally, in the above technical solution, if Figure 2 and Figure 3 As shown, the soybean germplasm small sample vitality detection model includes a feature extraction part, a feature fusion part and a decision fusion part, or the soybean germplasm small sample vitality detection model includes a feature extraction part and a feature fusion part;

[0065] The feature extraction part includes a volatile feature extraction module and a hyperspectral feature extraction module. The volatile feature extraction module includes multiple time series convolutions and uses continuous residual connections. Specifically, the output of the first time series convolution is used as the input of the second time series convolution and the input of the last time series convolution respectively. The output of the second time series convolution is used as the input of the third time series convolution and the input of the second to last time series convolution respectively. And so on, continuous residual connections are realized for multiple time series convolutions to obtain information between multiple different time dimensions from the preprocessed volatile data and obtain time series data features; The hyperspectral feature extraction module includes multiple one-dimensional convolutional neural networks and uses continuous residual connections. Specifically, the output of the first one-dimensional convolutional neural network is used as the input of the second one-dimensional convolutional neural network and the input of the last one-dimensional convolutional neural network respectively. The output of the second one-dimensional convolutional neural network is used as the input of the third one-dimensional convolutional neural network and the input of the second to last one-dimensional convolutional neural network respectively. And so on, continuous residual connections are realized for multiple one-dimensional convolutional neural networks to obtain spectral data features from the preprocessed hyperspectral data;

[0066] The feature fusion part includes a shallow feature fusion module and a deep feature fusion module. The shallow feature fusion module performs local weighted fusion on the time series data features and the spectral data features through one-dimensional convolution and Sigmoid activation function to obtain a first fusion result. Specifically, the shallow feature fusion module respectively processes the time series data features and the spectral data features through a one-dimensional convolution, splices the outputs of the two one-dimensional convolutions, and then inputs a one-dimensional convolution to split the one-dimensional convolution to obtain data corresponding to the time series data features and data corresponding to the spectral data features. These two data are respectively subjected to one-dimensional convolution and Sigmoid activation function, and are element-wise multiplied with the corresponding time series data features or spectral data features after one-dimensional convolution processing to obtain two output results. The two output results are adaptively fused to obtain an adaptive fusion output graph. The first fusion result includes these two output results and the adaptive fusion output graph.

[0067] The deep feature fusion module processes the first fusion result through the multi-head hybrid self-attention module to obtain the second fusion result. Specifically, the two output results of the first fusion result are layer-normalized and then input into the multi-head hybrid self-attention module (MHCA). The output of the multi-head hybrid self-attention module is processed by a feed-forward fully connected layer and element-wise added to the adaptive fusion output map to obtain the output of the deep feature fusion module. The output of the deep feature fusion module is processed by the classification layer (linear classification layer) to obtain the classification result.

[0068] The decision fusion part uses one-dimensional convolution to map the time series data features and spectral data features into two equal-dimensional classification scores, and fuses the two classification scores with the classification results of the feature fusion module obtained based on the second fusion result to obtain the vitality detection results of the tested soybeans.

[0069] The soybean germplasm small sample vitality detection model includes a feature extraction part and a feature fusion part, or the soybean germplasm small sample vitality detection model includes a feature extraction part, a feature fusion part and a decision fusion part;

[0070] When the soybean germplasm small sample vitality detection model only includes a feature extraction part and a feature fusion part, the classification result of the feature fusion module obtained based on the second fusion result is directly used as the vitality detection result of the soybean to be tested. Specifically, the hyperspectral data and volatile data of the soybean seeds to be tested are input into the feature extraction part for feature extraction, obtaining spectral data features and time series data features respectively. The spectral data features and time series data features are then fused by the shallow feature fusion module of the feature fusion part to obtain the first fusion result, which is then input into the deep feature fusion module for fusion processing to obtain feature-level deep fusion features. The output of the deep feature fusion module is processed through a multi-head hybrid self-attention module and a fully connected layer, and finally input into the linear classification module to obtain the classification result of the feature fusion module as the vitality detection result of the soybean to be tested.

[0071] When the soybean germplasm small sample vitality detection model includes a feature extraction part, a feature fusion part, and a decision fusion part, the classification score calculated by the decision fusion part is fused with the first fusion result to obtain the classification result of the decision fusion module as the vitality detection result of the soybean to be tested. Specifically, the feature extraction module is used to calculate the hyperspectral data and volatile data of the soybean seeds to be tested to obtain time series data features and spectral data features, and then the feature fusion part is used to calculate the time series data features and spectral data features to obtain the classification result of the feature fusion module, and the linear classification layer is used to calculate the time series data features and spectral data features to obtain the time series data feature classification result and the spectral data feature classification result. Finally, the decision fusion module is used to calculate the fusion result of the feature fusion module classification result, the time series data feature classification result, and the spectral data feature classification result, and the decision fusion module is used to obtain the classification result of the decision fusion module as the vitality detection result of the soybean to be tested.

[0072] Both the feature fusion part and the decision fusion part are connected to the feature extraction part to realize the rapid extraction of features from a small number of samples; the feature extraction part can be connected to the feature fusion part or the decision fusion part. According to the type and characteristics of the input small sample data, the operating parameters and processing flow of each part can be adjusted to realize a multi-level fusion process of data from input to output, greatly improving processing efficiency.

[0073] The soybean germplasm small-sample viability detection model deeply integrates feature extraction and decision fusion functions to address the overfitting of classification models due to the limited size of small sample data and the inability to fully extract seed characteristics and patterns. The model training process involves partitioning the dataset into training and test datasets, fusing the incoming spectral and volatile modal training datasets, and then training the fused dataset. During the training process, the weights and learning rates are adjusted in real time based on performance metrics such as the loss function of each model on the training dataset to obtain the final decision model. The model prediction process involves receiving the spectral and volatile test datasets of the small sample soybean seeds to be predicted, loading the trained decision model, and then performing feature extraction and decision fusion steps to obtain the viability detection results. The feature extraction function uses a hybrid feature extraction strategy to obtain cross-modal joint features, fully leveraging the correlation and complementary information between the two modal data of the small sample soybean seeds. To address the challenge of insufficient stability of individual classification models caused by the uneven distribution of small sample data, the decision fusion function effectively integrates the viability detection results from the volatile feature extraction module, the hyperspectral feature extraction module, and the feature fusion component, improving the reliability of the detection results.

[0074] The proposed soybean germplasm small sample viability detection model comprises feature extraction, feature fusion, and decision fusion. This algorithm integrates spectral and volatile data processing processes, creating a flexible and efficient integrated analysis system. Users can select three modules, the first two modules, or the last two modules to form different fusion models based on their specific needs: decision-level fusion for the three modules, feature-level fusion for the first two modules, and data-level fusion for the last two modules. This model fully leverages the vitality information contained in hyperspectral imagery and volatile signals, systematically analyzing and integrating deep and shallow features across modalities, and achieving the rational fusion of deep and shallow features across modalities.

[0075] The feature extraction component consists of two modules: volatile feature extraction and hyperspectral feature extraction, which are used to extract features from time series and spectral data. The volatile feature extraction module utilizes a time series convolutional neural network, using continuous residual connections to capture information across multiple time dimensions of volatility, enhancing the model's volatile feature extraction capabilities. The hyperspectral feature extraction module utilizes a one-dimensional convolutional neural network, using continuous residual connections to address the limited receptive field of one-dimensional convolutional neural networks. This combination of the two allows for the extraction of more discriminative viability features from a small sample size, laying the foundation for viability testing of small soybean germplasm samples.

[0076] The feature fusion component consists of shallow feature fusion and deep feature fusion. The shallow feature fusion module performs a locally weighted fusion of the output features of the volatile feature extraction and hyperspectral feature extraction modules using one-dimensional convolution and sigmoid activation functions. The deep feature fusion module interactively fuses volatile and hyperspectral features, centered around a multi-head hybrid self-attention module. Compared to late fusion, the deep feature fusion module utilizes simultaneous extraction and fusion, fully integrating volatile and hyperspectral features and avoiding feature loss caused by late fusion. This shallow and deep fusion approach fully captures the differential spectral information between samples with and without viability, laying the foundation for viability testing of small soybean germplasm samples.

[0077] The decision fusion component adds two linear classification layers and a decision fusion module to the feature-level fusion germplasm viability classification detection model. First, a one-dimensional convolution is used to map the classification results of the volatile feature extraction module and the hyperspectral feature extraction module into two equal classification scores. Next, the two mapped scores are fused with the classification results of the feature fusion module to obtain a fused output classification score. The decision fusion component combines, coordinates, and integrates the viability detection results from the volatile feature extraction, hyperspectral feature extraction, and feature fusion components to obtain more comprehensive, accurate, and reliable soybean germplasm viability decision detection results.

[0078] Optionally, in the above technical solution, the following is further included:

[0079] like Figure 5 As shown, through the visual interactive interface, the user's operation instructions are received to complete the vitality test of the soybean to be tested, specifically:

[0080] First, before the test begins, click Start System in the System Startup and Termination window to display the various windows of the interface. The upload data window is used to upload hyperspectral data. Select the .hdr file and the .dat file in turn according to the prompt box. After successful selection, the corresponding hyperspectral curve will be displayed in the middle graphic window 1. Click Volatile Data to upload the volatile data in .nos format. After successful selection, the volatile response curve will be displayed in the middle graphic window 2. Then, select one of them in the hyperspectral preprocessing window to perform corresponding preprocessing on the incoming hyperspectral data. Select one of them in the volatile preprocessing window to perform corresponding preprocessing on the incoming volatile data. Select one of the three levels, data level, feature level, and decision level, in the multimodal data fusion window to call the corresponding trained model to predict the vitality of the incoming preprocessed data and obtain the vitality prediction result. When the data level is selected, call Figure 4 The soybean germplasm small sample vitality detection data fusion model shown in the figure is used to predict vitality. When the feature level is selected, the Figure 2 The soybean germplasm small sample vitality detection model shown in the figure is used to predict vitality. When the decision level is selected, the Figure 3 The shown soybean germplasm small sample viability detection model performs viability prediction. Click Start Analysis below in the Data Analysis window to transfer the viability prediction results. The middle table window displays the seed serial number and the presence or absence of viability predictions. Click Save Results in the Save Results window to save the results in the table as an Excel file to the corresponding path. After the test is completed, click Terminate System in the System Startup and Termination window to close all windows on the interface.

[0081] The above-mentioned hyperspectral data and volatile formats are not limited to the above, and data reading codes can be written accordingly for reading.

[0082] The production process of the above user interface is as follows: define the UI_windows class to encapsulate the settings and layout related to the user interface, create seven QGroupBox objects under the central window to set the size, position layout and style of the related button components and display them in groups, create two QGraphicsView graphics view objects to set the size and position layout for displaying two-dimensional spectral images and volatility response curves, create a QTableWidget table control object to set the size and position layout to display table data; multiple control signals in the window are connected to the slot function through the connectSlotsByName function to execute the response signal, and the response function can be triggered by pressing the key.

[0083] Define the retranslateUi function to assign relevant titles and text content to the above user interface components, and use the setTitle method to assign group box titles to the seven QGroupBox objects, namely data reading, hyperspectral preprocessing, volatile preprocessing, multimodal data fusion, data analysis, result saving, and system startup and termination. The setText method sets the text of the two buttons under the upload data group box to hyperspectral data and volatile data to instruct the user to select the corresponding file, sets the text of two or more buttons under the hyperspectral and volatile preprocessing group boxes to the corresponding preprocessing method to instruct the user to select the corresponding processing, sets the text of the three buttons under the multimodal data fusion group box to data level, feature level, and decision level fusion, sets the text of the button under the data analysis group box to start analysis, sets the text of the button under the result saving group box to save results, and sets the text of the two buttons under the system startup and termination group box to start the system and terminate the system.

[0084] The present invention will be described in more detail by the following examples:

[0085] Hyperspectral data and volatile phase response data of soybean seeds are collected using hyperspectral equipment and olfactory detection equipment, and vitality labels of whether the seeds have germinated are obtained through standard germination tests. The collected hyperspectral data and volatile phase response data are preprocessed to obtain preprocessed hyperspectral data and volatile phase response data; then a data set is made, that is, the preprocessed multimodal data set is divided into a training set and a test set in a certain proportion, an initial learning rate and a maximum training round are given, and when the current training round is less than the given maximum training round, the training data set is passed to the soybean germplasm small sample vitality detection model, and the loss value is calculated in each model training to update the training model weight and learning rate, and it is iterated into the next training round until the maximum training round model output weight file is reached, and the model and the weight file are called in the test data set, and vitality detection is performed on the test data set to obtain the soybean germplasm resource vitality detection result, which is the discrimination result of whether there is vitality.

[0086] The above-mentioned soybean germplasm small sample vitality detection model can be divided into data set model, feature-level model and decision-level model, which can be selected according to specific needs.

[0087] The operating process of the software program used in the system of the present invention is as follows: first, before the test starts, click on the system start and end window to start the system and display various windows of the interface. Upload hyperspectral data in the Data Upload window. Select the .hdr file and the .dat file according to the prompts. Successful selections will display the corresponding hyperspectral curve in the first middle graph window. Click Volatile Data to upload volatile data in .nos format. Successful selection will display the volatile response curve in the second middle graph window. Then, select one of the following options in the Hyperspectral Preprocessing window to preprocess the incoming hyperspectral data. Select one of the following options in the Volatile Preprocessing window to preprocess the incoming volatile data. In the Multimodal Data Fusion window, select one of the three levels: Data Level, Feature Level, or Decision Level. The trained model will be used to predict viability of the incoming preprocessed data, resulting in a viability prediction result. Click Start Analysis below in the Data Analysis window to transfer the viability prediction result. The seed number and viability prediction result will be displayed in the middle table window. Click Save Results in the Save Results window to save the table data as an Excel file to the corresponding path. After the test is complete, click Terminate System in the System Startup and Termination window to close all windows.

[0088] The above-mentioned hyperspectral data and volatile formats are not limited to the above, and data reading codes can be written accordingly for reading;

[0089] The structure of the feature decision fusion multimodal Transformer soybean germplasm small sample vitality detection software of the present invention is as follows:

[0090] Define the UI_windows class to encapsulate the settings and layout related to the user interface, create seven QGroupBox objects under the central window, set the size, position layout and style of the related button components and display them in groups, create two QGraphicsView graphics view objects to display two-dimensional spectral images and volatility response curves, and create a QTableWidget table control object to display table data; multiple control signals in the window are connected to the slot function through the connectSlotsByName function to execute the response signal, and the response function can be triggered by pressing the key.

[0091] Define the retranslateUi function to update the control content on the interface, and use the setTitle method to assign group box titles to the seven QGroupBox objects, which are data reading, hyperspectral preprocessing, volatile preprocessing, multimodal data fusion, data analysis, result saving, and system startup and termination. The setText method sets the text of the two buttons under the upload data group box to hyperspectral data and volatile data to instruct the user to select the corresponding file, sets the text of two or more buttons under the hyperspectral and volatile preprocessing group boxes to the corresponding preprocessing method to instruct the user to select the corresponding processing, sets the text of the three buttons under the multimodal data fusion group box to data level, feature level, and decision level fusion, sets the text of the button under the data analysis group box to start analysis, sets the text of the button under the result saving group box to save results, and sets the text of the two buttons under the system startup and termination group box to start the system and terminate the system.

[0092] A custom upload_check function is used to detect whether the hyperspectral data button under the upload data group box is selected by the user. If it is selected, the open_specfile() function is executed to open a message prompt box to prompt the user to upload the hyperspectral image, and the Data_extract() function is called to segment the hyperspectral image of the seed area and calculate its average spectrum to obtain the hyperspectral dataset matrix; if the volatile data button is selected, the open_enosefile() function is executed to open a message prompt box to prompt the user to upload the volatile response data, which is read as the volatile response data matrix.

[0093] In the upload_check function, the custom open_specfile() function is used to open two file selection dialog boxes, save the selected file path in the instance attributes of the class, and display "Hyperspectral data submission completed" in a message prompt box. Specifically, first use the QFileDialog class to create a file selection dialog box, then use the getOpenFileName method to display the first file selection dialog box. Specifically, the dialog box title is set to "Submit hdr file" to prompt the user to upload a file with a .hdr suffix, and finally the path of the .hdr file selected by the user is stored in the instance attribute one of the class for subsequent calls; then use the getOpenFileName method to display the second file selection dialog box. Specifically, the dialog box title is set to "Submit dat file" to prompt the user to upload a file with a .dat suffix, and finally the path of the .dat file selected by the user is stored in the instance attribute two of the class for subsequent calls; then create an instance of QMessageBox msg_box to create a message box prompt "Data submission completed", and set an "OK" button in the message box. The user clicks the button to close the message box. Finally, use the exec_ method to display the message box and enter the modal loop, so that the user interacts with the message box, that is, the program will execute subsequent code only after clicking the "OK" button, otherwise the program will pause.

[0094] In the upload_check function, the Data_extract() function is customized, and the sepctral module is used to read the .hdr header file stored in the instance attribute 1 of the class and the .dat data file stored in the instance attribute 2 of the class. The wavelength data in the .csv format under the data path is opened, and a specific band is selected for the hyperspectral image to form a single-band image. Specifically, the selection of this specific band is generally based on manually dividing the seed area and the background area and calculating the wavelength corresponding to the maximum reflectance difference between the two, which is the specific band; then the single-band image is thresholded to remove the background and obtain the image of the seed area. Specifically, the manual A binarization threshold is automatically set. Pixels in the single-band image with values ​​greater than the threshold are retained, and pixels less than the threshold are set to 0 to obtain the resulting binarized single-band image. The connected region component is then used to segment the binarized single-band image to identify regions with pixel values ​​greater than 0. Multiple seed regions are contained in one image, thus forming multiple segmented seed region masks. These regions are then grouped and sorted according to their X and Y coordinate values, with the sorting order matching each seed number. This sorting process forms multiple seed region lists. A blank array of the same shape as the original hyperspectral image is created, and the hyperspectral data for the corresponding regions is sequentially extracted into this blank array to form a hyperspectral image for each seed region. Next, image segmentation is performed on each seed region hyperspectral image to obtain a hyperspectral dataset matrix. The Ref_read() function is then called with the file name parameter "seed + seed number," where the seed number is the specific numeric code for the seed. A loop is used to read the whole-grain germplasm reflectance into the data processing path, naming it "seed + seed number.csv."

[0095] A custom spec_processing_check function is used to detect whether the button under the hyperspectral preprocessing group box is selected by the user. If selected, the spec_processing preprocessing operation is executed accordingly to obtain the preprocessed hyperspectrum.

[0096] The spec_processing preprocessing operation mentioned above is the spec_processing function. The function receives hyperspectral data and a specific preprocessing method as input, and then uses a judgment statement to determine and execute the corresponding preprocessing operation. The preprocessing operation can be a single or combined operation such as normalization, multivariate scatter correction, and second-order derivative.

[0097] A custom enose_processing_check function is used to detect whether the button under the volatile preprocessing group box is selected by the user. If selected, the enose_processing preprocessing operation is performed accordingly to obtain the preprocessed volatile data.

[0098] The enose_processing preprocessing operation mentioned above is specifically the enose_processing function as follows: this function receives volatility data and a specific preprocessing method as input, and then uses a judgment statement to determine and execute the corresponding preprocessing operation. The preprocessing operation can be a single or combined operation such as SG smoothing and baseline correction.

[0099] A custom model_check function is used to detect whether the button under the multimodal data fusion group box is selected by the user. If selected, the training model is selected accordingly and the data is passed in for prediction operation to obtain the prediction result. The corresponding execution models are the data-level fusion model, the feature-level fusion model and the decision-level fusion model. The prediction result is the judgment result of whether the seed has vitality, that is, whether the given seed has vitality or not.

[0100] A custom analysis_check function is used to detect whether the button under the analysis data group box is selected by the user. If selected, the current seed number and corresponding prediction results are displayed in the table control of the interface.

[0101] A custom result_check function is used to detect whether the result save button under the area division group box is selected by the user. If selected, the result_save function is executed accordingly.

[0102] The above result_save function obtains the data in the QTableWidget table control window row by row and column by column into a DataFrame, and saves the DataFrame as an Excel file in the corresponding data result path.

[0103] A soybean germplasm resource vitality classification prediction method according to an embodiment of the present invention comprises the following steps:

[0104] The following examples describe a method for classifying and predicting the vitality of soybean germplasm resources according to the present invention, which specifically includes the following steps:

[0105] S1001. Use hyperspectral imaging equipment and olfactory detection equipment to obtain hyperspectral data and volatile data, and obtain vitality labels indicating whether seeds have germinated through standard germination tests to form a multimodal dataset. Preprocess and feature select the hyperspectral data and volatile data in the multimodal dataset respectively to obtain a preprocessed multimodal dataset.

[0106] S1002. Based on the preprocessed multimodal dataset, a training dataset and a test dataset are generated; the training dataset includes a hyperspectral training dataset and a volatility training dataset.

[0107] S1003. Based on the structure and characteristics of the training data set, a data-level fusion germplasm vitality classification detection model is constructed.

[0108] S1004. Based on the structure and high-throughput characteristics of the hyperspectral training dataset and the structure and time series characteristics of the volatile training dataset, a feature-level fusion germplasm vitality classification detection model is constructed.

[0109] S1005. Combining the high-throughput characteristics of the hyperspectral spectrum of S1004 and the time series characteristics of volatility, a decision-level fusion germplasm vitality classification detection model was constructed.

[0110] S1006. Conduct experiments and evaluations on the data-level fusion germplasm vitality classification detection model, the feature-level fusion germplasm vitality classification detection model, and the decision-level fusion germplasm vitality classification detection model. Once the evaluations are passed, use the data-level fusion germplasm vitality classification detection model, the feature-level fusion germplasm vitality classification detection model, or the decision-level fusion germplasm vitality classification detection model to classify the vitality of multiple soybean germplasm resources to be tested.

[0111] The data-level fusion germplasm viability classification detection model in S1003 is explained as follows:

[0112] The classification results of the data-level fusion germplasm viability classification detection model are viable (germination) or non-viable (no germination).

[0113] The data-level fusion germplasm vitality classification detection model consists of two linear layers, a linear rectification function (RELU) and a discard layer; the hyperspectral data X∈R B×C and volatility data Y∈R B×T Splicing to form data-level fusion data F∈R B ×N , where B is the number of batches, C is the number of hyperspectral data bands, T is the time dimension, N = C + T; two linear layers, Gaussian error linear unit (GELU) and dropout layer are used to map F to the subspace of vitality detection, and the vitality detection result SS1∈R is output. B×2 , where 2 represents two labels, specifically including a label corresponding to vitality and a label corresponding to non-vitality.

[0114] The characteristic-level fusion germplasm viability classification detection model in S1004 is explained as follows:

[0115] The feature-level fusion germplasm viability classification detection model includes a volatile feature extraction module, a hyperspectral feature extraction module, a feature fusion module and a linear classification module. The volatile feature extraction module contains five time series convolutional neural networks with a convolution kernel size of 1×1. Each time series convolutional neural network contains a time series convolution layer, a batch normalization layer (BN) and a linear rectifier function (RELU); the hyperspectral feature extraction module contains five one-dimensional convolutional neural networks with a convolution kernel size of 1×1. Each one-dimensional convolutional neural network contains a one-dimensional convolution layer, a batch normalization layer (BN) and a linear rectifier function (RELU); the feature fusion module contains two parts: shallow feature fusion and deep feature fusion. The shallow feature fusion part contains three one-dimensional convolutional neural networks with a convolution kernel size of 1×1. Each one-dimensional convolutional neural network contains a one-dimensional convolution layer, a batch normalization layer (BN) and a linear rectifier function (RELU); four one-dimensional convolutional neural networks with a convolution kernel size of 1×1, two linear rectifier functions (RELU) and two sigmoid activation functions, and the deep feature fusion part contains two layer normalization, a multi-head mixed attention and a feedforward fully connected layer. The hyperspectral data X∈R B×C and volatility data Y∈R B×T Input the hyperspectral feature extraction module and the volatile feature extraction module respectively to obtain two different feature maps: X1∈R B×C and Y1∈R B×T , where X1 is the hyperspectral feature map, Y1 is the volatility feature map, B is the number of batches, C is the number of hyperspectral data bands, and T is the time dimension; X1 and Y1 are input into the shallow part of the feature fusion module to obtain the feature-level shallow fusion feature F1∈R B×M Where M is the feature dimension of F1; F1 inputs the deep part of the feature fusion module to obtain the feature-level deep fusion feature F2∈R B×M ; F2 is input into the linear classification module to obtain the germplasm vitality test result SS2∈R B×2 .

[0116] The decision-level fusion germplasm viability classification detection model in S1005 is explained as follows:

[0117] The decision-level fusion germplasm vitality classification detection model adds two linear classification layers and a decision fusion module based on the feature-level fusion germplasm vitality classification detection model. The decision fusion module contains two convolution layers with a convolution kernel size of 1×1. The two different feature maps obtained by the S4 feature-level fusion model: X1∈R B×C and Y1∈R B×T Input the linear classification layer respectively and get two different classification results SS3∈R B×2 and SS4∈R B×2, where B is the number of batches, C is the number of hyperspectral data bands, and T is the time dimension; the germplasm vitality detection result SS2∈R obtained by the feature-level fusion germplasm vitality classification detection model B×2 , Volatility feature extraction module SS3∈R B×2 The germplasm vitality detection results SS4∈R obtained by the hyperspectral feature extraction module B×2 Input the decision fusion module to obtain the vitality detection result SS of the decision-level fusion germplasm vitality classification detection model e ∈R B×2 .

[0118] Furthermore, in S1001, the hyperspectral data and volatile data in the dataset are preprocessed and feature selected, which specifically includes the following steps:

[0119] S10011. Perform SG filtering on the three-dimensional volatile data sensor by sensor;

[0120] S10012. Selecting a collection time period during which three-dimensional volatile data changes stably;

[0121] S10013. Perform multivariate scattering correction, SG filtering, normalization, and second-order derivative processing on the hyperspectral data channel by channel.

[0122] S10014. Select all characteristic bands of hyperspectral data for modeling.

[0123] Furthermore, the data-level fusion germplasm viability classification detection model in S1003 is described as follows:

[0124] The data-level fusion germplasm viability classification detection model consists of two linear layers, a linear rectification function (RELU) and a dropout layer. The data splicing process is implemented by the following formula:

[0125] F=Concat(X,Y)

[0126] Among them, F is the data-level fusion data.

[0127] The spliced ​​data is input into the classification layer to obtain the vitality detection results of the data-level fusion germplasm vitality classification detection model. The specific implementation process is achieved through the following formula:

[0128] SS1=Linear(Dropout(Linear(RELU(F))))

[0129] Linear(F)=w·F+b

[0130] RELU(F)=max(F,0)

[0131] Among them, w is the weight matrix, b is the bias, and Dropout represents the zeroing operation.

[0132] Furthermore, the feature-level fusion germplasm vitality classification detection model in S1004 includes a volatile feature extraction module, a hyperspectral feature extraction module, a feature fusion module and a linear classification module. The volatility feature extraction module contains five time series convolutional neural networks with a convolution kernel size of 1×1. Each time series convolutional neural network contains a time series convolution layer, a batch normalization layer (BN) and a linear rectifier function (RELU); the hyperspectral feature extraction module contains five one-dimensional convolutional neural networks with a convolution kernel size of 1×1. Each one-dimensional convolutional neural network contains a one-dimensional convolution layer, a batch normalization layer (BN) and a linear rectifier function (RELU); the feature fusion module contains two parts: shallow feature fusion and deep feature fusion. The shallow feature fusion part contains three one-dimensional convolutional neural networks with a convolution kernel size of 1×1. Each one-dimensional convolutional neural network contains a one-dimensional convolution layer, a batch normalization layer (BN) and a linear rectifier function (RELU); four one-dimensional convolutional neural networks with a convolution kernel size of 1×1, two linear rectifier functions (RELU) and two sigmoid activation functions. The deep feature fusion part contains two layer normalization, a multi-head mixed attention and a feedforward fully connected layer. The specific steps are as follows:

[0133] The calculation process of the volatility feature extraction module is specifically implemented through the following formula:

[0134] z i =TCN(BN(RELU(TCN(z i +z i-1 ))),i∈{1,2,3,4,5,6}

[0135]

[0136] RELU(z)=max(z,0)

[0137] Among them, z i is the feature obtained by different layers of time series convolution layers, γ and β are learnable parameter vectors, λ is the numerical stability parameter, and TCN represents the temporal convolutional neural network operation.

[0138] The calculation process of the hyperspectral feature extraction module is specifically implemented through the following formula:

[0139] x i =Conv1d(BN(RELU(Conv1d(x i +x i-1 ))),i∈{1,2,3,4,5,6}

[0140]

[0141] RELU(x)=max(x,0)

[0142] Among them, x i It is the feature obtained by passing through different layers of one-dimensional convolutional layers.

[0143] Among them, the shallow fusion of the two feature maps is used to obtain the shallow fusion output F1 of the hyperspectral and volatile features, which is specifically achieved by the following formula:

[0144]

[0145] X1′, Y1′=Split(f)

[0146] X1″=φ(X1′)×X1′

[0147] Y1″=φ(Y1′)×Y1′

[0148] φ(x)=Sigmoid(Conv1d(RELU(Conv1d(x)))

[0149] F1=X1″+Y1″

[0150] Among them, X′, Y′, x, f, X″ and Y″ are the intermediate output feature maps during the operation process, φ and Represents different calculation processes, Split represents the splitting operation, and Sigmoid represents the sigmoid function (Sigmoid) operation.

[0151] Among them, the two feature maps are deeply fused to obtain the deep fusion output F2 of hyperspectral and volatile features, which is specifically achieved by the following formula:

[0152] X″′1=LayerNorm(X″1)

[0153] Y″′1=LayerNorm(Y″1)

[0154] F′1=FeedForward(LayerNorm(MHCA(X″′1,Y″′1)))

[0155] FeedForward(x)=Linear(RELU(Linear(x)))

[0156] MHCA(x,y)=AdaptiveFusion(Linear(x2),Linear(y2))

[0157] x2=Concat(SA i(x1)),i∈{1,2,…n}

[0158] y2=Concat(SA i (y1)),i∈{1,2,…n}

[0159]

[0160] Q,K,V=Linear(x)

[0161] F2=F1′+F1

[0162] Among them, X″′1, x, Y″′1, x2, y2, F′1 and y are the intermediate output feature maps during the operation process, MHCA represents the multi-head hybrid attention mechanism operation, and n is the number of heads of the multi-head hybrid attention mechanism. Q, K, V are all weight matrices, d k is the dimension of K, SA i is the attention mechanism of the i-th layer, AdaptiveFusion represents the adaptive fusion operation, LayerNorm represents the layer normalization operation, FeedForward represents the feedforward fully connected layer operation, and Softmax represents the normalized exponential function (Softmax) operation.

[0163] Among them, the output F2 of the deep fusion of hyperspectral and volatile features is input into the linear classification layer to obtain the vitality detection result SS2 of the feature-level fusion germplasm vitality classification detection model, which is specifically achieved through the following formula:

[0164] SS2=Linear(F2)

[0165] Furthermore, the decision-level fusion germplasm viability classification detection model constructed by S1005 includes two convolutional layers with a convolution kernel size of 1×1. The specific data processing process is as follows:

[0166] S10051, X1∈R B×C and Y1∈R B×T Input the linear classification layer respectively and get two different classification results SS3∈R B×2 and SS4∈R B×2 , which is specifically achieved through the following formula:

[0167] SS3=Linear(X1)

[0168] SS4=Linear(Y1)

[0169] S10052, the vitality detection results SS2 and SS3 of the feature-level fusion germplasm vitality classification detection model are ∈ R B×2 and SS4∈R B×2Input the decision fusion module to obtain the vitality detection result SS of the decision-level fusion germplasm vitality classification detection model e , which is specifically achieved through the following formula:

[0170] SS′3,SS″3=Conv1d(SS3)

[0171] SS′4,SS″4=Conv1d(SS4)

[0172]

[0173] Among them, SS′3, SS″, SS′4, SS″4, SS 3e and SS 4e is the intermediate output variable.

[0174] Furthermore, experiments and evaluations were conducted on a three-level fusion Transformer network germplasm viability classification detection model for multimodal data; the details are as follows:

[0175] Test experiments were conducted on the data-level fusion germplasm vitality classification detection model, the feature-level fusion germplasm vitality classification detection model and the decision-level fusion germplasm vitality classification detection model to verify the detection performance of the model.

[0176] Specifically, experiments can be conducted on multimodal datasets, using accuracy, F1-score, and the area under the ROC curve (AUC) to comprehensively evaluate the performance of the model's vitality classification detection.

[0177] Experiments were conducted on multimodal datasets using existing commonly used algorithms. Comparative analysis was performed using accuracy, F1-score, and area under the ROC curve (AUC) to provide a basis for comprehensively evaluating the performance of three different multimodal germplasm vitality detection algorithms at the data level, feature level, and decision level. Table 1 shows the dataset used for classification and prediction of soybean germplasm vitality, and Table 2 shows the multimodal multi-level classification prediction results of soybean germplasm vitality.

[0178] Table 1:

[0179] Dataset Vitality No vitality The first soybean variety 87 13 The second soybean variety 9 91 The third soybean variety 79 21

[0180] Table 2:

[0181]

[0182] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the scope of protection of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0183] like Figure 6 As shown, a soybean germplasm resource vitality small sample classification prediction system 200 according to an embodiment of the present invention includes a data acquisition module 201, a data preprocessing module 202, a data training module 203 and a prediction module 204;

[0184] The data acquisition module 201 is used to: acquire hyperspectral data and volatile data of each sample soybean seed in the sample soybean germplasm resources, and acquire the vitality label of each sample soybean seed in the sample soybean germplasm resources to form a multimodal dataset of the sample soybean germplasm resources;

[0185] The data preprocessing module 202 is used to: preprocess the multimodal data set;

[0186] The data training module 203 is used to: train a soybean germplasm small sample vitality detection model based on the preprocessed multimodal data set to obtain a soybean seed vitality detection model;

[0187] The prediction module 204 is used to obtain hyperspectral data and volatile data of the soybean seeds to be tested, and use the soybean seed vitality detection model to perform vitality detection on the soybean seeds to be tested, and obtain vitality detection results of the soybean seeds to be tested.

[0188] Optionally, in the above technical solution, the data preprocessing module 202 is specifically used to: perform multivariate scatter correction processing, SG filtering processing, normalization processing and second-order derivative processing on the hyperspectral data in the multimodal data set channel by channel; and perform SG filtering processing on the volatile data in the multimodal data set.

[0189] Optionally, in the above technical solution, the soybean germplasm small sample vitality detection model includes a feature extraction part, a feature fusion part and a decision fusion part, or the soybean germplasm small sample vitality detection model includes a feature extraction part and a feature fusion part;

[0190] The feature extraction part includes a volatile feature extraction module and a hyperspectral feature extraction module. The volatile feature extraction module includes time series convolution and uses continuous residual connections to obtain information between multiple time dimensions from the preprocessed volatile data to obtain time series data features. The hyperspectral feature extraction module includes a one-dimensional convolutional neural network and uses continuous residual connections to obtain spectral data features from the preprocessed hyperspectral data.

[0191] The feature fusion part includes a shallow feature fusion module and a deep feature fusion module. The shallow feature fusion module performs local weighted fusion of time series data features and spectral data features through one-dimensional convolution and Sigmoid activation function to obtain the first fusion result; the deep feature fusion module processes the first fusion result through a multi-head hybrid self-attention module to obtain the second fusion result;

[0192] In the decision fusion part, one-dimensional convolution is used to map the time series data features and spectral data features into two equal-dimensional classification scores, and the two classification scores are fused with the classification results of the feature fusion module obtained based on the second fusion result to obtain the vitality detection results of the tested soybeans.

[0193] Optionally, in the above technical solution, the data acquisition module 201 is specifically used to: obtain hyperspectral data of sample soybean germplasm resources using hyperspectral equipment, obtain volatile data of sample soybean germplasm resources through olfactory detection equipment, and obtain the vitality label of each sample soybean seed in the sample soybean germplasm resources through a standard germination test, and the vitality label is germination or no germination.

[0194] Optionally, the above technical solution further includes an interactive module, which is used to receive user operation instructions through a visual interactive interface to complete the vitality detection of the soybeans to be tested.

[0195] It should be noted that the beneficial effects of the small sample classification prediction system for soybean germplasm vitality provided by the above embodiment are the same as the beneficial effects of the small sample classification prediction method for soybean germplasm vitality provided by the above embodiment, and will not be repeated here. In addition, when the system provided by the above embodiment realizes its functions, it only uses the division of the above functional modules as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided by the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment, and will not be repeated here.

[0196] Among them, the small sample classification prediction system for soybean germplasm resources vitality of the present invention can be a computer program (including program code) running in a computer device. For example, the small sample classification prediction system for soybean germplasm resources vitality of the present invention is an application software that can be used to execute the corresponding steps in the small sample classification prediction method for soybean germplasm resources vitality of the present invention.

[0197] In some embodiments, the soybean germplasm resource vitality small sample classification prediction system of the present invention can be implemented by a combination of software and hardware. As an example, the soybean germplasm resource vitality small sample classification prediction system of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the soybean germplasm resource vitality small sample classification prediction method of the present invention. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs) or other electronic components.

[0198] The modules described in the embodiments of the present invention may be implemented in software or hardware, and the name of a module does not necessarily limit the module itself.

[0199] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, any of the above-mentioned methods for classifying and predicting the vitality of soybean germplasm resources in a small sample is implemented. That is, an electronic device according to an embodiment of the present invention may include but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the method for classifying and predicting the vitality of soybean germplasm resources in a small sample as shown in any embodiment of the present invention by calling the computer program.

[0200] In an alternative embodiment, an electronic device is provided, such as Figure 7 As shown, Figure 7 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0201] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0202] Bus 4002 may include a path for transmitting information between the above components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 In the figure, only one thick line is used to represent the bus 4002, but this does not mean that there is only one bus or one type of bus.

[0203] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0204] The memory 4003 is used to store application code (computer program) for executing the solution of the present invention, and is controlled by the processor 4001. The processor 4001 is used to execute the application code stored in the memory 4003 to implement the content shown in the above method embodiment.

[0205] Among them, the electronic device can also be a terminal device, and the terminal device can be any device that can install applications, including at least one of a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart TV, and a smart car device.

[0206] It should be noted that Figure 7 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0207] A computer-readable storage medium according to an embodiment of the present invention stores a computer program, which, when executed by a processor, implements any of the above-mentioned methods for classifying and predicting small samples of soybean germplasm resource vitality.

[0208] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0209] In an exemplary embodiment, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform any of the aforementioned methods for classifying and predicting the vitality of soybean germplasm resources using a small sample size.

[0210] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0211] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0212] The computer-readable storage medium provided in the embodiment of the present invention may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EEPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or component.

[0213] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0214] The above description is merely a preferred embodiment of the present invention and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present invention is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the present invention.

[0215] It should be noted that the terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects and to define a specific order or precedence. Where appropriate, the order used for similar objects may be interchanged, such that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0216] Those skilled in the art will appreciate that the present invention may be implemented as a system, method, or computer program product. Therefore, the present invention may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present invention may be implemented in the form of a computer program product embodied in one or more computer-readable media containing computer-readable program code.

[0217] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A small sample classification prediction method for soybean germplasm vitality, characterized in that: include: Obtain hyperspectral data and volatile data of each sample soybean seed in the sample soybean germplasm resources, and obtain the vitality label of each sample soybean seed in the sample soybean germplasm resources to form a multimodal dataset of the sample soybean germplasm resources; Preprocessing the multimodal dataset; Based on the preprocessed multimodal dataset, a soybean germplasm small sample vitality detection model was trained to obtain a soybean seed vitality detection model. Hyperspectral data and volatile data of the soybean seeds to be tested are obtained, and the soybean seed vitality detection model is used to perform vitality detection on the soybean seeds to obtain vitality detection results of the soybean seeds to be tested.

2. A small sample classification prediction method for soybean germplasm vitality according to claim 1, characterized in that: Preprocessing the multimodal dataset includes: The hyperspectral data in the multimodal dataset are subjected to multivariate scatter correction processing, SG filtering processing, normalization processing and second-order derivative processing channel by channel; and the volatile data in the multimodal dataset are subjected to SG filtering processing.

3. A small sample classification prediction method for soybean germplasm vitality according to claim 2, characterized in that: The soybean germplasm small sample vitality detection model includes a feature extraction part, a feature fusion part and a decision fusion part, or the soybean germplasm small sample vitality detection model only includes the feature extraction part and the feature fusion part; The feature extraction part includes a volatile feature extraction module and a hyperspectral feature extraction module. The volatile feature extraction module includes a time series convolution and obtains information between multiple time dimensions from the preprocessed volatile data through continuous residual connections to obtain time series data features. The hyperspectral feature extraction module includes a one-dimensional convolutional neural network and uses continuous residual connections to obtain spectral data features from the preprocessed hyperspectral data. The feature fusion part includes a shallow feature fusion module and a deep feature fusion module. The shallow feature fusion module performs local weighted fusion on the time series data features and the spectral data features through one-dimensional convolution and Sigmoid activation function to obtain a first fusion result; the deep feature fusion module processes the first fusion result through a multi-head hybrid self-attention module to obtain a second fusion result; The decision fusion part uses one-dimensional convolution to map the time series data features and the spectral data features into two equal-dimensional classification scores, and fuses the two classification scores with the classification results of the feature fusion module obtained based on the second fusion result to obtain the vitality detection results of the soybeans to be tested.

4. A small sample classification prediction method for soybean germplasm vitality according to any one of claims 1 to 3, characterized in that: Obtain hyperspectral data and volatile data of sample soybean germplasm resources, and obtain the vitality label of each sample soybean seed in the sample soybean germplasm resources, including: Hyperspectral data of sample soybean germplasm resources were obtained using hyperspectral equipment, volatile data of sample soybean germplasm resources were obtained through olfactory detection equipment, and the vitality label of each sample soybean seed in the sample soybean germplasm resources was obtained through standard germination test. The vitality label was germination or no germination.

5. A small sample classification prediction method for soybean germplasm vitality according to any one of claims 1 to 3, characterized in that: Also includes: Through the visual interactive interface, the user's operation instructions are received to complete the vitality detection of the soybeans to be tested.

6. A small sample classification prediction system for soybean germplasm vitality, characterized in that: It includes data acquisition module, data preprocessing module, data training module and prediction module; The data acquisition module is used to: acquire hyperspectral data and volatile data of each sample soybean seed in the sample soybean germplasm resources, and acquire a vitality label of each sample soybean seed in the sample soybean germplasm resources to form a multimodal dataset of the sample soybean germplasm resources; The data preprocessing module is used to: preprocess the multimodal data set; The data training module is used to: train a soybean germplasm small sample vitality detection model based on the preprocessed multimodal data set to obtain a soybean seed vitality detection model; The prediction module is used to obtain hyperspectral data and volatile data of soybean seeds to be tested, and use the soybean seed vitality detection model to perform vitality detection on the soybean seeds to be tested, so as to obtain vitality detection results of the soybean seeds to be tested.

7. A soybean germplasm resource vitality small sample classification prediction system according to claim 6, characterized in that: The data preprocessing module is specifically used to: perform multivariate scatter correction processing, SG filtering processing, normalization processing and second-order derivative processing on the hyperspectral data in the multimodal data set channel by channel; and perform SG filtering processing on the volatile data in the multimodal data set.

8. A soybean germplasm resource vitality small sample classification prediction system according to claim 7, characterized in that: The soybean germplasm small sample vitality detection model includes a feature extraction part, a feature fusion part and a decision fusion part, or the soybean germplasm small sample vitality detection model includes the feature extraction part and the feature fusion part; The feature extraction part includes a volatile feature extraction module and a hyperspectral feature extraction module. The volatile feature extraction module includes a time series convolution and obtains information between multiple time dimensions from the preprocessed volatile data through continuous residual connections to obtain time series data features. The hyperspectral feature extraction module includes a one-dimensional convolutional neural network and uses continuous residual connections to obtain spectral data features from the preprocessed hyperspectral data. The feature fusion part includes a shallow feature fusion module and a deep feature fusion module. The shallow feature fusion module performs local weighted fusion on the time series data features and the spectral data features through one-dimensional convolution and Sigmoid activation function to obtain a first fusion result; the deep feature fusion module processes the first fusion result through a multi-head hybrid self-attention module to obtain a second fusion result; The decision fusion part uses one-dimensional convolution to map the time series data features and the spectral data features into two equal-dimensional classification scores, and fuses the two classification scores with the classification results of the feature fusion module obtained based on the second fusion result to obtain the vitality detection results of the soybeans to be tested.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the computer program, a small sample classification prediction method for the vitality of soybean germplasm resources as described in any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for classifying and predicting the vitality of soybean germplasm resources in a small sample according to any one of claims 1 to 5 is implemented.

Images