Soybean germplasm resource viability small sample classification prediction method, system and device

Abstract

The application discloses a soybean germplasm resource viability small sample classification prediction method, system and device, relates to the soybean germplasm resource viability classification technical field, and the method comprises the steps of obtaining a multi-modal data set of a sample soybean germplasm resource; the multi-modal data set is pretreated; based on the pretreated multi-modal data set, a soybean germplasm small sample viability detection model is trained, and a soybean seed viability detection model is obtained; the hyperspectral data and volatile data of the soybean seed to be measured are obtained, and the soybean seed viability detection model is used to detect the viability of the soybean seed to be measured, and the viability detection result of the soybean seed to be measured is obtained. The application integrates the hyperspectral data and volatile data of the soybean seed to be measured, comprehensively considers the internal chemical properties of the soybean seed and the volatile substances released externally, so that the viability of the soybean seed can be more comprehensively and accurately evaluated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of soybean germplasm resource viability classification, and particularly relates to a soybean germplasm resource viability small sample classification prediction method, system and device. BACKGROUND

[0002] During the natural aging process of soybean seeds, the viability gradually decreases. Accurate detection of the viability of naturally aged soybean seeds is of great significance for seed quality control, germplasm resource preservation and agricultural production. However, the natural aging of soybeans is slow, and the sample size is extremely limited, which brings difficulties to the construction of a natural aging soybean germplasm viability detection model. The small sample size dataset brings an uneven distribution of data categories, and traditional detection techniques are difficult to fully capture the characteristics of the seeds, resulting in a serious lack of detection ability for a small number of categories, and the detection accuracy is difficult to meet the actual demand. SUMMARY

[0003] The technical problem to be solved by the present application is to overcome the deficiencies of the prior art, and specifically to provide a soybean germplasm resource viability small sample classification prediction method, system and device, as follows:

[0004] 1) In a first aspect, the present application provides a soybean germplasm resource viability small sample classification prediction method, and the specific technical solutions are as follows:

[0005] Obtain the hyperspectral data and volatile data of each sample soybean seed in the sample soybean germplasm resource, and obtain the viability label of each sample soybean seed in the sample soybean germplasm resource, to form a multi-modal dataset of the sample soybean germplasm resource;

[0006] Preprocess the multi-modal dataset;

[0007] Based on the preprocessed multi-modal dataset, train a soybean germplasm small sample viability detection model to obtain a soybean seed viability detection model;

[0008] Obtain the hyperspectral data and volatile data of the to-be-detected soybean seed, and use the soybean seed viability detection model to detect the viability of the to-be-detected soybean seed, to obtain the viability detection result of the to-be-detected soybean seed.

[0009] The soybean germplasm resource viability small sample classification prediction method provided by the present application has the following beneficial effects:

[0010] Soybean seeds release different volatile organic compounds (VOCs) under different viability states, and the types and concentrations of VOCs can provide additional information about the physiological state and viability of soybean seeds. The soybean seed viability detection model can integrate the hyperspectral data and volatile data of the soybean seeds to be tested, comprehensively consider the internal chemical properties of the soybean seeds and the externally released volatile substances, thereby more comprehensively and accurately evaluate the viability of the soybean seeds, and the soybean seed viability detection model with high classification accuracy can be trained by using a small amount of sample data (a small amount of sample hyperspectral data, volatile data and viability label of soybean seeds). The reasons are as follows: ① The hyperspectral data contains rich spectral information. The hyperspectral imaging technology can obtain a large amount of continuous spectral data in the visible light, near-infrared and other waveband ranges of the electromagnetic spectrum. The spectral reflection, absorption and other characteristics of soybean seeds in a specific waveband are closely related to the viability of the seeds. For example, the integrity of the cell structure, the physiological metabolic activity and the like of the seeds can affect the absorption and reflection of light. These spectral characteristics can reflect the viability state of the seeds from the molecular level, thereby providing detailed identification basis for model training. ② The volatile data also has high specificity. Different biochemical processes in the soybean seeds under different viability states can produce different VOCs. These VOCs can be used as chemical fingerprints of the viability of the seeds. For example, the seeds with high viability can have active respiration and metabolism, and can produce specific aldehydes, alcohols, esters and other volatile substances, and the types and contents of these substances can obviously change in the seeds with low viability. These rich characteristic information makes a small amount of samples also contain sufficient patterns for model learning. ③ The viability label is generally obtained by relatively accurate experimental methods, such as tetrazolium staining method, germination experiment and the like. These labels clearly indicate whether the seeds are high, medium or low in viability. When the model is trained, the model can learn the corresponding relationship between the hyperspectral data and volatile data of the samples and the clear viability label. Even if the sample quantity is small, as long as the label is accurate, the model can construct effective classification or prediction rules by mining the features related to the label in the data. ④ The soybean seed viability detection model proposed in the present application has strong feature extraction and pattern recognition capability, and can continuously adjust the parameters through the back propagation algorithm to minimize the error between the predicted value and the true viability label. Even if there are only a small amount of samples, as long as the data features of the samples can be effectively extracted and utilized by the model, the model can learn the key discriminant features of the seed viability, thereby constructing a high-quality soybean seed viability detection model. ⑤ If a small amount of samples can well represent the main types of soybean seeds under different viability states, the trained model can have high quality.For example, by reasonably designing the experimental scheme, seeds with different vitality levels (such as high, medium and low) are selected as samples, and these samples have certain diversity in variety, origin and other factors, so that these samples can cover the key feature space of soybean seed vitality detection. After the model is trained on these representative samples, it can be generalized to a wider range of seed samples to a certain extent.

[0011] On the basis of the above scheme, the soybean germplasm resource vitality small sample classification prediction method of the application can be further improved as follows.

[0012] Further, the multi-modal data set is preprocessed, including:

[0013] The hyperspectral data in the multi-modal data set is processed channel by channel for multivariate scattering correction, SG filtering, standardization and second derivative processing; and the volatile data in the multi-modal data set is processed for SG filtering.

[0014] The beneficial effects of the above further scheme are: it can further improve the accuracy of the vitality detection results of the soybean seeds to be tested.

[0015] Further, the soybean germplasm small sample vitality detection model comprises a feature extraction part, a feature fusion part and a decision fusion part, or the soybean germplasm small sample vitality detection model comprises a feature extraction part and a feature fusion part.

[0016] The feature extraction part includes a volatile feature extraction module and a hyperspectral feature extraction module. The volatile feature extraction module includes a time series convolution and obtains information between multiple different time dimensions from the preprocessed volatile data through continuous residual connection to obtain time series data features. The hyperspectral feature extraction module includes a one-dimensional convolutional neural network and uses continuous residual connection to obtain spectral data features from the preprocessed hyperspectral data.

[0017] The feature fusion part includes a shallow feature fusion module and a deep feature fusion module. The shallow feature fusion module performs local weighted fusion on the time series data features and the spectral data features through one-dimensional convolution and Sigmoid activation function to obtain a first fusion result. The deep feature fusion module processes the first fusion result through a multi-head mixed self-attention module to obtain a second fusion result.

[0018] The decision fusion part uses one-dimensional convolution to map the time series data features and the spectral data features into two classification scores with equal dimensions, respectively, and fuses the two classification scores with the feature fusion module classification result based on the second fusion result to obtain the vitality detection result of the soybean to be tested.

[0019] Further, the hyperspectral data and volatile data of the sample soybean germplasm resources are acquired, and a viability label of each sample soybean seed in the sample soybean germplasm resources is acquired, including:

[0020] The hyperspectral data of the sample soybean germplasm resources are acquired by using a hyperspectral device, the volatile data of the sample soybean germplasm resources are acquired by using an olfactory detection device, and the viability label of each sample soybean seed in the sample soybean germplasm resources is obtained by a standard germination test, and the viability label is germination or no germination.

[0021] Further, the method further comprises:

[0022] Through the visual interactive interface, the operation instruction of the user is received, so as to complete the viability detection of the soybean to be detected.

[0023] The beneficial effects of the above further scheme are that the operation instruction of the user is received through the visual interactive interface, the operation is more intuitive and convenient, the efficiency is improved, the learning cost of the user is reduced, and the user experience is improved.

[0024] 2) In a second aspect, the present application also provides a soybean germplasm resource viability small sample classification and prediction system, and the specific technical scheme is as follows:

[0025] The system comprises a data acquisition module, a data preprocessing module, a data training module and a prediction module.

[0026] The data acquisition module is used for acquiring the hyperspectral data and volatile data of each sample soybean seed in the sample soybean germplasm resources, and acquiring the viability label of each sample soybean seed in the sample soybean germplasm resources, to form a multi-modal data set of the sample soybean germplasm resources.

[0027] The data preprocessing module is used for preprocessing the multi-modal data set.

[0028] The data training module is used for training a soybean germplasm small sample viability detection model based on the preprocessed multi-modal data set, to obtain a soybean seed viability detection model.

[0029] The prediction module is used for acquiring the hyperspectral data and volatile data of the soybean seed to be detected, and using the soybean seed viability detection model to detect the viability of the soybean seed to be detected, to obtain the viability detection result of the soybean seed to be detected.

[0030] On the basis of the above scheme, the soybean germplasm resource viability small sample classification and prediction system of the present application can also be improved as follows.

[0031] Further, the data preprocessing module is specifically configured to: perform multivariate scattering correction processing, SG filtering processing, standardization processing and second derivative processing on the hyperspectral data in the multi-modal data set channel by channel; and perform SG filtering processing on the volatile data in the multi-modal data set.

[0032] Further, the soybean germplasm small sample viability detection model comprises a feature extraction part, a feature fusion part and a decision fusion part, or the soybean germplasm small sample viability detection model comprises a feature extraction part and a feature fusion part.

[0033] The feature extraction part comprises a volatile feature extraction module and a hyperspectral feature extraction module, the volatile feature extraction module comprises a time series convolution, and information between multiple different time dimensions is obtained from the preprocessed volatile data through continuous residual connection to obtain time series data features; and the hyperspectral feature extraction module comprises a one-dimensional convolutional neural network, and spectral data features are obtained from the preprocessed hyperspectral data through continuous residual connection.

[0034] The feature fusion part comprises a shallow feature fusion module and a deep feature fusion module, the shallow feature fusion module performs local weighted fusion on the time series data features and the spectral data features through one-dimensional convolution and a Sigmoid activation function to obtain a first fusion result; and the deep feature fusion module processes the first fusion result through a multi-head mixed self-attention module to obtain a second fusion result.

[0035] The decision fusion part uses one-dimensional convolution to respectively map the time series data features and the spectral data features into two classification scores with equal dimensions, and fuses the two classification scores with a feature fusion module classification result obtained based on the second fusion result to obtain a viability detection result of the sample soybean.

[0036] Further, the data acquisition module is specifically configured to: acquire the hyperspectral data of the sample soybean germplasm resources by using a hyperspectral device, acquire the volatile data of the sample soybean germplasm resources by using an olfactory detection device, and obtain a viability label of each sample soybean seed in the sample soybean germplasm resources through a standard germination test, wherein the viability label is germination or no germination.

[0037] Further, the method further comprises an interaction module, and the interaction module is configured to: receive an operation instruction of a user through a visual interactive interface to complete the viability detection of the sample soybean.

[0038] 3) In a third aspect, the present application also provides an electronic device, which comprises a processor and a memory, the memory stores at least one computer program, the at least one computer program is loaded and executed by the processor, so that the electronic device realizes any one of the soybean germplasm resource viability small sample classification prediction methods.

[0039] 4) In a fourth aspect, the present application also provides a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement any one of the soybean germplasm vitality small sample classification and prediction methods.

[0040] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present application and the corresponding possible implementation manners can be referred to the technical effects of the first aspect and its corresponding possible implementation manners described above, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced as follows:

[0042] Figure 1 A flowchart of a soybean germplasm vitality small sample classification and prediction method according to an embodiment of the present application;

[0043] Figure 2 A structural diagram of a soybean germplasm small sample vitality detection model;

[0044] Figure 3 A structural diagram of a soybean germplasm small sample vitality detection model;

[0045] Figure 4 A principle block diagram of a soybean germplasm small sample vitality detection data fusion model;

[0046] Figure 5 A software interface diagram of a Transformer soybean germplasm small sample vitality detection software;

[0047] Figure 6 A structural diagram of a soybean germplasm resource vitality small sample classification and prediction system according to an embodiment of the present application;

[0048] Figure 7 A structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] The principles and features of the present application are described below, and the examples are only used to explain the present application, and are not used to limit the scope of the present application.

[0050] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below. The following several specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0051] AsFigure 1 As shown, the soybean germplasm vitality small sample classification prediction method of the embodiment of the application comprises the following steps:

[0052] S1, obtain the hyperspectral data and volatile data of each sample soybean seed in the sample soybean germplasm resource, and obtain the vitality label of each sample soybean seed in the sample soybean germplasm resource, to form a multi-modal data set of the sample soybean germplasm resource;

[0053] Among them, the hyperspectral data of the sample soybean germplasm resource is obtained by using a hyperspectral device, the volatile data of the sample soybean germplasm resource is obtained by using an olfactory detection device, and the vitality label of each sample soybean seed in the sample soybean germplasm resource is obtained by using a standard germination test. The vitality label is germination or no germination.

[0054] Among them, based on the small sample condition of soybean, intact and undamaged seeds are selected, and the test process is ensured to be error-free to ensure the effectiveness of the sample.

[0055] S2, pre-process the multi-modal data set, specifically:

[0056] The hyperspectral data in the multi-modal data set is processed by multivariate scatter correction, SG filtering, standardization and second derivative processing channel by channel. The volatile data in the multi-modal data set is processed by SG filtering, which can further improve the accuracy of the vitality detection result of the soybean seed to be tested. The specific content is as follows:

[0057] Select a stable change of three-dimensional volatile data collection time period for analysis to reduce data interference caused by environmental factors and more accurately reflect the true situation of the measured gas, and then perform SG filtering processing on the three-dimensional volatile data sensor by sensor.

[0058] For the problems that may exist in the hyperspectral acquisition process: the size and shape of the seed sample cause spectral scattering, instrument noise, scale difference of spectral reflectance between different seed samples, and overlapping peaks and weak peak identification problems of near-infrared spectrum, multivariate scatter correction and SG filtering are performed on the hyperspectral data channel by channel, and standardization and second derivative processing are performed. The hyperspectral pre-processing process of soybean seeds is more targeted and flexible.

[0059] S3, based on the pre-processed multi-modal data set, train the soybean germplasm small sample vitality detection model to obtain a soybean seed vitality detection model, specifically:

[0060] The pre-processed multi-modal data set is divided into a training set and a test set, and the soybean germplasm small sample vitality detection model is trained based on the training set and the test set.

[0061] The soybean germplasm small sample vitality detection model can also be referred to as a feature decision fusion multi-modal Transformer soybean germplasm small sample vitality detection algorithm.

[0062] S4, obtaining hyperspectral data and volatile data of the soybean seeds to be tested, and using the soybean seed vitality detection model to detect the vitality of the soybean seeds to be tested, to obtain the vitality detection result of the soybean seeds to be tested.

[0063] Soybean seeds release different volatile organic compounds under different vitality states, and the types and concentrations of volatile organic compounds can provide additional information about the physiological state and vitality of soybean seeds. The soybean seed vitality detection model can integrate the hyperspectral data and volatile data of the soybean seeds to be tested, and comprehensively consider the internal chemical properties of the soybean seeds and the volatile substances released externally, so as to more comprehensively and accurately evaluate the vitality of the soybean seeds.

[0064] Optionally, in the above technical solution, as shown in Figure 2 and Figure 3 The soybean germplasm small sample vitality detection model comprises a feature extraction part, a feature fusion part and a decision fusion part, or the soybean germplasm small sample vitality detection model comprises a feature extraction part and a feature fusion part.

[0065] The feature extraction part comprises a volatile feature extraction module and a hyperspectral feature extraction module. The volatile feature extraction module comprises a plurality of time series convolutions, and is connected by continuous residual connections. Specifically, the output of the first time series convolution is respectively used as the input of the second time series convolution and the input of the last time series convolution, the output of the second time series convolution is respectively used as the input of the third time series convolution and the input of the penultimate time series convolution, and so on, to realize the continuous residual connection of the plurality of time series convolutions, to obtain time series data features from the preprocessed volatile data; the hyperspectral feature extraction module comprises a plurality of one-dimensional convolutional neural networks, and is connected by continuous residual connections. Specifically, the output of the first one-dimensional convolutional neural network is respectively used as the input of the second one-dimensional convolutional neural network and the input of the last one-dimensional convolutional neural network, the output of the second one-dimensional convolutional neural network is respectively used as the input of the third one-dimensional convolutional neural network and the input of the penultimate one-dimensional convolutional neural network, and so on, to realize the continuous residual connection of the plurality of one-dimensional convolutional neural networks, to obtain spectral data features from the preprocessed hyperspectral data.

[0066] The feature fusion part comprises a shallow feature fusion module and a deep feature fusion module. The shallow feature fusion module performs local weighted fusion on the time series data features and the spectral data features through one-dimensional convolution and a Sigmoid activation function to obtain a first fusion result. Specifically, the shallow feature fusion module processes the time series data features and the spectral data features through one one-dimensional convolution respectively, splices the outputs of the two one-dimensional convolutions, and then inputs the spliced output into another one-dimensional convolution. The one-dimensional convolution is split to obtain data corresponding to the time series data features and data corresponding to the spectral data features. The two data are respectively processed through one-dimensional convolution and a Sigmoid activation function, and are multiplied element by element with the corresponding time series data features or spectral data features processed through one-dimensional convolution to obtain two output results. The two output results are adaptively fused to obtain an adaptive fusion output graph. The first fusion result comprises the two output results and the adaptive fusion output graph.

[0067] The deep feature fusion module processes the first fusion result through a multi-head mixed self-attention module to obtain a second fusion result. Specifically, the two output results in the first fusion result are respectively processed through layer normalization, and then are input into the multi-head mixed self-attention module (MHCA). The output of the multi-head mixed self-attention module is processed through a feedforward fully connected layer, and is added element by element with the adaptive fusion output graph to obtain the output of the deep feature fusion module. The output of the deep feature fusion module is processed through a classification layer (linear classification layer) to obtain a classification result.

[0068] The decision fusion part uses one-dimensional convolution to respectively map the time series data features and the spectral data features into two classification scores with equal dimensions. The two classification scores are fused with the classification result of the feature fusion module based on the second fusion result to obtain the viability detection result of the soybean to be tested.

[0069] The soybean germplasm small sample viability detection model comprises a feature extraction part and a feature fusion part, or the soybean germplasm small sample viability detection model comprises a feature extraction part, a feature fusion part and a decision fusion part.

[0070] When the soybean germplasm small sample viability detection model only contains the feature extraction part and the feature fusion part, the classification result of the feature fusion module obtained according to the second fusion result is directly used as the viability detection result of the soybean to be detected. Specifically, the hyperspectral data and volatile data of the soybean seed to be detected are input into the feature extraction part for feature extraction, and spectral data features and time series data features are obtained respectively; then the spectral data features and the time series data features are input into the shallow feature fusion module of the feature fusion part to obtain the first fusion result, and the deep feature fusion module is used for fusion processing to obtain the deep fusion features of the feature level, and the output of the deep feature fusion module is obtained through the multi-head mixed self-attention module and the full connection layer processing, and finally input into the linear classification module to obtain the classification result of the feature fusion module as the viability detection result of the soybean to be detected.

[0071] When the soybean germplasm small sample viability detection model contains the feature extraction part, the feature fusion part and the decision fusion part, the classification score calculated by the decision fusion part is fused with the first fusion result to obtain the classification result of the decision fusion module as the viability detection result of the soybean to be detected. Specifically, the time series data features and the spectral data features are calculated by using the feature extraction module to calculate the hyperspectral data and volatile data of the soybean seed to be detected, and then the feature fusion module classification result is calculated by using the feature fusion part to calculate the time series data features and the spectral data features, and the time series data feature classification result and the spectral data feature classification result are calculated by using the linear classification layer to calculate the time series data features and the spectral data features, and finally the fusion result of the feature fusion module classification result, the time series data feature classification result and the spectral data feature classification result is calculated by using the decision fusion module to obtain the decision fusion module classification result as the viability detection result of the soybean to be detected.

[0072] The feature fusion part and the decision fusion part are connected with the feature extraction part to realize rapid feature extraction of a small amount of samples; the feature extraction part can be connected with the feature fusion part or the decision fusion part, and the running parameters and processing procedures of each part can be adjusted according to the type and characteristics of the input small sample data, that is, the multi-level fusion process of data from input to output can be realized, and the processing efficiency is greatly improved.

[0073] The soybean germplasm small sample life detection model deeply fuses feature extraction and decision fusion functions in view of the overfitting problem of a classification model caused by limited quantity of small sample data and the fact that seed characteristics and rules cannot be comprehensively extracted; the model training process is as follows: the data set is divided into a training data set and a test data set, the training data set of two modes of input spectrum and volatility is fused, and then the fused data set is trained, and the final decision model is obtained by adjusting the weight and learning rate in real time according to the loss function and other performance indicators of each model on the training data set during the training process; the model prediction process is as follows: the small sample soybean seed spectrum and volatility test set data to be predicted are received, the trained decision model is loaded to obtain the life detection result through the feature extraction and decision fusion steps. The feature extraction function uses a mixed feature extraction strategy to obtain cross-modal joint features, and fully utilizes the correlation and complementary information of the two modal data of the small sample soybean seeds; the decision fusion function effectively integrates the life detection results from the volatility feature extraction, hyperspectral feature extraction module and feature fusion part to improve the reliability of the detection result.

[0074] The soybean germplasm small sample life detection model proposed in the application comprises feature extraction, feature fusion and decision fusion, the algorithm integrates the spectrum and volatility data processing flow, and is an integrated analysis system with flexibility and high efficiency; users can select three modules or select the first two modules or select the last two modules to form different fusion models, that is, select three modules to combine decision-level fusion, select the first two modules to combine feature-level fusion, and select the last two modules to combine data-level fusion. The life information contained in the hyperspectral image and the volatility signal can be fully utilized, the systematic analysis and integration of deep and shallow features of each mode are realized, and reasonable fusion of cross-modal deep and shallow features is realized.

[0075] The feature extraction part comprises a volatility feature extraction module and a hyperspectral feature extraction module, which are used to extract time series data features and spectrum data features. The volatility feature extraction module is composed of time series convolution, and information between multiple different time dimensions of volatility is obtained through continuous residual connection to enhance the volatility feature extraction capability of the model; the hyperspectral feature extraction module is composed of a one-dimensional convolutional neural network, and continuous residual connection is used to overcome the limitation of the one-dimensional convolutional neural network receptive field. The organic combination of the two can fully obtain more life discriminative features from a small number of samples, and lay a foundation for soybean germplasm small sample life detection.

[0076] The feature fusion section comprises two parts: shallow feature fusion and deep feature fusion. The shallow feature fusion module uses one-dimensional convolution and a sigmoid activation function to locally weight and fuse the output features from the volatile feature extraction and hyperspectral feature extraction modules. The deep feature fusion module employs an interactive fusion method of volatile and hyperspectral features, centered on a multi-head hybrid self-attention module. Compared to later fusion, the deep feature fusion module performs extraction and fusion simultaneously, fully integrating volatile and hyperspectral features and avoiding feature loss caused by later fusion. This shallow-deep fusion approach effectively captures the differential spectral information between viable and non-viable samples, laying the foundation for viability testing of small soybean germplasm samples.

[0077] In the decision fusion section, two linear classification layers and a decision fusion module are added to the feature-level fusion germplasm viability classification and detection model. First, one-dimensional convolution is used to map the classification results from the volatile feature extraction module and the hyperspectral feature extraction module to two classification scores of equal dimensions. Second, the two sets of mapped scores are fused with the classification results from the feature fusion module to obtain the fused output classification score. The decision fusion section combines, coordinates, and integrates the viability detection results from the volatile feature extraction, hyperspectral feature extraction, and feature fusion modules to obtain more comprehensive, accurate, and reliable soybean germplasm viability decision detection results.

[0078] Optionally, the above technical solution also includes:

[0079] like Figure 5 As shown, the system receives user commands through a visual interactive interface to complete the viability test of the soybeans to be tested. Specifically:

[0080] Before starting the test, clicking "Start System" in the system startup and shutdown window will display the various windows on this interface. The "Upload Data" window is used to upload hyperspectral data. Following the prompts, select the .hdr and .dat files in sequence. Successful selection will display the corresponding hyperspectral curve in the middle graph window one. Clicking "Volatile Data" will upload volatile data in .nos format. Successful selection will display the volatile response curve in the middle graph window two. Next, selecting one option in the hyperspectral preprocessing window will preprocess the imported hyperspectral data accordingly. Selecting one option in the volatile preprocessing window will preprocess the imported volatile data accordingly. In the multimodal data fusion window, selecting one of the three levels—data level, feature level, or decision level—will call the corresponding trained model to predict the viability of the preprocessed data, obtaining the prediction result of whether or not viability exists. Specifically, when selecting the data level, the model will call... Figure 4 The soybean germplasm small sample viability detection data fusion model shown is used for viability prediction. When the feature level is selected, the following is called:Figure 2 The soybean germplasm small sample vitality detection model shown in the figure is used for vitality prediction. When a decision level is selected, the vitality prediction result is transmitted into the middle table window to display the seed serial number and the vitality prediction result. Clicking the result saving under the result saving window saves the result data in the table into the corresponding path in an excel file. After the test is completed, clicking the terminate system in the system start and terminate window closes the windows of the interface. Figure 3 The soybean germplasm small sample vitality detection model shown in the figure is used for vitality prediction. When a decision level is selected, the vitality prediction result is transmitted into the middle table window to display the seed serial number and the vitality prediction result. Clicking the result saving under the result saving window saves the result data in the table into the corresponding path in an excel file. After the test is completed, clicking the terminate system in the system start and terminate window closes the windows of the interface.

[0081] The hyperspectral data and volatile format are not limited to the above, and can be read by corresponding data reading code.

[0082] The production process of the user interface is as follows: defining a UI_windows class to encapsulate the settings and layout related to the user interface, creating seven QGroupBox objects under the central window to set the size, position layout and style of the related button components and display them in groups, creating two QGraphicsView graphic view objects to set the size and position layout for displaying two-dimensional spectral images and volatile response curves, and creating a QTableWidget table control object to set the size and position layout for displaying table data; the signal connection of multiple controls in the window is connected to the slot function through the connectSlotsByName function to execute the response signal, and the response function can be triggered by pressing the key.

[0083] The retranslateUi function is defined to give the above user interface components related titles and text contents. The setTitle method is used to give the seven QGroupBox objects group box titles, respectively, and the titles are data reading, hyperspectral preprocessing, volatile preprocessing, multi-modal data fusion, data analysis, result saving and system start and termination. The setText method is used to set the text of the two buttons under the data uploading group box to hyperspectral data and volatile data to indicate the user to select the corresponding file, set the text of the two or more buttons under the hyperspectral and volatile preprocessing group box to the corresponding preprocessing method to indicate the user to select the corresponding processing, set the text of the three buttons under the multi-modal data fusion group box to data level, feature level and decision level fusion, set the text of the button under the data analysis group box to start analysis, set the text of the button under the result saving group box to save result, and set the text of the two buttons under the system start and termination group box to start system and terminate system.

[0084] The application is more specifically described through the following embodiments:

[0085] The soybean seed high spectrum data and volatile phase response data are collected by using a hyperspectral equipment and an olfactory detection equipment, a life force label of whether the seed germinates is obtained through a standard germination test, the collected high spectrum data and volatile phase response data are preprocessed to obtain preprocessed high spectrum data and volatile phase response data; then, a data set is made, that is, the preprocessed multi-modal data set is divided into a training set and a test set at a certain ratio, an initial learning rate and a maximum training round are given, when the current training round is less than the given maximum training round, the training data set is input into the soybean germplasm small sample life force detection model, the loss value is calculated in each model training to update the training model weight and the learning rate, and iteration is entered into the next training round until the maximum training round is reached, the model outputs a weight file, then the model and the weight file are called in the test data set, the life force of the test data set is detected, and a soybean germplasm resource life force detection result, that is, a discrimination result of whether the life force exists or not, is obtained.

[0086] The soybean germplasm small sample life force detection model can be divided into a data set model, a feature level model and a decision level model, and can be selected according to specific needs.

[0087] The operation process of the software program used by the system of the application is as follows: first, click to start the system in the system start and stop window before the test starts, and then the windows of the interface are displayed. The high spectrum data is uploaded in the data upload window, the.hdr file and the.dat file are selected according to the prompt box, and the corresponding high spectrum curve is displayed in the middle graphic window after successful selection. The volatile data in the.nos format is uploaded, and the volatile response curve is displayed in the middle graphic window two after successful selection. Then, one of the high spectrum preprocessing windows is selected, and the input high spectrum data is preprocessed accordingly. One of the volatile preprocessing windows is selected, and the input volatile data is preprocessed accordingly. One of the three data levels, feature levels and decision levels in the multi-modal data fusion window is selected, and the corresponding trained model is called to predict the life force of the input preprocessed data, and the prediction result of whether the life force exists or not is obtained. Click to start analysis in the data analysis window, and then the life force prediction result is input, and the seed serial number and the life force prediction result are displayed in the middle table window. Click to save the result in the result saving window, and then the result data in the table is saved in the corresponding path in the form of an excel file. After the test is completed, click to terminate the system in the system start and stop window, and then the windows of the interface are closed.

[0088] The high spectrum data and volatile format are not limited to the above, and data reading code can be written accordingly for reading;

[0089] The construction of the feature decision fusion multi-modal Transformer soybean germplasm small sample life force detection software of the application is as follows:

[0090] Define UI_windows class to encapsulate user interface related settings and layout, create seven QGroupBox objects under the central window to set size, position, layout and style of related button components and display them in groups, create two QGraphicView objects to display two-dimensional spectral image and volatile response curve, create a QTableWidget object to display table data; connect multiple control signals to slot functions through the connectSlotsByName function to execute response signals when the keys are pressed.

[0091] Define retranslateUi function to update the content of the controls on the interface, use setTitle method to assign titles to the seven QGroupBox objects, the titles are data reading, hyperspectral preprocessing, volatile preprocessing, multi-modal data fusion, data analysis, result saving and system startup and termination, use setText method to set the text of the two buttons under the data upload group box to hyperspectral data and volatile data to indicate the user to select the corresponding file, set the text of two or more buttons under the hyperspectral and volatile preprocessing group box to the corresponding preprocessing method to indicate the user to select the corresponding processing, set the text of the three buttons under the multi-modal data fusion group box to data level, feature level and decision level fusion, set the text of the button under the data analysis group box to start analysis, set the text of the button under the result saving group box to save results, set the text of the two buttons under the system startup and termination group box to start system and terminate system.

[0092] Define upload_check function to detect whether the hyperspectral data button under the data upload group box is selected by the user, if selected, execute open_specfile() function to open a message prompt box to prompt the user to upload the hyperspectral image, and call Data_extract() function to segment the hyperspectral image of the seed region, calculate the average spectrum and get the hyperspectral data set matrix; if the volatile data button is selected, execute open_enosefile() function to open a message prompt box to prompt the user to upload the volatile response data, and read it as a volatile response data matrix.

[0093] In the upload_check function, the custom open_specfile() function is used to open two file selection dialog boxes, save the selected file paths in the instance properties of the class, and display a message box with the message "Hyperspectral data submission complete". Specifically, first, a file selection dialog box is created using the QFileDialog class. Then, the getOpenFileName method is used to display the first file selection dialog box. Specifically, the dialog box title is set to "Submit hdr file" to prompt the user to upload a file with the.hdr suffix. Finally, the path of the user-selected.hdr file is stored in the first instance property of the class for subsequent calls. Then, the getOpenFileName method is used to display the second file selection dialog box. Specifically, the dialog box title is set to "Submit dat file" to prompt the user to upload a file with the.dat suffix. Finally, the path of the user-selected.dat file is stored in the second instance property of the class for subsequent calls. Then, an instance of QMessageBox named msg_box is created to create a message box with the message "Data submission complete". A "Confirm" button is set in the message box. When the user clicks the button, the message box will be closed. Finally, the exec_ method is used to display the message box and enter a modal loop, so that the user interacts with the message box. That is, after clicking the "Confirm" button, the program will execute the subsequent code. Otherwise, the program will pause.

[0094] In the upload_check function, the Data_extract() function is defined to read the.hdr header file stored in the instance attribute one of the class and the.dat data file stored in the instance attribute two of the class by using the sepctral module, open the wavelength data in the.csv format under the data path, select a specific band for the hyperspectral image to form a single-band image, and specifically, the specific band is generally selected according to the manual division of the seed region and the background region and the calculation of the wavelength corresponding to the maximum reflectivity difference between the two regions, that is, the specific band; then, the single-band image is subjected to threshold segmentation to remove the background and obtain the image of the seed region, specifically, the binary threshold is manually set, the pixel points with a pixel value greater than the threshold in the single-band image are retained, the pixel points with a pixel value less than the threshold are set to 0 to obtain the binary single-band image, the connected region component is used to segment the pixel point region with a pixel value greater than 0 in the binary single-band image, and one image contains multiple seed regions, so multiple segmented seed region masks are formed, and the X and Y axis coordinate values are grouped and sorted, the sorting sequence number is matched with each seed number, multiple seed region lists are formed after sorting, an empty array with the same shape as the original hyperspectral image is created, and the hyperspectral data of the corresponding region is extracted into the empty array to form a hyperspectral image of each seed region. The hyperspectral data set matrix is obtained by performing image segmentation on each seed region hyperspectral image, and the Ref_read() function is called to give the file name parameter as “seed+seed number”, wherein the seed number is the specific numerical coding of the corresponding seed; the whole grain germ reflectivity is read in the data processing path by using the loop statement, and is named as: seed+seed number.csv.

[0095] The spec_processing_check function is defined to detect whether the button under the hyperspectral preprocessing grouping box is selected by the user, and if the button is selected, the spec_processing preprocessing operation is performed to obtain the preprocessed hyperspectral.

[0096] The spec_processing preprocessing operation is the spec_processing function, which specifically receives the hyperspectral data and the specific preprocessing method as input, and then determines and performs the corresponding preprocessing operation by using the judgment statement, and the preprocessing operation can be a single or combined operation of normalization, multivariate scatter correction, second derivative, etc.

[0097] The enose_processing_check function is defined to detect whether the button under the volatile preprocessing grouping box is selected by the user, and if the button is selected, the enose_processing preprocessing operation is performed to obtain the preprocessed volatile data.

[0098] The enose_processing preprocessing operation is specifically as follows: the function receives volatile data and a specific preprocessing method as input, then determines and executes the corresponding preprocessing operation by using a judgment statement, and the preprocessing operation can be a single or combined operation such as SG smoothing and baseline correction.

[0099] The model_check function is customized to detect whether the button under the multi-modal data fusion grouping box is selected by the user, and if selected, the corresponding training model is selected and data is transmitted for prediction operation to obtain a prediction result, and the corresponding executed model is a data-level fusion model, a feature-level fusion model and a decision-level fusion model, and the prediction result is a viability determination result, that is, the given seed is viable or non-viable.

[0100] The analysis_check function is customized to detect whether the button under the analysis data grouping box is selected by the user, and if selected, the current seed number and the corresponding prediction result are displayed in the table control of the interface.

[0101] The result_check function is customized to detect whether the result saving button under the region division grouping box is selected by the user, and if selected, the result_save function is executed.

[0102] The result_save function obtains data in the QTableWidget table control window row by row and column by column to a DataFrame, and saves the DataFrame as an excel file under the corresponding data result path.

[0103] The soybean germplasm resource viability classification prediction method of the embodiment of the application comprises the following steps:

[0104] The soybean germplasm resource viability classification prediction method of the application is described in detail through the following embodiments, and specifically comprises the following steps:

[0105] S1001, high spectral imaging equipment and olfactory detection equipment are used to obtain hyperspectral data and volatile data, and a standard germination test is used to obtain a viability label of whether the seed germinates, to form a multi-modal data set, and the hyperspectral data and volatile data in the multi-modal data set are preprocessed and feature selected to obtain a preprocessed multi-modal data set.

[0106] S1002, based on the preprocessed multi-modal data set, a training data set and a test data set are made; the training data set includes a hyperspectral training data set and a volatile training data set.

[0107] S1003. Construct a data-level fusion germplasm viability classification detection model based on the structure and features of the training data set.

[0108] S1004. Construct a feature-level fusion germplasm viability classification detection model based on the structure of the hyperspectral training data set and the high-throughput features and the structure of the volatile training data set and the time series features.

[0109] S1005. Construct a decision-level fusion germplasm viability classification detection model by combining the hyperspectral high-throughput features and the time series features of volatiles of S1004.

[0110] S1006. Experiment and evaluate the data-level fusion germplasm viability classification detection model, the feature-level fusion germplasm viability classification detection model, and the decision-level fusion germplasm viability classification detection model. When the evaluation is passed, use the data-level fusion germplasm viability classification detection model, the feature-level fusion germplasm viability classification detection model, or the decision-level fusion germplasm viability classification detection model to classify the viability of multiple soybean germplasm resources to be tested.

[0111] The data-level fusion germplasm viability classification detection model in S1003 is described as follows:

[0112] The classification result of the data-level fusion germplasm viability classification detection model is either viable (germination) or non-viable (no germination).

[0113] The data-level fusion germplasm viability classification detection model consists of two linear layers, a linear rectifier function (RELU), and a dropout layer; the hyperspectral data X ∈ R B×C and volatile data Y ∈ R B×T are spliced to form data-level fusion data F ∈ R B ×N , where B is the number of batches, C is the number of hyperspectral data bands, T is the time dimension, and N = C + T; F is mapped to the subspace of viability detection using two linear layers, Gaussian error linear units (GELU), and a dropout layer, and the viability detection result SS1 ∈ R B×2 , where 2 indicates two labels, specifically including a viable label and a non-viable label.

[0114] The feature-level fusion germplasm viability classification detection model in S1004 is described as follows:

[0115] The characteristic level fusion germplasm vitality classification detection model comprises a volatile feature extraction module, a hyperspectral feature extraction module, a feature fusion module and a linear classification module. The volatile feature extraction module comprises five time series convolutional neural networks with a convolution kernel size of 1*1, each time series convolutional neural network comprising a time series convolution layer, a batch normalization layer (BN) and a linear rectifier function (RELU); the hyperspectral feature extraction module comprises five one-dimensional convolutional neural networks with a convolution kernel size of 1*1, each one-dimensional convolutional neural network comprising a one-dimensional convolution layer, a batch normalization layer (BN) and a linear rectifier function (RELU); the feature fusion module comprises a shallow feature fusion part and a deep feature fusion part, the shallow feature fusion part comprising three one-dimensional convolutional neural networks with a convolution kernel size of 1*1, each one-dimensional convolutional neural network comprising a one-dimensional convolution layer, a batch normalization layer (BN) and a linear rectifier function (RELU); four one-dimensional convolutional neural networks with a convolution kernel size of 1*1, two linear rectifier functions (RELU) and two Sigmoid activation functions, and the deep feature fusion part comprising two layer normalization, a multi-head mixed attention and a feedforward fully connected layer. The hyperspectral data X e R B×C and the volatile data Y e R B×T are respectively input into the hyperspectral feature extraction module and the volatile feature extraction module, to obtain two different feature maps: X1 e R B×C and Y1 e R B×T , wherein X1 is a hyperspectral feature map, Y1 is a volatile feature map, B is the number of batches, C is the number of bands of the hyperspectral data, and T is the time dimension; X1 and Y1 are input into the shallow part of the feature fusion module to obtain a characteristic level shallow fusion feature F1 e R B×M , wherein M is the feature dimension of F1; F1 is input into the deep part of the feature fusion module to obtain a characteristic level deep fusion feature F2 e R B×M ; F2 is input into the linear classification module to obtain a germplasm vitality detection result SS2 e R B×2 .

[0116] The decision level fusion germplasm vitality classification detection model in S1005 is described as follows:

[0117] The decision level fusion germplasm vitality classification detection model adds two linear classification layers and a decision fusion module on the basis of the characteristic level fusion germplasm vitality classification detection model. The decision fusion module comprises two convolution layers with a convolution kernel size of 1*1. Two different feature maps X1 e R B×C and Y1 e R B×T obtained by the S4 characteristic level fusion model are respectively input into the linear classification layers to respectively obtain two different classification results SS3 e R B×2 and SS4 e R B×2wherein B is the number of batches, C is the number of hyperspectral data bands, and T is the time dimension; the life detection result SS2 of the germplasm life classification detection model obtained by feature-level fusion is in R B×2 The volatile feature extraction module SS3 is in R B×2 The life detection result SS4 of the germplasm life detection result obtained by the hyperspectral feature extraction module is in R B×2 The life detection result SS of the decision-level fusion germplasm life classification detection model obtained by inputting the decision fusion module is in R e B×2 .

[0118] Further, the hyperspectral data and volatile data in the data set in S1001 are preprocessed and feature selected, and the specific steps include the following steps:

[0119] S10011, the three-dimensional volatile data is processed by SG filtering per sensor;

[0120] S10012, the collection time period of stable change of three-dimensional volatile data is selected;

[0121] S10013, the hyperspectral data is processed by multivariate scatter correction and SG filtering per channel, standardization and second derivative processing.

[0122] S10014, the hyperspectral data is selected for modeling of all feature bands.

[0123] Further, the data-level fusion germplasm life classification detection model in S1003 is described as follows:

[0124] The data-level fusion germplasm life classification detection model is composed of two linear layers, a linear rectifier function (RELU) and a dropout layer, and the data splicing process is realized by the following formula:

[0125] F = Concat (X, Y)

[0126] Wherein, F is the data-level fusion data.

[0127] The spliced data is input into the classification layer to obtain the life detection result of the data-level fusion germplasm life classification detection model, and the specific implementation process is realized by the following formula:

[0128] SS1 = Linear (Dropout (Linear (RELU (F))))

[0129] Linear (F) = w·F + b

[0130] RELU (F) = max (F, 0)

[0131] ​where w is a weight matrix, b is a bias, and Dropout represents a zeroing operation.

[0132] Further, the characteristic level fusion germplasm vitality classification detection model in S1004 includes a volatile feature extraction module, a hyperspectral feature extraction module, a feature fusion module, and a linear classification module. The volatile feature extraction module includes five time series convolutional neural networks with a convolution kernel size of 1x1, each of which includes a time series convolutional layer, a batch normalization layer (BN), and a linear rectifier function (RELU); the hyperspectral feature extraction module includes five one-dimensional convolutional neural networks with a convolution kernel size of 1x1, each of which includes a one-dimensional convolutional layer, a batch normalization layer (BN), and a linear rectifier function (RELU); the feature fusion module includes two parts of shallow feature fusion and deep feature fusion, the shallow feature fusion part includes three one-dimensional convolutional neural networks with a convolution kernel size of 1x1, each of which includes a one-dimensional convolutional layer, a batch normalization layer (BN), and a linear rectifier function (RELU); four one-dimensional convolutional neural networks with a convolution kernel size of 1x1, two linear rectifier functions (RELU), and two Sigmoid activation functions, the deep feature fusion part includes two layer normalization, a multi-head hybrid attention, and a feedforward fully connected layer; the specific steps are as follows:

[0133] wherein the volatile feature extraction module calculation process is realized by the following formula:

[0134] z i = TCN(BN(RELU(TCN(z i +z i-1 ))), i e {1, 2, 3, 4, 5, 6}

[0135]

[0136] RELU(z) = max(z, 0)

[0137] wherein z i is a feature obtained through different layer time series convolutional layers, γ and β are learnable parameter vectors, λ is a numerical stability parameter, and TCN represents a time convolutional neural network operation.

[0138] wherein the hyperspectral feature extraction module calculation process is realized by the following formula:

[0139] x i = Conv1d(BN(RELU(Conv1d(x i +x i-1 ))), i e {1, 2, 3, 4, 5, 6}

[0140]

[0141] RELU(x)=max(x,0)

[0142] wherein, x i is a feature obtained by one-dimensional convolution of different layers.

[0143] wherein, the shallow layer fuses two feature maps to obtain hyperspectral and volatile feature shallow layer fusion output F1, which is realized by the following formula:

[0144]

[0145] X1′,Y1′=Split(f)

[0146] X1″=φ(X1′)×X1′

[0147] Y1″=φ(Y1′)×Y1′

[0148] φ(x)=Sigmoid(Conv1d(RELU(Conv1d(x)))

[0149] F1=X1″+Y1″

[0150] wherein, X′, Y′, x, f, X″ and Y″ are intermediate output feature maps in the operation process, and φ and represent different calculation processes, Split represents a splitting operation, and Sigmoid represents a sigmoid function (Sigmoid) operation.

[0151] wherein, the deep layer fuses two feature maps to obtain hyperspectral and volatile feature deep layer fusion output F2, which is realized by the following formula:

[0152] X″′1=LayerNorm(X″1)

[0153] Y″′1=LayerNorm(Y″1)

[0154] F′1=FeedForward(LayerNorm(MHCA(X″′1,Y″′1)))

[0155] FeedForward(x)=Linear(RELU(Linear(x)))

[0156] MHCA(x,y)=AdaptiveFusion(Linear(x2),Linear(y2))

[0157] x2=Concat(SA i(x1), i ∈ {1, 2, …n}

[0158] y2 = Concat(SA i (y1), i ∈ {1, 2, …n}

[0159]

[0160] Q, K, V = Linear(x)

[0161] F2 = F1' + F1

[0162] wherein, X'''1, x, Y'''1, x2, y2, F'1 and y are all intermediate output feature maps in the operation process, MHCA represents the multi-head mixed attention mechanism operation, n is the number of heads of the multi-head mixed attention mechanism. Q, K, V are all weight matrices, d k is the dimension of K, SA i is the i-th layer attention mechanism, AdaptiveFusion represents the adaptive fusion operation, LayerNorm represents the layer normalization operation, FeedForward represents the feedforward fully connected layer operation, and Softmax represents the normalization exponential function (Softmax) operation.

[0163] wherein, the hyperspectral and volatile feature deep layer fusion output F2 is input into the linear classification layer to obtain the feature-level fusion germplasm vitality classification detection model vitality detection result SS2, which is specifically realized through the following formula:

[0164] SS2 = Linear(F2)

[0165] Further, the decision-level fusion germplasm vitality classification detection model constructed by S1005 comprises two convolutional layers with a convolution kernel size of 1x1; the specific data processing process is as follows:

[0166] S10051, X1 ∈ R B×C and Y1 ∈ R B×T are respectively input into the linear classification layer to respectively obtain two different classification results SS3 ∈ R B×2 and SS4 ∈ R B×2 , which are specifically realized through the following formula:

[0167] SS3 = Linear(X1)

[0168] SS4 = Linear(Y1)

[0169] S10052, the vitality detection results SS2, SS3 ∈ R B×2 and SS4 ∈ R B×2The input decision fusion module obtains a vitality detection result SS of the decision-level fused germplasm vitality classification detection model e , which is achieved by the following formula:

[0170] SS'3, SS''3 = Conv1d (SS3)

[0171] SS'4, SS''4 = Conv1d (SS4)

[0172]

[0173] wherein SS'3, SS'', SS'4, SS''4, SS 3e and SS 4e are intermediate output variables.

[0174] Further, the three-level fusion Transformer network germplasm vitality classification detection model of the multi-modal data is experimented and evaluated; specifically as follows:

[0175] The data-level fused germplasm vitality classification detection model, the feature-level fused germplasm vitality classification detection model and the decision-level fused germplasm vitality classification detection model are tested to verify the detection performance of the model.

[0176] Specifically, experiments can be performed on a multi-modal data set, using accuracy (Accuracy), F1 score (F1-Score) and the area (AUC) enclosed by the ROC curve and the coordinate axes to comprehensively evaluate the performance of the model vitality classification detection.

[0177] Using existing commonly used algorithms, experiments are performed on a multi-modal data set, and accuracy (Accuracy), F1 score (F1-Score) and the area (AUC) enclosed by the ROC curve and the coordinate axes are used for comparative analysis, providing a basis for comprehensive evaluation of the performance of the three different multi-modal germplasm vitality detection algorithms at the data level, feature level and decision level. Table 1 shows the data set used for soybean germplasm resource vitality classification prediction, and Table 2 shows the multi-modal multi-level classification prediction results of soybean germplasm resource vitality.

[0178] Table 1:

[0179] Dataset Viable Non-viable First soybean variety 87 13 Second soybean variety 9 91 Third soybean variety 79 21

[0180] Table 2:

[0181]

[0182] In the above embodiments, although the steps are numbered S1, S2, etc., it is only a specific embodiment given by the present application, and those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, which is within the protection scope of the present application. It can be understood that in some embodiments, part or all of the above embodiments can be included.

[0183] As shown in Figure 6 The soybean germplasm vitality small sample classification prediction system 200 of the embodiment of the present application includes a data acquisition module 201, a data preprocessing module 202, a data training module 203, and a prediction module 204.

[0184] The data acquisition module 201 is configured to acquire hyperspectral data and volatile data of each sample soybean seed in the sample soybean germplasm resource, and acquire a vitality label of each sample soybean seed in the sample soybean germplasm resource, to form a multi-modal data set of the sample soybean germplasm resource.

[0185] The data preprocessing module 202 is configured to preprocess the multi-modal data set.

[0186] The data training module 203 is configured to train a soybean germplasm small sample vitality detection model based on the preprocessed multi-modal data set, to obtain a soybean seed vitality detection model.

[0187] The prediction module 204 is configured to acquire hyperspectral data and volatile data of a to-be-detected soybean seed, and use the soybean seed vitality detection model to detect the vitality of the to-be-detected soybean seed, to obtain a vitality detection result of the to-be-detected soybean seed.

[0188] Optionally, in the above technical solution, the data preprocessing module 202 is specifically configured to perform multivariate scatter correction processing, SG filtering processing, standardization processing, and second-order derivative processing on the hyperspectral data in the multi-modal data set channel by channel; and perform SG filtering processing on the volatile data in the multi-modal data set.

[0189] Optionally, in the above technical solution, the soybean germplasm small sample vitality detection model includes a feature extraction part, a feature fusion part, and a decision fusion part, or the soybean germplasm small sample vitality detection model includes a feature extraction part and a feature fusion part.

[0190] The feature extraction part includes a volatile feature extraction module and a hyperspectral feature extraction module. The volatile feature extraction module includes a time series convolution, and obtains information between multiple different time dimensions from the preprocessed volatile data through continuous residual connection, to obtain time series data features. The hyperspectral feature extraction module includes a one-dimensional convolutional neural network, and uses continuous residual connection to obtain spectral data features from the preprocessed hyperspectral data.

[0191] The feature fusion part comprises a shallow feature fusion module and a deep feature fusion module. The shallow feature fusion module performs local weighted fusion on the time series data features and the spectral data features through one-dimensional convolution and a Sigmoid activation function to obtain a first fusion result. The deep feature fusion module processes the first fusion result through a multi-head mixed self-attention module to obtain a second fusion result.

[0192] The decision fusion part uses one-dimensional convolution to respectively map the time series data features and the spectral data features into two classification scores with equal dimensions, and fuses the two classification scores with a feature fusion module classification result obtained based on the second fusion result to obtain the life force detection result of the to-be-detected soybean.

[0193] Optionally, in the above technical solution, the data acquisition module 201 is specifically configured to: acquire hyperspectral data of the sample soybean germplasm resources by using a hyperspectral device, acquire volatile data of the sample soybean germplasm resources by using an olfactory detection device, and obtain a life force label of each sample soybean seed in the sample soybean germplasm resources by a standard germination test. The life force label is germination or no germination.

[0194] Optionally, in the above technical solution, an interaction module is further included, and the interaction module is configured to: receive operation instructions of a user through a visual interactive interface to complete the life force detection of the to-be-detected soybean.

[0195] It should be noted that the beneficial effects of the soybean germplasm resource life force small sample classification prediction system provided in the above embodiments are the same as those of the soybean germplasm resource life force small sample classification prediction method, and will not be repeated here. In addition, when the system provided in the above embodiments implements its functions, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the system is divided into different functional modules according to actual conditions to complete all or part of the above described functions. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0196] The soybean germplasm resource life force small sample classification prediction system of the present application can be a computer program (including program code) running in a computer device. For example, the soybean germplasm resource life force small sample classification prediction system of the present application is an application software, which can be used to execute the corresponding steps in the soybean germplasm resource life force small sample classification prediction method of the present application.

[0197] In some embodiments, the soybean germplasm viability small sample classification prediction system of the present application can be implemented in a combination of hardware and software. For example, the soybean germplasm viability small sample classification prediction system of the present application can be a hardware decoding processor programmed to perform the soybean germplasm viability small sample classification prediction method of the present application. For example, the hardware decoding processor can be one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic elements.

[0198] The modules described in the embodiments of the present application can be implemented in software or hardware. In some cases, the names of the modules do not limit the modules themselves.

[0199] An electronic device according to an embodiment of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the soybean germplasm viability small sample classification prediction methods described above. That is, an electronic device according to an embodiment of the present application can include, but is not limited to, a processor and a memory. The memory is configured to store a computer program. The processor is configured to execute the soybean germplasm viability small sample classification prediction method according to any of the embodiments of the present application by invoking the computer program.

[0200] In an alternative embodiment, an electronic device is provided, as shown in Figure 7 Figure 7 The electronic device 4000 shown in the figure includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 can also include a transceiver 4004, which can be used for data exchange between the electronic device and other electronic devices, such as data transmission and / or data reception, etc. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not limit the embodiments of the present application.

[0201] ​The processor 4001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in connection with the present disclosure. The processor 4001 can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0202] The bus 4002 can include a path for transmitting information between the above-mentioned components. The bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 4002 can be divided into an address bus, a data bus, a control bus, and the like. For convenience of representation, Figure 7 The bus 4002 is represented by only one thick line, but it does not mean that there is only one bus or only one type of bus.

[0203] The memory 4003 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0204] The memory 4003 is configured to store application code (computer program) for implementing the scheme of the present application, and the processor 4001 is configured to control the execution. The processor 4001 is configured to execute the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0205] The electronic device can also be a terminal device, and the terminal device can be any device that can install an application, including at least one of a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart television, and a smart vehicle device.

[0206] It should be noted that, Figure 7 The electronic device shown is only an example and should not limit the functions and use range of the embodiments of the present application.

[0207] The computer readable storage medium of the embodiment of the present application, the computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement any one of the above-mentioned soybean germplasm resource viability small sample classification prediction methods.

[0208] Optionally, the computer readable storage medium can be a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a read-only compact disc (Compact Disc Read-Only Memory, CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0209] In the exemplary embodiments, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the electronic device to perform any one of the above-mentioned soybean germplasm resource viability small sample classification prediction methods.

[0210] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0211] It should be understood that the flowchart and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of various embodiments of the present application. In this regard, each block in the flowchart and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the block can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or combinations of hardware and software.

[0212] The computer readable storage medium of embodiments of the present application can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, the computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0213] The computer readable storage medium described above bears one or more programs, when the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0214] The above description is merely exemplary of the application and the application principles used. It should be understood by those skilled in the art that the disclosed scope of the application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or equivalent features without departing from the disclosed concept. For example, the above features can be replaced with the technical features disclosed in the application (but not limited to) having similar functions to form technical solutions.

[0215] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and represent a specific order or sequence. The order of use of similar objects can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described.

[0216] Those skilled in the art know that the application can be implemented as a system, a method or a computer program product, so the application can be specifically implemented as follows: it can be a complete hardware, a complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" in this paper. In addition, in some embodiments, the application can also be implemented as a computer program product in one or more computer readable media, which contains computer readable program code.

[0217] Although the embodiments of the application have been shown and described above, it should be understood that the above embodiments are exemplary and cannot be understood as limiting the application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the application.

Claims

1. A soybean germplasm viability small sample classification prediction method, characterized in that, include: The hyperspectral and volatile data of each soybean seed in the sample soybean germplasm resource were obtained, and the viability label of each soybean seed in the sample soybean germplasm resource was obtained to form a multimodal dataset of the sample soybean germplasm resource. The multimodal dataset is preprocessed; Based on the preprocessed multimodal dataset, a small sample viability detection model for soybean germplasm was trained to obtain a soybean seed viability detection model. The hyperspectral data and volatile data of the soybean seeds to be tested are obtained, and the viability of the soybean seeds to be tested is detected using the soybean seed viability detection model to obtain the viability detection results of the soybean seeds to be tested. Preprocessing of the multimodal dataset includes: The hyperspectral data in the multimodal dataset undergoes multivariate scattering correction, SG filtering, normalization, and second-derivative processing channel by channel; the volatile data in the multimodal dataset undergoes SG filtering. The soybean germplasm small sample viability detection model includes a feature extraction part, a feature fusion part, and a decision fusion part; or, the soybean germplasm small sample viability detection model only includes the feature extraction part and the feature fusion part. The feature extraction section includes a volatile feature extraction module and a hyperspectral feature extraction module. The volatile feature extraction module includes temporal convolution and obtains information between multiple different time dimensions from the preprocessed volatile data through continuous residual connections to obtain temporal data features. The hyperspectral feature extraction module includes a one-dimensional convolutional neural network and obtains spectral data features from the preprocessed hyperspectral data using continuous residual connections. The feature fusion section includes a shallow feature fusion module and a deep feature fusion module. The shallow feature fusion module performs local weighted fusion of the time-series data features and the spectral data features through one-dimensional convolution and the sigmoid activation function to obtain a first fusion result. The deep feature fusion module processes the first fusion result through a multi-head hybrid self-attention module to obtain a second fusion result. The decision fusion part uses one-dimensional convolution to map the time-series data features and spectral data features into two classification scores with equal dimensions. The two classification scores are then fused with the classification result of the feature fusion module based on the second fusion result to obtain the viability detection result of the soybean to be tested. The deep feature fusion module outputs a classification result after being processed by a linear classification layer. When the soybean germplasm small sample viability detection model only includes the feature extraction part and the feature fusion part, the classification result obtained based on the second fusion result is directly used as the viability detection result of the soybean to be tested.

2. The method according to claim 1, wherein the method is characterized by, Obtain hyperspectral and volatile data of the sample soybean germplasm resources, and obtain the viability tags of each soybean seed in the sample soybean germplasm resources, including: The hyperspectral data of the sample soybean germplasm resources is obtained by using a hyperspectral device, the volatile data of the sample soybean germplasm resources is obtained by using an olfactory detection device, and the viability label of each sample soybean seed in the sample soybean germplasm resources is obtained through a standard germination test, and the viability label is germination or no germination.

3. The method according to claim 1, wherein the method is characterized by, Further comprising: Through a visual interactive interface, an operation instruction of a user is received to complete the viability detection of the soybean to be detected.

4. A soybean germplasm viability small sample classification prediction system, characterized in that, Comprising a data acquisition module, a data preprocessing module, a data training module and a prediction module; The data acquisition module is configured to: acquire the hyperspectral data and the volatile data of each sample soybean seed in the sample soybean germplasm resources, and acquire the viability label of each sample soybean seed in the sample soybean germplasm resources, to form a multi-modal data set of the sample soybean germplasm resources; The data preprocessing module is configured to: preprocess the multi-modal data set; The data training module is configured to: based on the preprocessed multi-modal data set, train a soybean germplasm small sample viability detection model to obtain a soybean seed viability detection model; The prediction module is configured to: acquire the hyperspectral data and the volatile data of the soybean seed to be detected, and use the soybean seed viability detection model to perform viability detection on the soybean seed to be detected, to obtain a viability detection result of the soybean seed to be detected; The data preprocessing module is specifically configured to: perform multivariate scatter correction processing, SG filtering processing, standardization processing and second derivative processing on the hyperspectral data in the multi-modal data set channel by channel; and perform SG filtering processing on the volatile data in the multi-modal data set; The soybean germplasm small sample viability detection model comprises a feature extraction part, a feature fusion part and a decision fusion part, or the soybean germplasm small sample viability detection model comprises the feature extraction part and the feature fusion part; The feature extraction part comprises a volatile feature extraction module and a hyperspectral feature extraction module, the volatile feature extraction module comprises a time series convolution, and information between a plurality of different time dimensions is acquired from the preprocessed volatile data through continuous residual connection to obtain time series data features; the hyperspectral feature extraction module comprises a one-dimensional convolutional neural network, and spectral data features are obtained from the preprocessed hyperspectral data using continuous residual connection; The feature fusion part comprises a shallow feature fusion module and a deep feature fusion module, the shallow feature fusion module performs local weighted fusion on the time series data features and the spectral data features through one-dimensional convolution and a Sigmoid activation function to obtain a first fusion result; the deep feature fusion module processes the first fusion result through a multi-head mixed self-attention module to obtain a second fusion result; The decision fusion part uses one-dimensional convolution to respectively map the time series data features and the spectral data features into two dimensions of equal classification scores, and fuses the two classification scores with a classification result based on the second fusion result to obtain the viability detection result of the soybean to be detected. The deep feature fusion module outputs a classification result after being processed by a linear classification layer.

5. An electronic device, comprising: The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the soybean germplasm resource life small sample classification prediction method in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the soybean germplasm resource life small sample classification prediction method in any one of claims 1 to 3.

Images