A method and system for agricultural product quality perception based on cross-feature domain knowledge transfer

Abstract

The present application provides a method and system for perceiving the quality of agricultural products based on cross-feature domain knowledge transfer, constructs an image acquisition module, obtains high-spectral three-dimensional images of multiple target agricultural products, and constructs a training data set; and uses a low-field nuclear magnetic resonance instrument to detect multiple target agricultural products corresponding to the training data set to obtain labeling results; constructs a convolutional neural network, and uses the training data set and the low-field nuclear magnetic resonance labeling results to train the convolutional neural network to obtain a trained convolutional neural network; uses the trained convolutional neural network to perform quality identification on the test data set to generate agricultural product quality identification results. In this way, the physical information detected by the high-order low-field nuclear magnetic resonance instrument is used to guide the network training process, so that the trained network can directly identify the quality of the high-spectral three-dimensional images of agricultural products, and realizes cross-feature domain knowledge transfer from the low-field nuclear magnetic resonance information dimension to the high-spectral three-dimensional image spectral information dimension.

Description

Technical Field

[0001] The present invention relates to the field of agricultural product detection, and in particular to a method and system for perceiving agricultural product quality based on cross-feature domain knowledge transfer. Background Art

[0002] Agricultural product quality identification takes supplying high-quality agricultural products to the market as its primary goal, and is a key link in promoting high-quality agricultural product development and realizing agricultural and rural modernization. At present, non-destructive identification technologies such as near-infrared spectroscopy, hyperspectral imaging, computer vision, and information fusion are widely used in agricultural product quality testing. The biggest advantage of non-destructive testing technology is that it does not come into direct contact with the agricultural products being tested and will not cause secondary damage. It can use the response changes of the internal chemical composition and external characteristics of the agricultural products being tested to light, electricity, magnetism, etc. to identify their internal chemical composition and external characteristics.

[0003] Among them, hyperspectral technology integrates machine vision and spectroscopy, and can simultaneously obtain spatial and spectral information from the inspected agricultural products. It obtains more bands and has important research and application value in the field of agricultural product quality identification. However, this technology is easily affected by various factors, resulting in the recognition model's ability to predict unknown samples is not accurate enough, and cannot meet the demand for precision in the process of rapid and non-destructive identification of agricultural product quality.

[0004] Therefore, providing a method and system for identifying agricultural product quality with high accuracy has become a technical problem that urgently needs to be solved. Summary of the invention

[0005] In view of the above problems, this application is proposed to provide a method for agricultural product quality perception based on cross-feature domain knowledge transfer, including steps S1 to S3:

[0006] S1: construct an image acquisition module to obtain high-spectral three-dimensional images of multiple target agricultural products and construct a training data set; and use a low-field nuclear magnetic resonance instrument to detect multiple target agricultural products corresponding to the training data set, and then annotate the corresponding training data set to obtain the low-field nuclear magnetic resonance annotation results of the training data set;

[0007] S2: construct a convolutional neural network, and train the convolutional neural network using the training data set and the low-field nuclear magnetic resonance annotation results to obtain a trained convolutional neural network;

[0008] S3: Using the trained convolutional neural network, perform quality identification on the test data set to generate agricultural product quality identification results, wherein the test data set includes a hyperspectral three-dimensional image of the target agricultural product.

[0009] Further, a convolutional neural network is constructed, and the convolutional neural network is trained using the training data set to obtain a trained convolutional neural network, which specifically includes steps S21-S25:

[0010] S21: construct an adaptive attention module to perform weighted calculation on the hyperspectral 3D images in the training dataset to obtain a 3D attention feature map;

[0011] S22: construct a self-adjusting preprocessing module to remove the interference frequency band information of the three-dimensional attention feature map to obtain a preprocessing feature map;

[0012] S23: construct a feature extraction module to extract features from the preprocessed feature map to obtain an extracted feature map;

[0013] S24: construct a classifier, perform softmax classification on the extracted feature graph, and obtain the agricultural product quality recognition results of the hyperspectral three-dimensional image in the training data set;

[0014] S25: Construct a loss function module, and perform parameter optimization training on the convolutional neural network according to the agricultural product quality recognition results of the hyperspectral three-dimensional images and the low-field nuclear magnetic resonance annotation results in the training data set to obtain a trained convolutional neural network.

[0015] Further, weighted calculation is performed on the hyperspectral three-dimensional image in the training data set to obtain a three-dimensional attention feature map, including steps S211 to S216:

[0016] S211: average the hyperspectral three-dimensional images in the training data set along the spectral dimension to obtain a spatial dimension attention feature map;

[0017] S212: performing average processing on the hyperspectral three-dimensional images in the training data set along the spatial dimension to obtain a spectral dimension attention feature map;

[0018] S213: dilate the spatial dimension attention feature map three-dimensionally along the spectral dimension, and dilate the spectral dimension attention feature map three-dimensionally along the spatial dimension, to obtain a spatial dimension dilation feature map and a spectral dimension dilation feature map respectively;

[0019] S214: multiplying the spatial dimension expansion feature map and the spectral dimension expansion feature map element by element to obtain a three-dimensional mixed feature map;

[0020] S215: Perform Sigmoid processing on the three-dimensional mixed feature map to obtain a three-dimensional attention weight;

[0021] S216: Multiply the hyperspectral three-dimensional image by the three-dimensional attention weight to obtain a three-dimensional attention feature map.

[0022] Furthermore, the hyperspectral three-dimensional image is averaged along the spectral dimension, which is expressed as:

[0023] ;

[0024] in, is the spatial dimension attention feature map, , H, W, C are the height, width and spectrum number of the hyperspectral three-dimensional image F, , is the spatial dimension feature map of the cth spectrum of the hyperspectral three-dimensional image, c∈[1,C];

[0025] The hyperspectral 3D image is averaged along the spatial dimension, expressed as:

[0026] ;

[0027] in, is the spectral dimension attention feature map, y C (p,q) is the pixel feature at position (p,q) of the hyperspectral three-dimensional image, p∈[1,H], q∈[1,W].

[0028] Furthermore, the three-dimensional attention feature map can be expressed as:

[0029] ;

[0030] in, is the three-dimensional attention feature map, is element-wise multiplication, is the Sigmoid function.

[0031] Further, removing the interference frequency band information of the three-dimensional attention feature map to obtain a preprocessed feature map includes steps S221 to S223:

[0032] S221: performing regularization processing on the three-dimensional attention feature map spectrum by spectrum to obtain a regularized feature map;

[0033] S222: Obtain the frequency band mean and frequency band variance of each spectrum of the three-dimensional attention feature map;

[0034] S223: According to the regularized feature map, the frequency band mean and the frequency band variance, the interference frequency band information is removed to obtain a preprocessed feature map.

[0035] Furthermore, the frequency band mean and frequency band variance of each spectrum of the three-dimensional attention feature map are expressed as:

[0036] ;

[0037] ;

[0038] in, , is the frequency band mean and frequency band variance of each spectrum of the three-dimensional attention feature map, is the pixel feature at position (p,q) of the three-dimensional attention feature map, is a preset constant.

[0039] Furthermore, the preprocessed feature map is expressed as:

[0040] ;

[0041] in, To preprocess the feature map, is the regularized feature map, , Self-regulation coefficient.

[0042] Furthermore, a feature extraction module is constructed to extract features from the preprocessed feature map to obtain an extracted feature map, which specifically includes:

[0043] S231: constructing an overall feature extraction branch to extract overall features from the preprocessed feature graph;

[0044] S232: constructing an image segmentation module, performing overlapping segmentation on the preprocessed feature map to obtain a plurality of segmentation feature maps, wherein two adjacent segmentation feature maps have a preset overlapping rate;

[0045] S233: construct multiple detail feature extraction branches, respectively extract detail features from multiple segmentation feature maps, and obtain multiple detail feature maps;

[0046] S234: construct a feature merging module, perform sum processing on the overall feature map and multiple detail feature maps, and obtain an extracted feature map.

[0047] In addition, the present application also provides an agricultural product quality perception system based on cross-feature domain knowledge transfer, including:

[0048] The first construction module is used to construct an image acquisition module, obtain high-spectral three-dimensional images of multiple target agricultural products, and construct a training data set; and use a low-field nuclear magnetic resonance instrument to detect multiple target agricultural products corresponding to the training data set, and then annotate the corresponding training data set to obtain a low-field nuclear magnetic resonance annotation result of the training data set;

[0049] The second construction module is used to construct a convolutional neural network, and train the convolutional neural network using the training data set and the low-field nuclear magnetic resonance annotation results to obtain a trained convolutional neural network;

[0050] The recognition module is used to use the trained convolutional neural network to perform quality recognition on the test data set and generate agricultural product quality recognition results, wherein the test data set includes a hyperspectral three-dimensional image of the target agricultural product.

[0051] The agricultural product quality perception method and system based on cross-feature domain knowledge transfer provided by the present application utilizes the hyperspectral three-dimensional images of the training data set to train the convolutional neural network, and utilizes the low-field nuclear magnetic resonance annotation results of the training data set to optimize and adjust the parameters of the convolutional neural network, thereby improving the accuracy of the agricultural product quality identification results. In addition, in this process, since the training data set is annotated using low-field nuclear magnetic resonance technology, that is, the detection information of low-field nuclear magnetic resonance is utilized to guide the training process of the convolutional neural network, the trained convolutional neural network can directly identify the quality of the hyperspectral three-dimensional images of the agricultural product, thereby realizing the cross-feature domain knowledge transfer from the low-field nuclear magnetic resonance information dimension to the hyperspectral three-dimensional information dimension, thereby reducing the cost of agricultural product quality identification and perception.

[0052] On the other hand, the present application also constructs an adaptive attention module. First, by averaging along the spatial dimension, the spectral features of high concern are strengthened and the influence of the spectral features of low concern is reduced. Secondly, by averaging along the spectral dimension, the spatial region of high concern is strengthened, while the spatial region of low concern is suppressed. Finally, through three-dimensional expansion processing and Sigmoid processing, a three-dimensional attention weight is obtained, which can adapt to the three-dimensional changes of the hyperspectral three-dimensional image, keep the size of the feature map unchanged, adjust the attention weights of the spatial dimension and the spectral dimension, and selectively discard and pick up the three-dimensional features of the hyperspectral three-dimensional image, highlight the most important part of the hyperspectral three-dimensional image, suppress environmental interference features, and improve the accuracy of agricultural product quality identification.

[0053] On the other hand, the present application also constructs a self-adjusting preprocessing module, which takes into account the correlation and relationship between adjacent spectral bands, calculates the band mean and band variance according to the changes in adjacent spectral data, performs a linear transformation on the three-dimensional attention feature map, removes interfering frequency band information, and makes it robust to different noise distributions.

[0054] On the other hand, the present application also constructs a feature extraction module, uses overlapping segmentation to obtain multiple segmentation feature maps, and uses a small convolution kernel to extract detail features to ensure that the changes in frequency band details in the hyperspectral three-dimensional image can be obtained. At the same time, a large convolution kernel is used for overall feature extraction to ensure that all frequency band information in the hyperspectral three-dimensional image can be obtained, thereby comprehensively improving the accuracy of hyperspectral three-dimensional image feature extraction and improving the accuracy of agricultural product quality identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 It is a schematic diagram of a scenario for applying the agricultural product quality perception method based on cross-feature domain knowledge transfer according to the present application;

[0057] Figure 2 This is a flow chart of a method for perceiving agricultural product quality based on cross-feature domain knowledge transfer according to an embodiment of the present application;

[0058] Figure 3 A specific schematic diagram of a convolutional neural network provided in one embodiment of the present application;

[0059] Figure 4 An execution flow chart of a convolutional neural network provided in one embodiment of the present application;

[0060] Figure 5 An execution flow chart of an adaptive attention module provided in one embodiment of the present application;

[0061] Figure 6 An execution flow chart of a self-adjusting preprocessing module provided in an embodiment of the present application;

[0062] Figure 7 An execution flow chart of a feature extraction module provided in one embodiment of the present application;

[0063] Figure 8 It is a module diagram of an agricultural product quality perception system based on cross-feature domain knowledge transfer according to an embodiment of the present application;

[0064] Fig. 9 It is a hardware structure diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "said", and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.

[0067] Ginseng Figure 1 In a typical system architecture of the present application, a server and a terminal may be included. A user may use the terminal to interact with the server to receive or send messages, etc. The agricultural product quality perception method based on cross-feature domain knowledge transfer disclosed in the present application may be executed by a server, and accordingly, the agricultural product quality perception system based on cross-feature domain knowledge transfer disclosed in the present application may be set in the server.

[0068] The user can instruct the server through the terminal to obtain high-spectral three-dimensional images of multiple target agricultural products, construct a training data set, and use a low-field nuclear magnetic resonance instrument to detect multiple target agricultural products corresponding to the training data set, and then annotate the corresponding training data set to obtain the low-field nuclear magnetic resonance annotation results of the training data set; thereafter, the server can further use the training data set and the low-field nuclear magnetic resonance annotation results to train the convolutional neural network to obtain a trained convolutional neural network; the trained convolutional neural network can be saved locally (i.e., the terminal), and can be used to use the trained convolutional neural network through the terminal to perform quality identification on the test data set and generate agricultural product quality identification results.

[0069] Of course, in some system architectures, the agricultural product quality perception method based on cross-feature domain knowledge transfer disclosed in the present application can also be executed by a server cluster that can communicate with a terminal device. Accordingly, the agricultural product quality perception system based on cross-feature domain knowledge transfer disclosed in the present application can be set in a server cluster.

[0070] In a system architecture where the terminal can provide matching computing power, the agricultural product quality perception method based on cross-feature domain knowledge transfer disclosed in the present application can also be directly executed by the terminal. Accordingly, the agricultural product quality perception system based on cross-feature domain knowledge transfer disclosed in the present application can be set in the terminal.

[0071] In the overall concept of this application, considering the uncontrollability of the cost of low-field nuclear magnetic resonance detection, it is expected that the quality of agricultural products can be directly identified by using hyperspectral three-dimensional images with relatively controllable costs. To this end, a method and system for perceiving agricultural product quality based on cross-feature domain knowledge transfer are provided, and the convolutional neural network is trained using the hyperspectral three-dimensional images of the training data set, and the parameters of the convolutional neural network are optimized and adjusted using the low-field nuclear magnetic resonance annotation results of the training data set, so as to improve the accuracy of the agricultural product quality identification results. In this process, since the training data set is annotated using low-field nuclear magnetic resonance technology, that is, the training process of the convolutional neural network is guided by the detection information of low-field nuclear magnetic resonance, the trained convolutional neural network can directly identify the quality of the hyperspectral three-dimensional images of agricultural products, and the cross-feature domain knowledge transfer from the low-field nuclear magnetic resonance information dimension to the hyperspectral three-dimensional information dimension is realized.

[0072] At the same time, the present application also constructs an adaptive attention module. First, by averaging along the spatial dimension, the spectral features of high concern are strengthened and the influence of the spectral features of low concern is reduced. Secondly, by averaging along the spectral dimension, the spatial region of high concern is strengthened, while the spatial region of low concern is suppressed. Finally, through three-dimensional expansion processing and Sigmoid processing, the three-dimensional attention weight is obtained, which can adapt to the three-dimensional changes of the hyperspectral three-dimensional image, keep the size of the feature map unchanged, adjust the attention weights of the spatial dimension and the spectral dimension, and selectively discard and pick up the three-dimensional features of the hyperspectral three-dimensional image, highlight the most important part of the hyperspectral three-dimensional image, suppress environmental interference features, and improve the accuracy of agricultural product quality identification.

[0073] Secondly, this application also constructs a self-adjusting preprocessing module, which takes into account the correlation and relationship between adjacent spectral bands, calculates the band mean and band variance according to the changes in adjacent spectral data, performs a linear transformation on the three-dimensional attention feature map, removes interfering frequency band information, and makes it robust to different noise distributions.

[0074] Finally, this application also constructs a feature extraction module, uses overlapping segmentation to obtain multiple segmentation feature maps, and uses a small convolution kernel to extract detail features to ensure that the changes in frequency band details in the hyperspectral three-dimensional image can be obtained. At the same time, a large convolution kernel is used for overall feature extraction to ensure that all frequency band information in the hyperspectral three-dimensional image can be obtained, thereby comprehensively improving the accuracy of hyperspectral three-dimensional image feature extraction and improving the accuracy of agricultural product quality identification.

[0075] The present application is further described below in conjunction with the accompanying drawings and specific embodiments.

[0076] like Figure 2As shown, in one embodiment, the present application provides a method for perceiving agricultural product quality based on cross-feature domain knowledge transfer. The method specifically includes the following steps S1 to S3:

[0077] S1: Construct an image acquisition module to obtain high-spectral three-dimensional images of multiple target agricultural products and construct a training data set; and use a low-field nuclear magnetic resonance instrument to detect multiple target agricultural products corresponding to the training data set, and then annotate the corresponding training data set to obtain the low-field nuclear magnetic resonance annotation results of the training data set.

[0078] Specifically, a hyperspectral imager is used to obtain a hyperspectral three-dimensional image of the target agricultural product. The hyperspectral image is a three-dimensional cube composed of images of different bands, which contains rich texture-spectral joint features. The hyperspectral image can be used to obtain the basic image for agricultural product quality identification. Compared with the hyperspectral imager, the low-field nuclear magnetic resonance instrument is not easily affected by the environment and has high accuracy. Therefore, this application uses a low-field nuclear magnetic resonance instrument to annotate multiple target agricultural products corresponding to the training data set to obtain the low-field nuclear magnetic resonance annotation results corresponding to the training data set.

[0079] S2: Construct a convolutional neural network, and train the convolutional neural network using the training data set and the low-field nuclear magnetic resonance annotation results to obtain a trained convolutional neural network.

[0080] S3: Use the trained convolutional neural network to perform quality identification on the test data set and generate agricultural product quality identification results.

[0081] This application uses the hyperspectral three-dimensional images of the training data set to train the convolutional neural network, and uses the low-field nuclear magnetic resonance annotation results of the training data set to optimize and adjust the parameters of the convolutional neural network to improve the accuracy of agricultural product quality identification results.

[0082] In an application scenario of the present application, the moisture content of corn can be detected based on the method provided in the above embodiment of the present application. The moisture content of corn is directly related to its quality. Too high a moisture content will cause a series of safety problems, resulting in a large loss of grain. For example, a high moisture content will enhance the respiration of grain and cause a large amount of its nutrients to be consumed; a high moisture environment will be conducive to the growth and reproduction of harmful microorganisms and pests, resulting in the production of a large amount of toxic and harmful substances; too high a moisture content will also cause its seeds to germinate prematurely.

[0083] First, a multispectral imaging system can be used to collect multispectral images at 10 discontinuous wavelengths at 405, 450, 470, 525, 590, 645, 700, 780, 870, and 970 nm. The image obtained by multispectral imaging technology is a three-dimensional (x, y, λ) multispectral photo, where x and y represent multispectral image information and λ represents multispectral wavelength information. These images can be used as basic data for constructing a training data set. Subsequently, a low-field nuclear magnetic resonance instrument is used to perform nuclear magnetic resonance imaging on the corn corresponding to the training data set to obtain moisture content data as a reference, and then the training data set is annotated. Finally, the training data set is used to train a convolutional neural network to obtain a convolutional neural network that can directly detect and identify moisture in the hyperspectral three-dimensional image of corn, and deploy it on a suitable terminal or server.

[0084] Specifically, in one embodiment of the present application, the specific principle diagram of the convolutional neural network is as follows Figure 3 As shown, a convolutional neural network is constructed, and the convolutional neural network is trained using a training data set to obtain a trained convolutional neural network, which specifically includes steps S21-S25, such as Figure 4 As shown:

[0085] S21: Construct an adaptive attention module to perform weighted calculation on the hyperspectral 3D images in the training dataset to obtain a 3D attention feature map.

[0086] Specifically, Figure 5 As shown, weighted calculation is performed on the hyperspectral three-dimensional image in the training data set to obtain a three-dimensional attention feature map, including the following steps S211 to S216:

[0087] S211: The hyperspectral three-dimensional images in the training data set are averaged along the spectral dimension to obtain a spatial dimension attention feature map.

[0088] The hyperspectral 3D image is averaged along the spectral dimension, which can be expressed as:

[0089] ;

[0090] in, is the spatial dimension attention feature map, , H, W, C are the height, width and spectrum number of the hyperspectral three-dimensional image F, , is the spatial dimension feature map of the cth spectrum of the hyperspectral three-dimensional image, c∈[1,C];

[0091] S212: The hyperspectral three-dimensional images in the training data set are averaged along the spatial dimension to obtain a spectral dimension attention feature map.

[0092] The hyperspectral 3D image is averaged along the spatial dimension and can be expressed as:

[0093] ;

[0094] in, is the spectral dimension attention feature map, y C (p,q) is the pixel feature at position (p,q) of the hyperspectral three-dimensional image, p∈[1,H], q∈[1,W].

[0095] S213: Expand the spatial dimension attention feature map three-dimensionally along the spectral dimension, and expand the spectral dimension attention feature map three-dimensionally along the spatial dimension, to obtain a spatial dimension expansion feature map and a spectral dimension expansion feature map respectively.

[0096] in, Dilate the feature map for the spatial dimension, , is the feature map expanded for the spectral dimension, .

[0097] S214: Multiply the spatial dimension expansion feature map and the spectral dimension expansion feature map element by element to obtain a three-dimensional mixed feature map.

[0098] S215: Perform Sigmoid processing on the three-dimensional mixed feature map to obtain a three-dimensional attention weight.

[0099] S216: Multiply the hyperspectral three-dimensional image by the three-dimensional attention weight to obtain a three-dimensional attention feature map.

[0100] The three-dimensional attention feature map can be expressed as:

[0101] ;

[0102] in, is the three-dimensional attention feature map, is element-wise multiplication, is the Sigmoid function.

[0103] In the present application, the adaptive attention module includes a spatial dimension attention branch and a spectral dimension attention branch. First, by averaging along the spatial dimension, each element in the spectral dimension attention feature map is used to represent the average value of the two-dimensional spatial feature, so that the spectral features of high concern are strengthened and the influence of the spectral features of low concern is reduced. Secondly, by averaging along the spectral dimension, each element in the spatial dimension attention feature map is used to represent the average value of the channel dimension, so that the spatial region of high concern is strengthened, while the spatial region of low concern is suppressed. Finally, through three-dimensional expansion processing and Sigmoid processing, the three-dimensional attention weight is obtained, which can adapt to the three-dimensional changes of the hyperspectral three-dimensional image, keep the size of the feature map unchanged, adjust the spatial dimension and spectral dimension attention weights, selectively discard and pick up the three-dimensional features of the hyperspectral three-dimensional image, highlight the most important part of the hyperspectral three-dimensional image, suppress environmental interference features, and improve the accuracy of subsequent agricultural product quality identification.

[0104] S22: Construct a self-adjusting preprocessing module to remove the interference frequency band information of the three-dimensional attention feature map to obtain a preprocessing feature map.

[0105] Specifically, Figure 6 As shown, removing the interference frequency band information of the three-dimensional attention feature map to obtain a preprocessed feature map includes the following steps S221 to S223:

[0106] S221: Regularize the three-dimensional attention feature map spectrum by spectrum to obtain a regularized feature map.

[0107] S222: Obtain the frequency band mean and frequency band variance of each spectrum of the three-dimensional attention feature map.

[0108] The frequency band mean and frequency band variance of each spectrum of the three-dimensional attention feature map can be expressed as:

[0109] ;

[0110] ;

[0111] in, , is the frequency band mean and frequency band variance of each spectrum of the three-dimensional attention feature map, is the pixel feature at position (p,q) of the three-dimensional attention feature map, is a preset constant.

[0112] S223: According to the regularized feature map, the frequency band mean and the frequency band variance, the interference frequency band information is removed to obtain a preprocessed feature map.

[0113] The preprocessed feature map can be expressed as:

[0114] ;

[0115] in, To preprocess the feature map, is the regularized feature map of each spectrum of the three-dimensional attention feature map, , Self-regulation coefficient.

[0116] In this application, the hyperspectral image is composed of a large number of bands, each of which captures different spectral information of a specific wavelength, but inevitably introduces interfering spectral information from other wavelengths. The self-adjusting preprocessing module considers the correlation and relationship between adjacent spectral bands, calculates the band mean and band variance based on the changes in adjacent spectral data, performs a linear transformation on the three-dimensional attention feature map, removes interfering frequency band information, and makes it robust to different noise distributions.

[0117] Furthermore, the preset constant Preferably ,prevent is zero.

[0118] S23: construct a feature extraction module, perform feature extraction on the preprocessed feature map, and obtain an extracted feature map.

[0119] Specifically, Figure 7 As shown, a feature extraction module is constructed to extract features from the preprocessed feature map to obtain an extracted feature map, which specifically includes steps S231-S234:

[0120] S231: construct an overall feature extraction branch to perform overall feature extraction on the preprocessed feature map.

[0121] Furthermore, six convolution kernels and three maximum pooling layers may be connected in sequence to perform overall feature extraction on the preprocessed feature map to obtain an overall feature map.

[0122] S232: construct an image segmentation module, perform overlapping segmentation on the preprocessed feature map, and obtain multiple segmentation feature maps, wherein two adjacent segmentation feature maps have a preset overlapping rate.

[0123] S233: construct multiple detail feature extraction branches, respectively extract detail features from multiple segmentation feature maps, and obtain multiple detail feature maps.

[0124] Furthermore, each detail feature extraction branch may use 4 convolution kernels and 2 maximum pooling layers connected in sequence to extract detail features from the segmentation feature map to obtain a detail feature map. The size of the convolution kernel in the detail feature extraction branch is smaller than the size of the convolution kernel in the overall feature extraction branch.

[0125] S234: construct a feature merging module, perform sum processing on the overall feature map and multiple detail feature maps, and obtain an extracted feature map.

[0126] In the present application, the feature extraction module uses overlapping segmentation to obtain multiple segmentation feature maps, and uses a small convolution kernel to extract detail features to ensure that the changes in frequency band details in the hyperspectral three-dimensional image can be obtained. At the same time, a large convolution kernel is used for overall feature extraction to ensure that all frequency band information in the hyperspectral three-dimensional image can be obtained, thereby comprehensively improving the accuracy of feature extraction of hyperspectral three-dimensional images and improving the accuracy of subsequent agricultural product quality identification.

[0127] S24: Construct a classifier, perform softmax classification on the extracted feature map, and obtain the agricultural product quality recognition result of the hyperspectral three-dimensional image in the test data set.

[0128] S25: Construct a loss function module, and perform parameter optimization training on the convolutional neural network according to the agricultural product quality recognition results of the hyperspectral three-dimensional images and the low-field nuclear magnetic resonance annotation results in the training data set to obtain a trained convolutional neural network.

[0129] Ginseng Figure 8 In another embodiment of the present application, a system for perceiving agricultural product quality based on cross-feature domain knowledge transfer is provided, comprising a first building module, a second building module and a recognition module.

[0130] The first construction module is used to construct an image acquisition module to obtain high-spectral three-dimensional images of multiple target agricultural products and construct a training data set; and use a low-field nuclear magnetic resonance instrument to detect multiple target agricultural products corresponding to the training data set, and then annotate the corresponding training data set to obtain the low-field nuclear magnetic resonance annotation results of the training data set; the second construction module is used to construct a convolutional neural network, and train the convolutional neural network using the training data set and the low-field nuclear magnetic resonance annotation results to obtain a trained convolutional neural network; the recognition module is used to use the trained convolutional neural network to perform quality recognition on the data set to be tested and generate agricultural product quality recognition results, wherein the data set to be tested includes a high-spectral three-dimensional image of the target agricultural product.

[0131] In one embodiment, the second construction module is specifically used to construct an adaptive attention module, perform weighted calculation on the hyperspectral three-dimensional image in the training data set, and obtain a three-dimensional attention feature map; construct a self-adjusting preprocessing module, remove the interference frequency band information of the three-dimensional attention feature map, and obtain a preprocessing feature map;

[0132] A feature extraction module is constructed to extract features from the preprocessed feature map to obtain an extracted feature map. A classifier is constructed to perform softmax classification on the extracted feature map to obtain the agricultural product quality recognition results of the hyperspectral three-dimensional images in the training data set. A loss function module is constructed to perform parameter optimization training on the convolutional neural network based on the agricultural product quality recognition results of the hyperspectral three-dimensional images in the training data set and the low-field nuclear magnetic resonance annotation results to obtain a trained convolutional neural network.

[0133] In one embodiment, the second construction module is specifically used to average the hyperspectral three-dimensional images in the training data set along the spectral dimension to obtain a spatial dimension attention feature map; average the hyperspectral three-dimensional images in the training data set along the spatial dimension to obtain a spectral dimension attention feature map; three-dimensionally expand the spatial dimension attention feature map along the spectral dimension, and three-dimensionally expand the spectral dimension attention feature map along the spatial dimension to obtain a spatial dimension expansion feature map and a spectral dimension expansion feature map, respectively; element-wise multiply the spatial dimension expansion feature map and the spectral dimension expansion feature map to obtain a three-dimensional mixed feature map; perform Sigmoid processing on the three-dimensional mixed feature map to obtain a three-dimensional attention weight; multiply the hyperspectral three-dimensional image by the three-dimensional attention weight to obtain a three-dimensional attention feature map.

[0134] In one embodiment, the hyperspectral three-dimensional image is averaged along the spectral dimension, which is expressed as:

[0135] ;

[0136] in, is the spatial dimension attention feature map, , H, W, C are the height, width and spectrum number of the hyperspectral three-dimensional image F, , is the spatial dimension feature map of the cth spectrum of the hyperspectral three-dimensional image, c∈[1,C];

[0137] The hyperspectral 3D image is averaged along the spatial dimension, expressed as:

[0138] ;

[0139] in, is the spectral dimension attention feature map, y C (p,q) is the pixel feature at position (p,q) of the hyperspectral three-dimensional image, p∈[1,H], q∈[1,W].

[0140] In one embodiment, the three-dimensional attention feature map is represented as:

[0141] ;

[0142] in, is the three-dimensional attention feature map, is element-wise multiplication, is the Sigmoid function.

[0143] In one embodiment, the second construction module is specifically used to regularize the three-dimensional attention feature map spectrum by spectrum to obtain a regularized feature map; obtain the band mean and band variance of each spectrum of the three-dimensional attention feature map; remove interference frequency band information according to the regularized feature map, the band mean and the band variance to obtain a preprocessed feature map.

[0144] In one embodiment, the frequency band mean and frequency band variance of each spectrum of the three-dimensional attention feature map are expressed as:

[0145] ;

[0146] ;

[0147] in, , is the frequency band mean and frequency band variance of each spectrum of the three-dimensional attention feature map, is the pixel feature at position (p,q) of the three-dimensional attention feature map, is a preset constant.

[0148] In one embodiment, the preprocessing feature map is represented as:

[0149] ;

[0150] in, To preprocess the feature map, is the regularized feature map, , Self-regulation coefficient.

[0151] In one embodiment, the second construction module is specifically used to construct an overall feature extraction branch to perform overall feature extraction on the preprocessed feature map; construct an image segmentation module to perform overlapping segmentation on the preprocessed feature map to obtain multiple segmentation feature maps, wherein two adjacent segmentation feature maps have a preset overlap rate; construct multiple detail feature extraction branches to perform detail feature extraction on multiple segmentation feature maps respectively to obtain multiple detail feature maps; construct a feature merging module to sum the overall feature map and multiple detail feature maps to obtain an extracted feature map.

[0152] Fig. 9 FIG. 1 shows a hardware structure diagram of an electronic device according to an embodiment of the present specification. Fig. 9As shown, the electronic device 30 may include at least one processor 31, a memory 32 (e.g., a non-volatile memory), a memory 33, and a communication interface 34, and the at least one processor 31, the memory 32, the memory 33, and the communication interface 34 are connected together via an internal bus 35. At least one processor 31 executes at least one computer-readable instruction stored or encoded in the memory 32. Furthermore, the memory 32 and the memory 33 store a program corresponding to the agricultural product quality perception method.

[0153] It should be understood that the computer executable instructions stored in the memory 32, when executed, cause at least one processor 31 to perform the above combined operations in various embodiments of the present specification. Figures 1 to 7 Describes the various operations and functions.

[0154] In the embodiments of the present specification, the electronic device 30 may include, but is not limited to, personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile electronic devices, smart phones, tablet computers, cellular phones, personal digital assistants (PDAs), handheld devices, messaging devices, wearable electronic devices, consumer electronic devices, and the like.

[0155] According to one embodiment, a program product such as a machine-readable medium is provided. The machine-readable medium may have instructions (i.e., the above-mentioned elements implemented in the form of software), which, when executed by a machine, causes the machine to perform the above-mentioned combination of various embodiments of this specification. Figure 1-Figure 7 Specifically, a system or device equipped with a readable storage medium may be provided, on which a software program code implementing the functions of any of the above-mentioned embodiments is stored, and a computer or processor of the system or device reads and executes the instructions stored in the readable storage medium.

[0156] In this case, the program code itself read from the machine-readable medium can realize the function of any one of the above embodiments, and thus the machine-readable code and the machine-readable storage medium storing the machine-readable code constitute part of this specification.

[0157] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or a cloud via a communication network.

[0158] Those skilled in the art should understand that the various embodiments disclosed above can be modified and altered in various ways without departing from the essence of the invention. Therefore, the protection scope of this specification should be defined by the appended claims.

[0159] It should be noted that not all steps and units in the above-mentioned processes and system structure diagrams are necessary, and some steps or units can be ignored according to actual needs. The execution order of each step is not fixed and can be determined as needed. The device structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some units may be implemented by the same physical client, or some units may be implemented by multiple physical clients, or some components in multiple independent devices may be implemented together.

[0160] In the above embodiments, the hardware unit or module can be realized by mechanical or electrical means. For example, a hardware unit, module or processor can include permanent dedicated circuit or logic (such as special processor, FPGA or ASIC) to complete the corresponding operation. The hardware unit or processor can also include programmable logic or circuit (such as general-purpose processor or other programmable processor), which can be temporarily set by software to complete the corresponding operation. Specific implementation (mechanical method or dedicated permanent circuit or temporary circuit) can be determined based on cost and time consideration.

[0161] The specific embodiments described above in conjunction with the accompanying drawings describe exemplary embodiments, but do not represent all embodiments that can be implemented or fall within the scope of protection of the claims. The term "exemplary" used throughout this specification means "used as an example, instance or illustration" and does not mean "preferred" or "having advantages" over other embodiments. For the purpose of providing an understanding of the described technology, the specific embodiments include specific details. However, these technologies can be implemented without these specific details. In some instances, in order to avoid making the concepts of the described embodiments difficult to understand, well-known structures and devices are shown in block diagram form.

[0162] The above description of the present disclosure is provided to enable any person of ordinary skill in the art to implement or use the present disclosure. Various modifications to the present disclosure will be apparent to those of ordinary skill in the art, and the general principles corresponding to the present disclosure may be applied to other variations without departing from the scope of protection of the present disclosure. Therefore, the present disclosure is not limited to the examples and designs described herein, but is consistent with the widest range of principles and novel features disclosed herein.

Claims

1. A method for agricultural product quality perception based on cross-feature domain knowledge transfer, characterized in that: The method comprises: S1: construct an image acquisition module to obtain high-spectral three-dimensional images of multiple target agricultural products and construct a training data set; and use a low-field nuclear magnetic resonance instrument to detect multiple target agricultural products corresponding to the training data set, and then annotate the corresponding training data set to obtain the low-field nuclear magnetic resonance annotation results of the training data set; S2: constructing a convolutional neural network, and training the convolutional neural network using the training data set and the low-field nuclear magnetic resonance annotation results to obtain a trained convolutional neural network; S3: using the trained convolutional neural network to perform quality recognition on the test data set to generate agricultural product quality recognition results, wherein the test data set includes a hyperspectral three-dimensional image of the target agricultural product; Among them, S2 specifically includes: S21: construct an adaptive attention module to perform weighted calculation on the hyperspectral 3D images in the training dataset to obtain a 3D attention feature map; S22: construct a self-adjusting preprocessing module to remove the interference frequency band information of the three-dimensional attention feature map to obtain a preprocessing feature map; S23: construct a feature extraction module to extract features from the preprocessed feature map to obtain an extracted feature map; S24: construct a classifier, perform softmax classification on the extracted feature graph, and obtain the agricultural product quality recognition results of the hyperspectral three-dimensional image in the training data set; S25: constructing a loss function module, performing parameter optimization training on the convolutional neural network according to the agricultural product quality recognition results of the hyperspectral three-dimensional image and the low-field nuclear magnetic resonance annotation results in the training data set, and obtaining a trained convolutional neural network; In S21, weighted calculation is performed on the hyperspectral three-dimensional image in the training data set to obtain a three-dimensional attention feature map, which specifically includes: S211: average the hyperspectral three-dimensional images in the training data set along the spectral dimension to obtain a spatial dimension attention feature map; S212: performing average processing on the hyperspectral three-dimensional images in the training data set along the spatial dimension to obtain a spectral dimension attention feature map; S213: dilate the spatial dimension attention feature map three-dimensionally along the spectral dimension, and dilate the spectral dimension attention feature map three-dimensionally along the spatial dimension, to obtain a spatial dimension dilation feature map and a spectral dimension dilation feature map respectively; S214: multiplying the spatial dimension expansion feature map and the spectral dimension expansion feature map element by element to obtain a three-dimensional mixed feature map; S215: Perform Sigmoid processing on the three-dimensional mixed feature map to obtain a three-dimensional attention weight; S216: Multiply the hyperspectral three-dimensional image by the three-dimensional attention weight to obtain a three-dimensional attention feature map.

2. The method according to claim 1, characterized in that The hyperspectral 3D image is averaged along the spectral dimension, expressed as: ； in, is the spatial dimension attention feature map, , H, W, C are the height, width and spectrum number of the hyperspectral three-dimensional image F, , is the spatial dimension feature map of the cth spectrum of the hyperspectral three-dimensional image, c∈[1,C]; The hyperspectral 3D image is averaged along the spatial dimension, expressed as: ； in, is the spectral dimension attention feature map, y C (p,q) is the pixel feature at position (p,q) of the hyperspectral three-dimensional image, p∈[1,H], q∈[1,W].

3. The method according to claim 2, characterized in that The three-dimensional attention feature map is represented as: ； in, is the three-dimensional attention feature map, is element-wise multiplication, is the Sigmoid function.

4. The method according to claim 3, characterized in that Removing interference frequency band information of the three-dimensional attention feature map to obtain a preprocessed feature map includes steps S221 to S223: S221: performing regularization processing on the three-dimensional attention feature map spectrum by spectrum to obtain a regularized feature map; S222: Obtain the frequency band mean and frequency band variance of each spectrum of the three-dimensional attention feature map; S223: According to the regularized feature map, the frequency band mean and the frequency band variance, the interference frequency band information is removed to obtain a preprocessed feature map.

5. The method according to claim 4, characterized in that The frequency band mean and frequency band variance of each spectrum of the three-dimensional attention feature map are expressed as: ； in, , is the frequency band mean and frequency band variance of each spectrum of the three-dimensional attention feature map, is the pixel feature at position (p,q) of the three-dimensional attention feature map, is a preset constant.

6. The method according to claim 5, characterized in that The preprocessed feature map is expressed as: ； in, To preprocess the feature map, is the regularized feature map, , Self-regulation coefficient.

7. The method according to claim 6, characterized in that Construct a feature extraction module to extract features from the preprocessed feature map to obtain the extracted feature map, which specifically includes: S231: constructing an overall feature extraction branch to extract overall features from the preprocessed feature graph; S232: constructing an image segmentation module, performing overlapping segmentation on the preprocessed feature map to obtain a plurality of segmentation feature maps, wherein two adjacent segmentation feature maps have a preset overlapping rate; S233: construct multiple detail feature extraction branches, respectively extract detail features from multiple segmentation feature maps, and obtain multiple detail feature maps; S234: construct a feature merging module, perform sum processing on the overall feature map and multiple detail feature maps, and obtain an extracted feature map.

8. An agricultural product quality perception system based on cross-feature domain knowledge transfer, characterized in that: include: The first construction module is used to construct an image acquisition module to acquire hyperspectral three-dimensional images of multiple target agricultural products and construct a training data set; A low-field nuclear magnetic resonance instrument is used to detect multiple target agricultural products corresponding to the training data set, and then the corresponding training data set is annotated to obtain the low-field nuclear magnetic resonance annotation results of the training data set; The second construction module is used to construct a convolutional neural network, and train the convolutional neural network using the training data set and the low-field nuclear magnetic resonance annotation results to obtain a trained convolutional neural network; An identification module is used to use a trained convolutional neural network to perform quality identification on a test data set to generate an agricultural product quality identification result, wherein the test data set includes a hyperspectral three-dimensional image of a target agricultural product; The convolutional neural network is constructed and trained using a training data set to obtain a trained convolutional neural network, which specifically includes steps S21-S25: S21: construct an adaptive attention module to perform weighted calculation on the hyperspectral 3D images in the training dataset to obtain a 3D attention feature map; S22: construct a self-adjusting preprocessing module to remove the interference frequency band information of the three-dimensional attention feature map to obtain a preprocessing feature map; S23: construct a feature extraction module to extract features from the preprocessed feature map to obtain an extracted feature map; S24: construct a classifier, perform softmax classification on the extracted feature graph, and obtain the agricultural product quality recognition results of the hyperspectral three-dimensional image in the training data set; S25: constructing a loss function module, performing parameter optimization training on the convolutional neural network according to the agricultural product quality recognition results of the hyperspectral three-dimensional image and the low-field nuclear magnetic resonance annotation results in the training data set, and obtaining a trained convolutional neural network; In S21, weighted calculation is performed on the hyperspectral three-dimensional image in the training data set to obtain a three-dimensional attention feature map, which specifically includes steps S211 to S216: S211: average the hyperspectral three-dimensional images in the training data set along the spectral dimension to obtain a spatial dimension attention feature map; S212: performing average processing on the hyperspectral three-dimensional images in the training data set along the spatial dimension to obtain a spectral dimension attention feature map; S213: dilate the spatial dimension attention feature map three-dimensionally along the spectral dimension, and dilate the spectral dimension attention feature map three-dimensionally along the spatial dimension, to obtain a spatial dimension dilation feature map and a spectral dimension dilation feature map respectively; S214: multiplying the spatial dimension expansion feature map and the spectral dimension expansion feature map element by element to obtain a three-dimensional mixed feature map; S215: Perform Sigmoid processing on the three-dimensional mixed feature map to obtain a three-dimensional attention weight; S216: Multiply the hyperspectral three-dimensional image by the three-dimensional attention weight to obtain a three-dimensional attention feature map.

Images