Agricultural product detection method, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention provides an agricultural product detection method, electronic equipment and a computer readable storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring an agricultural product color image and agricultural product two-dimensional near infrared spectrum data; determining a global feature vector, a deep feature vector and a discrete feature vector based on the agricultural product color image; the method comprises the following steps: firstly, determining a near infrared spectrum vector based on two-dimensional near infrared spectrum data of agricultural products, and then determining a near infrared spectrum query vector, a near infrared spectrum key vector and a near infrared spectrum numerical vector according to the near infrared spectrum vector; fusing the vectors based on a first cross attention mechanism, a second cross attention mechanism, a gating fusion mechanism and a self-attention mechanism to obtain a target agricultural product detection vector; and finally, classifying the target agricultural product detection vector to obtain an agricultural product detection classification result. The accuracy of agricultural product detection is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an agricultural product detection method, an electronic device, and a computer-readable storage medium. Background Art

[0002] Agricultural products (such as Fuji apples) vary in appearance, taste, nutritional value, etc. between different production areas, which makes the quality of agricultural products from different production areas significantly different. Therefore, it is necessary to conduct agricultural product testing to obtain the production area or origin of agricultural products.

[0003] In the existing technology, two major technical paths are mainly adopted: non-destructive testing and destructive analysis. On the one hand, although destructive testing can provide accurate data for agricultural product testing, it has some limitations. For example, not only does the test process consume samples, but some of their characteristics may also be changed during sample processing, thereby affecting the reliability of the results. On the other hand, although non-destructive testing technology, especially methods based on spectral analysis, provides a safer, faster and non-destructive testing method, these technologies also have their shortcomings. For example, poor stability and low accuracy. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide an agricultural product detection method, an electronic device, and a computer-readable storage medium, thereby improving the accuracy of agricultural product detection.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application proposes a method for detecting agricultural products, the method comprising:

[0006] Obtain color images of agricultural products and two-dimensional near-infrared spectrum data of agricultural products;

[0007] Performing global feature extraction on the agricultural product color image by a global feature extractor to obtain a global feature vector;

[0008] Performing deep feature extraction on the agricultural product color image by a deep feature extractor to obtain a deep feature vector;

[0009] Based on a two-dimensional discrete wavelet transform function, discrete features are extracted from the agricultural product color image to obtain a discrete feature vector;

[0010] Extracting spectral features from the two-dimensional near-infrared spectral data of the agricultural product by a spectral feature extractor to obtain a near-infrared spectral vector;

[0011] Determine a near infrared spectrum query vector, a near infrared spectrum key vector and a near infrared spectrum value vector according to the near infrared spectrum vector;

[0012] Based on a first cross attention mechanism, attention calculation is performed on the near infrared spectrum query vector, the global feature vector, the deep feature vector and the discrete feature vector to obtain a first agricultural product detection vector;

[0013] Based on a second cross attention mechanism, attention calculation is performed on the near infrared spectrum key vector, the near infrared spectrum numerical vector, the global feature vector, the deep feature vector and the discrete feature vector to obtain a second agricultural product detection vector;

[0014] Based on the gated fusion mechanism, the first agricultural product detection vector and the second agricultural product detection vector are fused to obtain a target fusion vector;

[0015] Based on the self-attention mechanism, attention calculation is performed on the target fusion vector to obtain a target agricultural product detection vector;

[0016] The target agricultural product detection vector is classified to obtain the agricultural product detection classification result.

[0017] Optionally, the step of extracting discrete features from the agricultural product color image based on a two-dimensional discrete wavelet transform function to obtain a discrete feature vector includes:

[0018] Determine the mean and variance of three channels of the agricultural product color image to obtain the channel mean and channel variance;

[0019] Binarizing the agricultural product color image to obtain an agricultural product grayscale image;

[0020] Performing secondary decomposition on the agricultural product grayscale image based on the two-dimensional discrete wavelet transform function to obtain secondary decomposition subbands of the image;

[0021] Determine the mean, variance and energy of the secondary decomposition sub-bands of the image to obtain the sub-band mean, sub-band variance and sub-band energy;

[0022] Feature fusion is performed according to the channel mean, the channel variance, the sub-band mean, the sub-band variance and the sub-band energy to obtain the discrete feature vector.

[0023] Optionally, the performing feature fusion according to the channel mean, the channel variance, the subband mean, the subband variance and the subband energy to obtain the discrete feature vector includes:

[0024] Performing vector splicing on the channel mean, the channel variance, the subband mean, the subband variance and the subband energy to obtain a color splicing vector of agricultural products;

[0025] The agricultural product color splicing vector is normalized to obtain the discrete feature vector.

[0026] Optionally, the gated fusion mechanism includes a target splicing weight matrix and a target activation function;

[0027] The first agricultural product detection vector and the second agricultural product detection vector are fused based on the gated fusion mechanism to obtain a target fusion vector, including:

[0028] splicing the first agricultural product detection vector and the second agricultural product detection vector to obtain a target agricultural product splicing vector;

[0029] Multiplying the target splicing weight matrix and the target agricultural product splicing vector to obtain an initial weight vector;

[0030] Calculate a target weight vector according to the target activation function and the initial weight vector;

[0031] Performing a dot product on the first agricultural product detection vector and the target weight vector to obtain a first dot product vector;

[0032] Subtracting each element of the target weight vector from 1 to obtain a difference vector, and performing a dot product between the difference vector and the second agricultural product detection vector to obtain a second dot product vector;

[0033] A target fusion vector is obtained by adding the first dot product vector and the second dot product vector.

[0034] Optionally, the self-attention mechanism includes an agricultural product query weight matrix, an agricultural product key weight matrix, and an agricultural product query value matrix;

[0035] The method of performing attention calculation on the target fusion vector based on the self-attention mechanism to obtain the target agricultural product detection vector includes:

[0036] The agricultural product query vector is obtained by multiplying the agricultural product query weight matrix and the target fusion vector;

[0037] The agricultural product key vector is obtained by multiplying the agricultural product key weight matrix and the target fusion vector;

[0038] Multiplying the agricultural product query value matrix and the target fusion vector to obtain an agricultural product value vector;

[0039] Self-attention calculation is performed based on the agricultural product query vector, the agricultural product key vector and the agricultural product value vector to obtain the target agricultural product detection vector.

[0040] Optionally, determining a near infrared spectrum query vector, a near infrared spectrum key vector and a near infrared spectrum value vector according to the near infrared spectrum vector includes:

[0041] Obtain near infrared spectrum query weight matrix, near infrared spectrum key weight matrix and near infrared spectrum numerical weight matrix;

[0042] The near infrared spectrum query vector is obtained by multiplying the near infrared spectrum query weight matrix and the near infrared spectrum vector;

[0043] The near infrared spectrum key vector is obtained by multiplying the near infrared spectrum key weight matrix and the near infrared spectrum vector;

[0044] The near-infrared spectrum numerical weight matrix is ​​multiplied by the near-infrared spectrum vector to obtain the near-infrared spectrum numerical vector.

[0045] Optionally, the first cross-attention mechanism includes a first cross-attention function, a first key weight matrix and a first numerical weight matrix, a second key weight matrix and a second numerical weight matrix, and a third key weight matrix and a third numerical weight matrix;

[0046] The method of performing attention calculation on the near infrared spectrum query vector, the global feature vector, the deep feature vector and the discrete feature vector based on the first cross attention mechanism to obtain a first agricultural product detection vector includes:

[0047] Obtain a first image key vector by multiplying the first key weight matrix and the global eigenvector;

[0048] Obtain a first image numerical vector by multiplying the first numerical weight matrix and the global eigenvector;

[0049] Based on the first cross-attention function, performing cross-attention calculation on the near-infrared spectrum query vector, the first image key vector and the first image numerical vector to obtain a first cross-attention vector;

[0050] Obtain a second image key vector by multiplying the second key weight matrix and the deep feature vector;

[0051] Multiplying the second numerical weight matrix and the deep feature vector to obtain a second image numerical vector;

[0052] Based on the first cross-attention function, performing cross-attention calculation on the near-infrared spectrum query vector, the second image key vector and the second image numerical vector to obtain a second cross-attention vector;

[0053] Obtain a third image key vector by multiplying the third key weight matrix and the discrete feature vector;

[0054] Multiplying the third numerical weight matrix and the discrete feature vector to obtain a third image numerical vector;

[0055] Based on the first cross-attention function, performing cross-attention calculation on the near-infrared spectrum query vector, the third image key vector and the third image numerical vector to obtain a third cross-attention vector;

[0056] The first agricultural product detection vector is obtained by concatenating the first cross-attention vector, the second cross-attention vector and the third cross-attention vector.

[0057] Optionally, the second cross-attention mechanism includes a second cross-attention function, a first query weight matrix, a second query weight matrix, and a third query weight matrix;

[0058] The method of performing attention calculation on the near infrared spectrum key vector, the near infrared spectrum numerical vector, the global feature vector, the deep feature vector and the discrete feature vector based on the second cross attention mechanism to obtain a second agricultural product detection vector includes:

[0059] Obtain a first image query vector by multiplying the first query weight matrix and the global feature vector;

[0060] Based on the second cross-attention function, performing cross-attention calculation on the near-infrared spectrum key vector, the near-infrared spectrum numerical vector and the first image query vector to determine a fourth cross-attention vector;

[0061] Obtain a second image query vector by multiplying the second query weight matrix and the deep feature vector;

[0062] Based on the second cross-attention function, performing cross-attention calculation on the near-infrared spectrum key vector, the near-infrared spectrum numerical vector and the second image query vector to obtain a fifth cross-attention vector;

[0063] Obtain a third image query vector by multiplying the third query weight matrix and the discrete feature vector;

[0064] Based on the second cross-attention function, performing cross-attention calculation on the near-infrared spectrum key vector, the near-infrared spectrum numerical vector and the third image query vector to determine a sixth cross-attention vector;

[0065] The second agricultural product detection vector is obtained by splicing the fourth cross-attention vector, the fifth cross-attention vector and the sixth cross-attention vector.

[0066] To achieve the above-mentioned purpose, the second aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0067] To achieve the above objectives, the third aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the method described in the first aspect is implemented.

[0068] The present application proposes a method for detecting agricultural products, an electronic device, and a computer-readable storage medium. The method extracts image features of agricultural products three times (including global feature extraction, deep feature extraction, and discrete feature extraction) to obtain global feature vectors, deep feature vectors, and discrete feature vectors. The method also extracts spectral features of agricultural products' two-dimensional near-infrared spectrum data to obtain near-infrared spectrum vectors. After obtaining the above four vectors, the first cross-attention mechanism and the second cross-attention mechanism are introduced to obtain the first agricultural product detection vector and the second agricultural product detection vector, respectively. The gated fusion mechanism is also introduced to fuse the first agricultural product detection vector and the second agricultural product detection vector to obtain a target fusion vector. Then, the target fusion vector is also calculated based on the self-attention mechanism to obtain the target agricultural product detection vector. Finally, the target agricultural product detection vector is classified to obtain the agricultural product detection classification result. The present application can realize the effective feature fusion of near-infrared spectrum and image domain by utilizing the joint action of three-time image feature extraction, spectral feature extraction, the first cross-attention mechanism, the second cross-attention mechanism, the gated fusion mechanism, and the self-attention mechanism, and can obtain a target agricultural product detection vector with high accuracy, thereby obtaining an agricultural product detection classification result with high accuracy, that is, improving the accuracy of agricultural product detection.

[0069] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 is an optional flow chart of the agricultural product detection method provided in the embodiment of the present application;

[0071] Figure 2 is a general framework diagram of the agricultural product detection method according to an embodiment of the present application;

[0072] Figure 3 yes Figure 1 Flowchart of step 104;

[0073] Figure 4 is a schematic diagram of determining a first agricultural product detection vector based on a first cross attention mechanism;

[0074] Figure 5 is a schematic diagram of determining a second agricultural product detection vector based on a second cross attention mechanism;

[0075] Figure 6 It is a schematic diagram of determining the target fusion vector based on the gated fusion mechanism;

[0076] Figure 7 This is a schematic diagram of determining the target agricultural product detection vector based on the self-attention mechanism;

[0077] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0079] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0080] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0081] First, some nouns involved in this application are analyzed:

[0082] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0083] Back propagation: The general principle of back propagation is: input the training set data into the input layer of the neural network, pass through the hidden layer of the neural network, and finally reach the output layer of the neural network and output the result; since there is an error between the output result of the neural network and the actual result, the error between the estimated value and the actual value is calculated, and the error is backpropagated from the output layer to the hidden layer until it propagates to the input layer; in the process of back propagation, the values ​​of various parameters are adjusted according to the error; the above process is continuously iterated until convergence.

[0084] At present, although destructive analysis methods can provide accurate data for agricultural product origin traceability, they have significant limitations. (1) First, such methods usually require cutting, squeezing or chemical treatment of samples (such as apples) to extract the necessary test data. This destructive test not only consumes the sample, but may also change some of its characteristics during the sample processing, thereby affecting the reliability of the results. (2) Moreover, due to the complexity and time-consuming nature of the required steps, this method is difficult to apply to large-scale or real-time testing. In addition to the operational complexity, destructive analysis also requires sophisticated and expensive experimental equipment, which increases the cost of testing. (3) At the same time, it is also highly dependent on professional operators, and experienced technicians are required to ensure accurate data collection and analysis. These factors combined restrict the practical application of destructive analysis methods in agricultural production and product circulation.

[0085] On the other hand, although nondestructive testing technologies, especially those based on spectral analysis, provide a safer, faster and nondestructive testing method, these technologies also have their shortcomings. (1) The accuracy of spectral analysis depends largely on the stability of the measurement conditions, such as light source, ambient temperature, lighting conditions, etc. (2) Spectra mainly provide chemical information of apples, lack description of the appearance characteristics of apples, and the information dimension is limited. (3) Spectral data lacks intuitiveness and interpretability for ordinary users. (4) When dealing with problems such as noise and outliers, the lack of complementary verification of multi-source data will affect the reliability of agricultural product traceability.

[0086] In order to solve at least one of the above problems, the embodiment of the present application proposes to utilize the combined effects of three-dimensional image feature extraction, spectral feature extraction, a first cross-attention mechanism, a second cross-attention mechanism, a gated fusion mechanism and a self-attention mechanism to achieve effective feature fusion of near-infrared spectrum and image domain, and obtain a target agricultural product detection vector with higher accuracy, thereby obtaining an agricultural product detection and classification result with higher accuracy, thereby improving the accuracy of agricultural product detection.

[0087] The agricultural product detection method of the embodiment of the present application can be executed by the server alone, or by the terminal alone, or by the terminal and the server together. In addition, the agricultural product detection method provided in the embodiment of the present application can also be software running on the server. The server can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the agricultural product detection method, etc., but is not limited to the above forms.

[0088] The embodiments of the present application provide an agricultural product detection method, an electronic device, and a computer-readable storage medium, which are specifically illustrated by the following embodiments. First, the agricultural product detection method in the embodiments of the present application is described.

[0089] See also Figure 1 According to one embodiment of the present application, a method for detecting agricultural products includes:

[0090] Step 101, obtaining color images of agricultural products and two-dimensional near-infrared spectrum data of agricultural products;

[0091] Step 102, extracting global features from the agricultural product color image by a global feature extractor to obtain a global feature vector;

[0092] Step 103, performing deep feature extraction on the agricultural product color image by a deep feature extractor to obtain a deep feature vector;

[0093] Step 104, extracting discrete features of the agricultural product color image based on a two-dimensional discrete wavelet transform function to obtain a discrete feature vector;

[0094] Step 105, extracting spectral features from the two-dimensional near-infrared spectral data of agricultural products by a spectral feature extractor to obtain a near-infrared spectral vector;

[0095] Step 106, determining a near infrared spectrum query vector, a near infrared spectrum key vector and a near infrared spectrum value vector according to the near infrared spectrum vector;

[0096] Step 107, based on the first cross attention mechanism, performing attention calculation on the near infrared spectrum query vector, the global feature vector, the deep feature vector and the discrete feature vector to obtain a first agricultural product detection vector;

[0097] Step 108, based on a second cross attention mechanism, performing attention calculation on the near infrared spectrum key vector, the near infrared spectrum numerical vector, the global feature vector, the deep feature vector and the discrete feature vector to obtain a second agricultural product detection vector;

[0098] Step 109, based on the gated fusion mechanism, the first agricultural product detection vector and the second agricultural product detection vector are fused to obtain a target fusion vector;

[0099] Step 110, based on the self-attention mechanism, performing attention calculation on the target fusion vector to obtain the target agricultural product detection vector;

[0100] Step 111, classify the target agricultural product detection vector to obtain the agricultural product detection classification result.

[0101] Reference Figure 2 , Figure 2 It is an overall framework diagram of the agricultural product detection method according to the embodiment of the present application.

[0102] Figure 2 On the far left are the agricultural product color image and agricultural product two-dimensional near-infrared spectrum data in step 101. Agricultural product color image refers to the color image of agricultural products (such as apples), also known as RGB image. Agricultural product color image can be obtained by photographing agricultural products with a camera, mobile phone or other image acquisition equipment. Agricultural product two-dimensional near-infrared spectrum data refers to the two-dimensional near-infrared spectrum data of agricultural products. Agricultural product two-dimensional near-infrared spectrum data can be obtained in the following way: first, the agricultural product is tested by a near-infrared spectrometer to obtain the one-dimensional near-infrared spectrum data of the agricultural product; then, the Gram angle field ( Figure 2The GAF (shown) converts the one-dimensional near-infrared spectrum data of agricultural products to obtain the two-dimensional near-infrared spectrum data of agricultural products. The full name of GAF is Gramian Angu l ar Field.

[0103] In one example, the color image of agricultural products is obtained by the following steps: first, an apple is placed on a black background, and a smartphone is held at a distance of 25 cm to collect images of the top, bottom, and sides of the apple, including one top image, one bottom image, and three side images. For each image, the background is black and the apple is located in the center of the image. The resolution of each image is 3000*3000. Then, the cv2 toolkit in Python is used to reduce the image size from 3000*3000 to 500*500 to obtain the color image of agricultural products X. 1 .

[0104] In one example, one-dimensional near infrared spectroscopy data of agricultural products is obtained by the following steps:

[0105] (1) First, wipe the surface of the apple clean and place it at 25 degrees Celsius for 24 hours. (2) Turn on the near-infrared spectrometer and preheat it for 30 minutes. (3) Adjust the intensity of the HL2000 halogen light source to level 5, perform dark spectrum and total reflection spectrum calibration in the OceanView software and set the following parameters: integration time 50ms, take the average of 5 scans as the result, and the sliding average width is 2. (4) Perform spectrum acquisition: When collecting the spectrum, the fiber optic reflection probe is perpendicular to the apple surface and 1 cm away from the apple surface. Select 3 points 120 degrees apart along the equator of the apple to measure the near-infrared reflection spectrum, and the average of the 3 points is used as the near-infrared reflection spectrum measurement value of the apple. (5) The spectrum acquisition range is 900nm-1720nm, and the resolution is 1.60nm. There are a total of 512 sample points, and the exported spectral data is a .txt file, which records in detail the spectrometer sampling parameters, spectral sampling points, and corresponding reflectivity.

[0106] In one example, the process of processing one-dimensional near-infrared spectrum data of agricultural products to obtain two-dimensional near-infrared spectrum data of agricultural products may include:

[0107] (1) First, use the numpy toolkit in Python to read the collected .txt file. After reading, the data format is numpy array format with a dimension of 512.

[0108] (2) Then, the near-infrared spectrum is filtered using Savitzky-Gol ay (SG) filtering: SG filtering is a filtering method based on local polynomial least squares fitting in the time domain. The biggest feature of this method is that it can ensure that the shape of the spectrum remains unchanged while filtering out noise.

[0109] (3) Considering that the size differences between different features of the near-infrared spectrum are too large, which is not conducive to the convergence of subsequent classification model training, maximum and minimum normalization processing is performed after SG filtering, and the final one-dimensional near-infrared spectrum data range is between 0 and 1.

[0110] (4) Since one-dimensional near-infrared spectral data is one-dimensional data, it is difficult to build a deep learning model for one-dimensional data, and convolutional networks have achieved great success in processing two-dimensional image classification tasks, the Gram angle field can be used to convert the one-dimensional near-infrared spectrum into two-dimensional near-infrared spectral data of agricultural products x 2 .

[0111] In step 102, the global feature extractor is a neural network model for extracting global features of an image. The global feature extractor may be a VIT-Base (Vision Transformer) model.

[0112] In one example, the pre-trained Vi T-Base model is used as a global feature extractor for agricultural product color images, as shown in the following formula:

[0113] F 1 =f 1 (x 1 ) Formula (1)

[0114] Among them, X 1 is the color image of agricultural products, f 1 is the pre-trained ViT-Base model, F 1 is the global eigenvector. Figure 2 As shown, the agricultural product color image X 1 Input the VIT model to extract global features and obtain the global feature vector F 1 .

[0115] In step 103, the deep feature extractor is a neural network model for extracting deep features of the image. The deep feature extractor can be a ResNet (Res i dual Networks, ResNets) model.

[0116] In one example, a pre-trained ResNet is used as a deep feature extractor, as shown in the following formula:

[0117] F2 =f 2 (x 1 ) Formula (2)

[0118] Among them, X 1 is the color image of agricultural products, f 2 is the pre-trained ResNet model, F 2 is the deep feature vector. Figure 2 As shown, the agricultural product color image X 1 Input the ResNet model to extract deep features and obtain the deep feature vector F 2 .

[0119] In step 104, a two-dimensional discrete wavelet transform function (2D-Discrete Wavelet Transform, 2D-DWT) is a function used to extract discrete features of an image. Figure 2 As shown, the agricultural product color image X 1 Input 2D-DWT function to extract discrete features and obtain discrete feature vector F 3 .

[0120] In one embodiment, referring to Figure 3 , step 104 comprises:

[0121] Step 301, determining the mean and variance of three channels of the agricultural product color image to obtain the channel mean and channel variance;

[0122] Step 302, binarizing the agricultural product color image to obtain an agricultural product grayscale image;

[0123] Step 303, performing secondary decomposition on the agricultural product grayscale image based on a two-dimensional discrete wavelet transform function to obtain secondary decomposition subbands of the image;

[0124] Step 304, determining the mean, variance and energy of the sub-bands of the secondary decomposition of the image, and obtaining the sub-band mean, sub-band variance and sub-band energy;

[0125] Step 305 , performing feature fusion according to the channel mean, channel variance, sub-band mean, sub-band variance and sub-band energy to obtain a discrete feature vector.

[0126] In one embodiment, step 305 includes: firstly performing vector splicing on the channel mean, channel variance, subband mean, subband variance and subband energy to obtain the agricultural product color splicing vector; and then normalizing the agricultural product color splicing vector to obtain a discrete feature vector. Splicing first and then normalizing can achieve fusion of multiple features while enhancing the association between features, which helps to improve the accuracy of agricultural product detection.

[0127] For example, first, the channel mean of the three channels of the agricultural product color image is obtained by statistics, with a dimension of 3, and the channel variance of the three channels of the RGB image is obtained by statistics. Secondly, the cv2 toolkit in Python is used to realize the color transformation of the agricultural product color image, and the agricultural product color image is converted into the agricultural product grayscale image. After the agricultural product grayscale image is subjected to the second-level decomposition of the two-dimensional discrete wavelet transform, the subbands LH1, HL1, HH1, LH2, HL2 and HH2 represent the detail wavelet coefficients. The mean, variance and energy of subband HH1 and subband HH2 are calculated respectively. Then, the channel mean, channel variance, subband HH1 mean, subband HH1 variance, subband HH1 energy, subband HH2 mean, subband HH2 variance, and subband HH2 energy of the three channels of the agricultural product color image are calculated. These features are spliced ​​together to obtain the agricultural product color splicing vector, and then the maximum and minimum normalization is used to make the vector element size of the agricultural product color splicing vector between 0 and 1, and finally the discrete feature vector F is obtained. 3 .

[0128] The benefit of the embodiment of step 301 to step 305 is that the ability of discrete feature vectors to carry potential information of images in channels and sub-bands can be improved, thereby helping to improve the accuracy of agricultural product detection.

[0129] In step 105, the spectral feature extractor is a neural network model for extracting spectral features of two-dimensional near-infrared spectral data. The spectral feature extractor can be a ResNet (Res idua lNetworks, ResNets) model.

[0130] In one example, a pre-trained ResNet is used as a spectral feature extractor, as shown in the following formula:

[0131] F 4 =f(x 2 ) Formula (3)

[0132] Among them, X 2 is the two-dimensional near-infrared spectral data, f is the pre-trained ResNet model, F 4 is the near infrared spectrum vector. Figure 2 As shown, the two-dimensional near-infrared spectral data X 2 Input the ResNet model for deep feature extraction to obtain the near-infrared spectrum vector F 4 .

[0133] In step 106, according to the near infrared spectrum vector, a near infrared spectrum query vector, a near infrared spectrum key vector and a near infrared spectrum value vector can be determined respectively.

[0134] In one embodiment, step 106 includes: obtaining a near-infrared spectrum query weight matrix, a near-infrared spectrum key weight matrix and a near-infrared spectrum numerical weight matrix; multiplying the near-infrared spectrum query weight matrix and the near-infrared spectrum vector to obtain a near-infrared spectrum query vector; multiplying the near-infrared spectrum key weight matrix and the near-infrared spectrum vector to obtain a near-infrared spectrum key vector; multiplying the near-infrared spectrum numerical weight matrix and the near-infrared spectrum vector to obtain a near-infrared spectrum numerical vector.

[0135] The near-infrared spectrum query weight matrix, the near-infrared spectrum key weight matrix and the near-infrared spectrum numerical weight matrix are all preset weight matrices.

[0136] Specifically, Figure 4 and Figure 5 As shown, the near infrared spectrum query weight matrix W 41 and near-infrared spectral characteristics F 4 Multiply them to get the near infrared spectrum query vector Q 41 , the near infrared spectroscopy key weight matrix W 42 and near-infrared spectral characteristics F 4 Multiply them to get the near-infrared spectrum key vector Q 42 , the near infrared spectrum key numerical matrix W 43 and near-infrared spectral characteristics F 4 Multiply them to get the near-infrared spectrum numerical vector Q 43 .

[0137] In step 107, the first cross attention mechanism refers to a mechanism commonly used in deep learning to enhance the expressive power of the neural network model. When processing sequence data or implementing semantic association tasks, the first cross attention mechanism can help the model capture the association information between different inputs. The first cross attention mechanism of the present embodiment includes a first cross attention function, a first key weight matrix and a first numerical weight matrix, a second key weight matrix and a second numerical weight matrix, and a third key weight matrix and a third numerical weight matrix.

[0138] In one embodiment, step 107 includes:

[0139] A first image key vector is obtained by multiplying the first key weight matrix and the global eigenvector;

[0140] Multiplying the first numerical weight matrix and the global eigenvector to obtain a first image numerical vector;

[0141] Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectrum query vector, the first image key vector and the first image numerical vector to obtain a first cross-attention vector;

[0142] Multiplying the second key weight matrix and the deep feature vector to obtain a second image key vector;

[0143] Multiplying the second numerical weight matrix and the deep feature vector to obtain a second image numerical vector;

[0144] Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectrum query vector, the second image key vector and the second image numerical vector to obtain a second cross-attention vector;

[0145] A third image key vector is obtained by multiplying the third key weight matrix and the discrete feature vector;

[0146] Multiplying the third numerical weight matrix and the discrete eigenvector to obtain a third image numerical vector;

[0147] Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectrum query vector, the third image key vector and the third image numerical vector to obtain a third cross-attention vector;

[0148] The first agricultural product detection vector is obtained by concatenating the first cross-attention vector, the second cross-attention vector and the third cross-attention vector.

[0149] The first criss-cross attention function is as follows:

[0150]

[0151] Among them, Q represents the query vector, K represents the key vector, V represents the numerical vector, and D is the number of dimensions of Q.

[0152] Specifically, Figure 4 and Figure 5 As shown, the process of determining the first agricultural product detection vector may include:

[0153] (1) The first key weight matrix W 12 and the global eigenvector F 1 Multiply them to get the first image key vector K 12 , the first key value matrix W 13 and the global eigenvector F 1 Multiply them to get the first image numerical vector V 13 . The near infrared spectrum query vector Q 41 , the first image key vector K 12 and the first image numerical vector V 13 Substituting into formula (3), we can get the first cross attention vector F A1 .

[0154] (2) The second key weight matrix W 22 and the deep feature vector F 2 Multiply them to get the second image key vector K 22 , the second numerical weight matrix W 23 and the deep feature vector F 2 Multiply them to get the second image numerical vector V 23 . The near infrared spectrum query vector Q 41 , the second image key vector K 22 and the second image numerical vector V 23 Substituting into formula (3), we can get the second cross attention vector F A2 .

[0155] (3) The third key weight matrix W 32 and the discrete eigenvector F 3 Multiply them to get the third image key vector K 22 , the third numerical weight matrix W 33 and the discrete eigenvector F 3 Multiply them to get the third image numerical vector V 33 . The near infrared spectrum query vector Q 41 , the third image key vector K 32 and the third image numerical vector V 33 Substituting into formula (3), we can get the third cross attention vector F A3 .

[0156] (4) The first cross attention vector F A1 , the second cross attention vector F A2 And the third cross attention vector F A3 Splice and get the first agricultural product detection vector F A .

[0157] The benefit of the above embodiment is that the first cross-attention mechanism is used to achieve effective feature fusion between the near-infrared spectral query vector, the global feature vector, the deep feature vector and the discrete feature vector, which can fully reflect the preference of the first agricultural product detection vector for the global feature vector, the deep feature vector and the discrete feature vector.

[0158] In step 108, the second cross attention mechanism is similar to the first cross attention mechanism, and also refers to a mechanism commonly used in deep learning for enhancing the expressive power of a neural network model. When processing sequence data or implementing a semantic association task, the second cross attention mechanism can help the model capture the association information between different inputs. The second cross attention mechanism of the present embodiment includes a second cross attention function, a first query weight matrix, a second query weight matrix, and a third query weight matrix.

[0159] In one embodiment, step 108 includes:

[0160] A first image query vector is obtained by multiplying the first query weight matrix and the global feature vector;

[0161] Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectrum key vector, the near-infrared spectrum numerical vector and the first image query vector to determine a fourth cross-attention vector;

[0162] Multiplying the second query weight matrix and the deep feature vector to obtain a second image query vector;

[0163] Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectrum key vector, the near-infrared spectrum numerical vector and the second image query vector to obtain a fifth cross-attention vector;

[0164] A third image query vector is obtained by multiplying the third query weight matrix and the discrete feature vector;

[0165] Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectrum key vector, the near-infrared spectrum numerical vector and the third image query vector to determine a sixth cross-attention vector;

[0166] The second agricultural product detection vector is obtained by concatenating the fourth cross-attention vector, the fifth cross-attention vector and the sixth cross-attention vector.

[0167] The second criss-cross attention function is as follows:

[0168]

[0169] Among them, Q represents the query vector, K represents the key vector, V represents the numerical vector, and D is the number of dimensions of Q.

[0170] Specifically, Figure 4 and Figure 5 As shown, the process of determining the second agricultural product detection vector may include:

[0171] (1) The first query weight matrix W 11 and the global eigenvector F 1 Multiply them to get the first image query vector K 11 ; Near infrared spectroscopy key vector Q 42 、Near infrared spectrum numerical vector Q 43 And the first image query vector K 11 Substituting into formula (4), we can get the fourth cross attention vector F B1 .

[0172] (2) The second query weight matrix W 21 and the deep feature vector F 2 Multiply them to get the second image query vector K 21 ; Near infrared spectroscopy key vector Q 42 、Near infrared spectrum numerical vector Q 43 and the second image query vector K 21 Substituting into formula (4), we can get the fifth cross attention vector F B2 .

[0173] (3) The third query weight matrix W 31 and the discrete eigenvector F 3 Multiply them to get the third image query vector K 31 ; Near infrared spectroscopy key vector Q 42 、Near infrared spectrum numerical vector Q 43 And the third image query vector K 31 Substituting into formula (4), we can get the sixth cross attention vector F B2 .

[0174] (4) The fourth cross attention vector F B1 , the fifth cross attention vector F B2 And the sixth cross attention vector F B3 Splice and get the second agricultural product detection vector F B .

[0175] The benefit of the above embodiment is that the second cross-attention mechanism is used to achieve effective feature fusion between the near-infrared spectral query vector, the global feature vector, the deep feature vector and the discrete feature vector, which can fully reflect the preference of the second agricultural product detection vector for the near-infrared spectral vector.

[0176] In step 109, a common application of the gated fusion mechanism is to control the information interaction and fusion between different inputs through the gated mechanism. The gated fusion mechanism of this embodiment includes a target splicing weight matrix and a target activation function.

[0177] In one embodiment, step 109 includes:

[0178] splicing the first agricultural product detection vector and the second agricultural product detection vector to obtain a target agricultural product splicing vector;

[0179] The target splicing weight matrix and the target agricultural product splicing vector are multiplied to obtain an initial weight vector;

[0180] According to the target activation function and the initial weight vector, the target weight vector is calculated;

[0181] Performing a dot product on the first agricultural product detection vector and the target weight vector to obtain a first dot product vector;

[0182] Subtracting each element of the target weight vector from 1 to obtain a difference vector, and performing a dot product between the difference vector and the second agricultural product detection vector to obtain a second dot product vector;

[0183] The target fusion vector is obtained by adding the first dot product vector and the second dot product vector.

[0184] Specifically, Figure 6 As shown, the process of determining the target fusion vector may include:

[0185] (1) According to the first agricultural product detection vector F A and the second agricultural product detection vector F B Perform splicing to obtain the target agricultural product splicing vector F AB =[F A : F B ].

[0186] (2) According to the target splicing weight matrix W p and the target agricultural product splicing vector F AB Multiply them to get the initial weight vector F w1 .

[0187] (3) The calculation process of the target weight vector is as follows:

[0188] G=σ(W p [F A :F B ]+b) Formula (5)

[0189] Where G represents the target weight vector, σ(·) represents the target activation function, and b represents the network bias.

[0190] (4) The calculation process of the target fusion vector is as follows:

[0191] F C =G·F A +(1-G)·F B Formula (6)

[0192] Among them, F C Represents the target fusion vector.

[0193] The benefit of the above embodiment is that the target fusion vector is obtained by using the gated fusion mechanism to perform vector fusion, which improves the ability of the target fusion vector to carry the characteristic information of agricultural products and helps to improve the accuracy of agricultural product detection.

[0194] In step 110, the self-attention mechanism is a mechanism for capturing global dependencies within sequence data. In the self-attention mechanism, each input element (such as a feature vector) can interact with other elements in the sequence and learn the correlation between them. The self-attention mechanism of this embodiment includes an agricultural product query weight matrix, an agricultural product key weight matrix, and an agricultural product query value matrix.

[0195] In one embodiment, step 110 includes:

[0196] The agricultural product query vector is obtained by multiplying the agricultural product query weight matrix and the target fusion vector;

[0197] The agricultural product key vector is obtained by multiplying the agricultural product key weight matrix and the target fusion vector;

[0198] The agricultural product query value matrix and the target fusion vector are multiplied to obtain the agricultural product value vector;

[0199] Self-attention calculation is performed based on the agricultural product query vector, agricultural product key vector and agricultural product value vector to obtain the target agricultural product detection vector.

[0200] Specifically, Figure 7 As shown, the process of determining the target agricultural product detection vector may include:

[0201] (1) According to the agricultural product query weight matrix W s1 and the target fusion vector F C Multiply them to get the agricultural product query vector Q s1 .

[0202] (2) According to the agricultural product key weight matrix W s2 and the target fusion vector F C Multiply them to get the agricultural product query vector K s2 .

[0203] (3) According to the agricultural product numerical weight matrix W s3 and the target fusion vector F C Multiply them to get the agricultural product query vector K s3 .

[0204] (4) The self-attention function is as follows:

[0205]

[0206] Among them, Q represents the query vector, K represents the key vector, V represents the numerical vector, and D is the number of dimensions of Q. s1 , agricultural product key vector K s2 and the agricultural product value vector Vs3 Substituting into formula (7), we get the target agricultural product detection vector F final .

[0207] The benefit of the above embodiment is that, on the basis of utilizing the first cross-attention mechanism, the second cross-attention mechanism and the gated fusion mechanism, the self-attention mechanism is further utilized to further explore the correlation between the various vector elements of the target fusion vector, thereby improving the energy of the target agricultural product detection vector in carrying the agricultural product feature information and improving the accuracy of agricultural product detection.

[0208] In step 111, the target agricultural product detection vector can be classified by the classification model to obtain the agricultural product detection classification result. The agricultural product detection classification result can be the origin category. For example, if the agricultural product is apple, after classifying the target agricultural product detection vector corresponding to apple, the origin category is obtained as City A.

[0209] In one embodiment, the classification model is trained by the following steps:

[0210] (1) Collecting agricultural products: In this embodiment, there are four apples from different origins: A, B, C, and D. A total of 324 apples are collected. The number of apples from each origin is shown in Table 1. When purchasing apples, select fruits with uniform maturity, moderate size, no pests or diseases, and no mechanical damage.

[0211] Origin A place B C D place total Number 90 88 98 48 324

[0212] Table 1

[0213] (2) Dataset division: The color images and two-dimensional near-infrared spectral data of agricultural products are obtained to obtain the data set, which is divided into training set and validation set in a 4:1 ratio, where training set: validation set = 259:65.

[0214] (3) Label the training set and validation set, as shown in Table 2

[0215] Origin A place B C D place Label 0 1 2 3

[0216] Table 2

[0217] (4) Determine the classification model: As shown in Table 3, the classification model is a multi-layer perceptron, the activation function is a rectified linear unit ReLU, and it is initialized to xavier_uniform.

[0218] Fully connected layer 1 Dimensions (512,128) Fully connected layer 2 Dimension (128,128) Fully connected layer 3 Dimension (128,64) Fully connected layer 4 Dimension (64,4) Activation Function ReLU initialization xavier_uniform

[0219] (4) Training parameters: the number of training iterations is 10000, the batch size is 256, the optimizer is Adam, the learning rate decreases exponentially (initial 0.005, and decreases to 90% of the original every 2000 iterations), and the loss function is the cross entropy loss function.

[0220] This application uses non-destructive testing technology to achieve efficient identification of agricultural product origins by combining near-infrared spectroscopy and RGB image information while maintaining sample integrity. Compared with traditional destructive methods, non-destructive testing is simple and fast to operate, suitable for large-scale real-time detection, reducing detection costs and reducing dependence on professional operators. By fusing the chemical information provided by near-infrared spectroscopy and the appearance information provided by RGB images, a more comprehensive feature description is obtained, which significantly improves the detection and classification accuracy and enhances the robustness under different environmental conditions. In addition, the introduction of attention mechanisms (including cross-attention mechanisms and self-attention mechanisms) enables the model to focus on the most critical features, thereby improving the effectiveness of feature fusion and prediction accuracy. The intuitive visual information provided by RGB images not only enhances the robustness of detection, but also improves the interpretability of detection results, making it easier for users to understand and apply. Compared with the prior art, these innovations not only overcome the limitations of single modality and traditional fusion methods, but also achieve a deeper level of feature understanding and interaction.

[0221] The embodiment of the present application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the above agricultural product detection method is realized when the program is executed by the processor. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0222] See also Figure 8 , Figure 8 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0223] The processor 801 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0224] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 802, and the processor 801 calls and executes the agricultural product detection method of the embodiment of this application;

[0225] Input / output interface 803, used to implement information input and output;

[0226] The communication interface 804 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WI FI, Bluetooth, etc.);

[0227] A bus 805 that transmits information between the various components of the device (e.g., the processor 801, the memory 802, the input / output interface 803, and the communication interface 804);

[0228] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0229] An embodiment of the present application also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned agricultural product detection method.

[0230] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0231] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0232] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0233] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0234] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0235] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0236] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0237] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0238] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0239] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for detecting agricultural products, characterized in that: The method comprises: Obtain color images of agricultural products and two-dimensional near-infrared spectrum data of agricultural products; Performing global feature extraction on the agricultural product color image by a global feature extractor to obtain a global feature vector; Performing deep feature extraction on the agricultural product color image by a deep feature extractor to obtain a deep feature vector; Based on a two-dimensional discrete wavelet transform function, discrete features are extracted from the agricultural product color image to obtain a discrete feature vector; Extracting spectral features from the two-dimensional near-infrared spectral data of the agricultural product by a spectral feature extractor to obtain a near-infrared spectral vector; Determine a near infrared spectrum query vector, a near infrared spectrum key vector and a near infrared spectrum value vector according to the near infrared spectrum vector; Based on a first cross attention mechanism, attention calculation is performed on the near infrared spectrum query vector, the global feature vector, the deep feature vector and the discrete feature vector to obtain a first agricultural product detection vector; Based on a second cross attention mechanism, attention calculation is performed on the near infrared spectrum key vector, the near infrared spectrum numerical vector, the global feature vector, the deep feature vector and the discrete feature vector to obtain a second agricultural product detection vector; Based on the gated fusion mechanism, the first agricultural product detection vector and the second agricultural product detection vector are fused to obtain a target fusion vector; Based on the self-attention mechanism, attention calculation is performed on the target fusion vector to obtain a target agricultural product detection vector; The target agricultural product detection vector is classified to obtain the agricultural product detection classification result.

2. The method according to claim 1, characterized in that The discrete feature extraction of the agricultural product color image based on the two-dimensional discrete wavelet transform function to obtain a discrete feature vector includes: Determine the mean and variance of three channels of the agricultural product color image to obtain the channel mean and channel variance; Binarizing the agricultural product color image to obtain an agricultural product grayscale image; Performing secondary decomposition on the agricultural product grayscale image based on the two-dimensional discrete wavelet transform function to obtain secondary decomposition subbands of the image; Determine the mean, variance and energy of the secondary decomposition sub-bands of the image to obtain the sub-band mean, sub-band variance and sub-band energy; Feature fusion is performed according to the channel mean, the channel variance, the sub-band mean, the sub-band variance and the sub-band energy to obtain the discrete feature vector.

3. The method according to claim 2, characterized in that The step of performing feature fusion according to the channel mean, the channel variance, the subband mean, the subband variance and the subband energy to obtain the discrete feature vector includes: Performing vector splicing on the channel mean, the channel variance, the subband mean, the subband variance and the subband energy to obtain a color splicing vector of agricultural products; The agricultural product color splicing vector is normalized to obtain the discrete feature vector.

4. The method according to any one of claims 1 to 3, characterized in that: The gated fusion mechanism includes a target splicing weight matrix and a target activation function; The first agricultural product detection vector and the second agricultural product detection vector are fused based on the gated fusion mechanism to obtain a target fusion vector, including: splicing the first agricultural product detection vector and the second agricultural product detection vector to obtain a target agricultural product splicing vector; Multiplying the target splicing weight matrix and the target agricultural product splicing vector to obtain an initial weight vector; Calculate a target weight vector according to the target activation function and the initial weight vector; Performing a dot product on the first agricultural product detection vector and the target weight vector to obtain a first dot product vector; Subtracting each element of the target weight vector from 1 to obtain a difference vector, and performing a dot product between the difference vector and the second agricultural product detection vector to obtain a second dot product vector; A target fusion vector is obtained by adding the first dot product vector and the second dot product vector.

5. The method according to any one of claims 1 to 3, characterized in that: The self-attention mechanism includes an agricultural product query weight matrix, an agricultural product key weight matrix, and an agricultural product query value matrix; The method of performing attention calculation on the target fusion vector based on the self-attention mechanism to obtain the target agricultural product detection vector includes: The agricultural product query vector is obtained by multiplying the agricultural product query weight matrix and the target fusion vector; The agricultural product key vector is obtained by multiplying the agricultural product key weight matrix and the target fusion vector; Multiplying the agricultural product query value matrix and the target fusion vector to obtain an agricultural product value vector; Self-attention calculation is performed based on the agricultural product query vector, the agricultural product key vector and the agricultural product value vector to obtain the target agricultural product detection vector.

6. The method according to any one of claims 1 to 3, characterized in that: The method of determining a near infrared spectrum query vector, a near infrared spectrum key vector and a near infrared spectrum numerical vector according to the near infrared spectrum vector comprises: Obtain near infrared spectrum query weight matrix, near infrared spectrum key weight matrix and near infrared spectrum numerical weight matrix; The near infrared spectrum query vector is obtained by multiplying the near infrared spectrum query weight matrix and the near infrared spectrum vector; The near infrared spectrum key vector is obtained by multiplying the near infrared spectrum key weight matrix and the near infrared spectrum vector; The near-infrared spectrum numerical weight matrix is ​​multiplied by the near-infrared spectrum vector to obtain the near-infrared spectrum numerical vector.

7. The method according to any one of claims 1 to 3, characterized in that: The first cross-attention mechanism includes a first cross-attention function, a first key weight matrix and a first numerical weight matrix, a second key weight matrix and a second numerical weight matrix, and a third key weight matrix and a third numerical weight matrix; The method of performing attention calculation on the near infrared spectrum query vector, the global feature vector, the deep feature vector and the discrete feature vector based on the first cross attention mechanism to obtain a first agricultural product detection vector includes: Obtain a first image key vector by multiplying the first key weight matrix and the global eigenvector; Obtain a first image numerical vector by multiplying the first numerical weight matrix and the global eigenvector; Based on the first cross-attention function, performing cross-attention calculation on the near-infrared spectrum query vector, the first image key vector and the first image numerical vector to obtain a first cross-attention vector; Obtain a second image key vector by multiplying the second key weight matrix and the deep feature vector; Multiplying the second numerical weight matrix and the deep feature vector to obtain a second image numerical vector; Based on the first cross-attention function, performing cross-attention calculation on the near-infrared spectrum query vector, the second image key vector and the second image numerical vector to obtain a second cross-attention vector; Obtain a third image key vector by multiplying the third key weight matrix and the discrete feature vector; Multiplying the third numerical weight matrix and the discrete feature vector to obtain a third image numerical vector; Based on the first cross-attention function, performing cross-attention calculation on the near-infrared spectrum query vector, the third image key vector and the third image numerical vector to obtain a third cross-attention vector; The first agricultural product detection vector is obtained by concatenating the first cross-attention vector, the second cross-attention vector and the third cross-attention vector.

8. The method according to any one of claims 1 to 3, characterized in that: The second crisscross attention mechanism includes a second crisscross attention function, a first query weight matrix, a second query weight matrix, and a third query weight matrix; The method of performing attention calculation on the near infrared spectrum key vector, the near infrared spectrum numerical vector, the global feature vector, the deep feature vector and the discrete feature vector based on the second cross attention mechanism to obtain a second agricultural product detection vector includes: Obtain a first image query vector by multiplying the first query weight matrix and the global feature vector; Based on the second cross-attention function, performing cross-attention calculation on the near-infrared spectrum key vector, the near-infrared spectrum numerical vector and the first image query vector to determine a fourth cross-attention vector; Obtain a second image query vector by multiplying the second query weight matrix and the deep feature vector; Based on the second cross-attention function, performing cross-attention calculation on the near-infrared spectrum key vector, the near-infrared spectrum numerical vector and the second image query vector to obtain a fifth cross-attention vector; Obtain a third image query vector by multiplying the third query weight matrix and the discrete feature vector; Based on the second cross-attention function, performing cross-attention calculation on the near-infrared spectrum key vector, the near-infrared spectrum numerical vector and the third image query vector to determine a sixth cross-attention vector; The second agricultural product detection vector is obtained by splicing the fourth cross-attention vector, the fifth cross-attention vector and the sixth cross-attention vector.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images