Agricultural product testing methods, electronic devices and computer-readable storage media

Abstract

This application provides a method, electronic device, and computer-readable storage medium for agricultural product detection, relating to the field of artificial intelligence technology. The method includes: acquiring color images and two-dimensional near-infrared spectral data of agricultural products; determining global feature vectors, deep feature vectors, and discrete feature vectors based on the color images; determining near-infrared spectral vectors based on the two-dimensional near-infrared spectral data, and then determining near-infrared spectral query vectors, near-infrared spectral key vectors, and near-infrared spectral numerical vectors based on the near-infrared spectral vectors; fusing the above vectors based on a first cross-attention mechanism, a second cross-attention mechanism, a gating fusion mechanism, and a self-attention mechanism to obtain a target agricultural product detection vector; and finally classifying the target agricultural product detection vector to obtain an agricultural product detection classification result. This application improves the accuracy of agricultural product detection.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for detecting agricultural products, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Agricultural products (such as Fuji apples) vary in appearance, taste, and nutritional value between different production areas, resulting in significant differences in quality. Therefore, it is necessary to conduct agricultural product testing to determine the production area or origin of the product.

[0003] Existing technologies primarily employ two major technical approaches: non-destructive testing (NDT) and destructive analysis. On the one hand, while destructive testing can provide accurate data for agricultural product testing, it has some limitations. For example, the testing process not only consumes samples but may also alter certain characteristics of the samples during processing, thus affecting the reliability of the results. On the other hand, although non-destructive testing technologies, especially those based on spectral analysis, offer a safer, faster, and non-destructive testing method, these technologies also have their shortcomings, such as poor stability and lower accuracy. Summary of the Invention

[0004] The main objective of this application is to propose a method for detecting agricultural products, an electronic device, and a computer-readable storage medium, thereby improving the accuracy of agricultural product detection.

[0005] To achieve the above objectives, a first aspect of this application provides a method for detecting agricultural products, the method comprising:

[0006] Acquire color images and two-dimensional near-infrared spectral data of agricultural products;

[0007] The global feature vector is obtained by performing global feature extraction on the color image of the agricultural product using a global feature extractor.

[0008] The deep feature vector is obtained by extracting deep features from the color image of the agricultural product using a deep feature extractor.

[0009] Discrete feature vectors are obtained by performing discrete feature extraction on the color image of the agricultural products based on the two-dimensional discrete wavelet transform function.

[0010] The spectral features of the two-dimensional near-infrared spectral data of the agricultural products are extracted using a spectral feature extractor to obtain near-infrared spectral vectors.

[0011] Based on the near-infrared spectral vector, determine the near-infrared spectral query vector, the near-infrared spectral key vector, and the near-infrared spectral numerical vector;

[0012] Based on the first cross-attention mechanism, attention is calculated on the near-infrared spectral query vector, the global feature vector, the deep feature vector, and the discrete feature vector to obtain the first agricultural product detection vector;

[0013] Based on the second cross-attention mechanism, attention calculation is performed on the near-infrared spectral key vector, the near-infrared spectral numerical vector, the global feature vector, the deep feature vector, and the discrete feature vector to obtain the second agricultural product detection vector;

[0014] Based on the gating fusion mechanism, the first agricultural product detection vector and the second agricultural product detection vector are fused to obtain the target fusion vector;

[0015] Based on the self-attention mechanism, attention calculation is performed on the target fusion vector to obtain the target agricultural product detection vector;

[0016] The detection vectors of the target agricultural products are classified to obtain the agricultural product detection classification results.

[0017] Optionally, the step of extracting discrete features from the color image of the agricultural product based on the two-dimensional discrete wavelet transform function to obtain a discrete feature vector includes:

[0018] The mean and variance of the three channels of the color image of the agricultural product are determined to obtain the channel mean and channel variance;

[0019] The color image of the agricultural product is binarized to obtain a grayscale image of the agricultural product;

[0020] The grayscale image of the agricultural product is decomposed into two levels based on the two-dimensional discrete wavelet transform function to obtain image two-level decomposed sub-bands.

[0021] The mean, variance, and energy of the sub-bands of the image's second-order decomposition are determined to obtain the sub-band mean, sub-band variance, and sub-band energy.

[0022] The discrete feature vector is obtained by fusing the channel mean, the channel variance, the sub-band mean, the sub-band variance, and the sub-band energy.

[0023] Optionally, the step of performing feature fusion based on the channel mean, the channel variance, the sub-band mean, the sub-band variance, and the sub-band energy to obtain the discrete feature vector includes:

[0024] The channel mean, channel variance, sub-band mean, sub-band variance, and sub-band energy are vectorized and concatenated to obtain the agricultural product color splicing vector;

[0025] The discrete feature vector is obtained by normalizing the color splicing vector of the agricultural products.

[0026] Optionally, the gating fusion mechanism includes a target splicing weight matrix and a target activation function;

[0027] The method based on the gated fusion mechanism fuses the first agricultural product detection vector and the second agricultural product detection vector to obtain the target fusion vector, including:

[0028] The first agricultural product detection vector and the second agricultural product detection vector are concatenated to obtain the target agricultural product concatenation vector.

[0029] The initial weight vector is obtained by multiplying the target splicing weight matrix and the target agricultural product splicing vector.

[0030] The target weight vector is calculated based on the target activation function and the initial weight vector.

[0031] The first dot product vector is obtained by performing a dot product between the first agricultural product detection vector and the target weight vector;

[0032] Subtracting each element of the target weight vector from 1 yields a difference vector, and a dot product is performed between the difference vector and the second agricultural product detection vector to obtain a second dot product vector;

[0033] The target fusion vector is obtained by adding the first dot product vector and the second dot product vector.

[0034] Optionally, the self-attention mechanism includes an agricultural product query weight matrix, an agricultural product key weight matrix, and an agricultural product query value matrix;

[0035] The step of performing attention calculation on the target fusion vector based on a self-attention mechanism to obtain the target agricultural product detection vector includes:

[0036] The agricultural product query vector is obtained by multiplying the agricultural product query weight matrix and the target fusion vector.

[0037] The key weight matrix of agricultural products and the target fusion vector are multiplied together to obtain the key vector of agricultural products;

[0038] The agricultural product numerical vector is obtained by multiplying the agricultural product query numerical matrix and the target fusion vector.

[0039] The target agricultural product detection vector is obtained by performing self-attention calculation based on the agricultural product query vector, the agricultural product key vector, and the agricultural product numerical vector.

[0040] Optionally, determining the near-infrared spectral query vector, the near-infrared spectral key vector, and the near-infrared spectral numerical vector based on the near-infrared spectral vector includes:

[0041] Obtain the near-infrared spectral query weight matrix, the near-infrared spectral key weight matrix, and the near-infrared spectral numerical weight matrix;

[0042] The near-infrared spectral query vector is obtained by multiplying the near-infrared spectral query weight matrix and the near-infrared spectral vector.

[0043] The near-infrared spectral key weight matrix and the near-infrared spectral vector are multiplied to obtain the near-infrared spectral key vector;

[0044] The near-infrared spectral numerical vector is obtained by multiplying the near-infrared spectral numerical weight matrix and the near-infrared spectral vector.

[0045] Optionally, the first cross-attention mechanism includes a first cross-attention function, a first key weight matrix and a first numerical weight matrix, a second key weight matrix and a second numerical weight matrix, and a third key weight matrix and a third numerical weight matrix.

[0046] The first agricultural product detection vector is obtained by performing attention calculations on the near-infrared spectral query vector, the global feature vector, the deep feature vector, and the discrete feature vector based on the first cross-attention mechanism, including:

[0047] The first image key vector is obtained by multiplying the first key weight matrix and the global feature vector.

[0048] The first image numerical vector is obtained by multiplying the first numerical weight matrix and the global feature vector.

[0049] Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectral query vector, the first image key vector, and the first image numerical vector to obtain the first cross-attention vector;

[0050] The second image key vector is obtained by multiplying the second key weight matrix and the deep feature vector.

[0051] The second image numerical vector is obtained by multiplying the second numerical weight matrix and the deep feature vector.

[0052] Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectral query vector, the second image key vector, and the second image numerical vector to obtain the second cross-attention vector;

[0053] The third image key vector is obtained by multiplying the third key weight matrix and the discrete feature vector.

[0054] The third image numerical vector is obtained by multiplying the third numerical weight matrix and the discrete feature vector.

[0055] Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectral query vector, the third image key vector, and the third image numerical vector to obtain the third cross-attention vector;

[0056] The first agricultural product detection vector is obtained by concatenating the first cross-attention vector, the second cross-attention vector, and the third cross-attention vector.

[0057] Optionally, the second cross-attention mechanism includes a second cross-attention function, a first query weight matrix, a second query weight matrix, and a third query weight matrix;

[0058] The second agricultural product detection vector is obtained by performing attention calculations on the near-infrared spectral key vector, the near-infrared spectral numerical vector, the global feature vector, the deep feature vector, and the discrete feature vector based on the second cross-attention mechanism, including:

[0059] The first image query vector is obtained by multiplying the first query weight matrix and the global feature vector.

[0060] Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectral key vector, the near-infrared spectral numerical vector, and the first image query vector to determine the fourth cross-attention vector;

[0061] The second image query vector is obtained by multiplying the second query weight matrix and the deep feature vector.

[0062] Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectral key vector, the near-infrared spectral numerical vector, and the second image query vector to obtain the fifth cross-attention vector;

[0063] The third image query vector is obtained by multiplying the third query weight matrix and the discrete feature vector.

[0064] Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectral key vector, the near-infrared spectral numerical vector, and the third image query vector to determine the sixth cross-attention vector;

[0065] The second agricultural product detection vector is obtained by concatenating the fourth cross-attention vector, the fifth cross-attention vector, and the sixth cross-attention vector.

[0066] To achieve the above objectives, a second aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0067] To achieve the above objectives, a third aspect of the present application provides a storage medium, which is a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0068] This application proposes an agricultural product detection method, electronic device, and computer-readable storage medium. It extracts image features from a color image of the agricultural product three times (including global feature extraction, deep feature extraction, and discrete feature extraction) to obtain global feature vectors, deep feature vectors, and discrete feature vectors. It also extracts spectral features from the two-dimensional near-infrared spectral data of the agricultural product to obtain a near-infrared spectral vector. After obtaining these four vectors, a first cross-attention mechanism and a second cross-attention mechanism are introduced to obtain a first agricultural product detection vector and a second agricultural product detection vector, respectively. A gating fusion mechanism is also introduced to fuse the first and second agricultural product detection vectors to obtain a target fused vector. Then, an attention calculation is performed on the target fused vector based on a self-attention mechanism to obtain a target agricultural product detection vector. Finally, the target agricultural product detection vector is classified to obtain the agricultural product detection classification result. This application, by utilizing the combined effects of three image feature extractions, spectral feature extractions, the first and second cross-attention mechanisms, the gating fusion mechanism, and the self-attention mechanism, can achieve effective feature fusion of the near-infrared spectrum and the image domain, obtaining a highly accurate target agricultural product detection vector, thereby obtaining a highly accurate agricultural product detection classification result, thus improving the accuracy of agricultural product detection.

[0069] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0070] Figure 1 This is an optional flowchart of the agricultural product testing method provided in the embodiments of this application;

[0071] Figure 2 This is a general framework diagram of the agricultural product testing method according to the embodiments of this application;

[0072] Figure 3 yes Figure 1 Flowchart for step 104;

[0073] Figure 4 This is a schematic diagram illustrating the determination of the first agricultural product detection vector based on the first cross-attention mechanism;

[0074] Figure 5 This is a schematic diagram illustrating the determination of the second agricultural product detection vector based on the second cross-attention mechanism;

[0075] Figure 6 This is a schematic diagram illustrating the determination of the target fusion vector based on a gating fusion mechanism;

[0076] Figure 7 This is a schematic diagram illustrating the determination of the detection vector for the target agricultural product based on a self-attention mechanism;

[0077] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0079] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0080] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0081] First, let's analyze some of the terms used in this application:

[0082] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. AI attempts to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0083] Backpropagation: The general principle of backpropagation is as follows: The training data is input into the input layer of the neural network, passes through the hidden layer, and finally reaches the output layer of the neural network to output the result; Since there is an error between the output result of the neural network and the actual result, the error between the estimated value and the actual value is calculated, and this error is backpropagated from the output layer to the hidden layer until it reaches the input layer; During the backpropagation process, the values ​​of various parameters are adjusted according to the error; The above process is iterated continuously until convergence.

[0084] Currently, while destructive analysis methods can provide accurate data for tracing the origin of agricultural products, they have significant limitations. (1) First, these methods typically require cutting, pressing, or chemically treating samples (such as apples) to extract the necessary test data. This destructive testing not only consumes the sample but may also alter some of its properties during the sample processing, thus affecting the reliability of the results. (2) Moreover, due to the complexity and time-consuming nature of the required steps, this method is difficult to apply to large-scale or real-time testing. In addition to operational complexity, destructive analysis also requires sophisticated and expensive experimental equipment, which increases the cost of testing. (3) At the same time, it is highly dependent on professional operators, requiring experienced technicians to ensure accurate data collection and analysis. These factors combined restrict the practical application of destructive analysis methods in agricultural production and product circulation.

[0085] On the other hand, although non-destructive testing technologies, especially those based on spectral analysis, offer a safer, faster, and non-destructive testing method, these technologies also have their shortcomings. (1) The accuracy of spectral analysis largely depends on the stability of measurement conditions, such as light source, ambient temperature, and illumination conditions. (2) Spectra mainly provide chemical information about apples, lacking descriptions of their appearance characteristics, resulting in limited information dimensions. (3) Spectral data lacks intuitiveness and interpretability for ordinary users. (4) When dealing with noise, outliers, and other issues, the lack of complementary verification from multiple data sources can affect the reliability of agricultural product traceability.

[0086] To address at least one of the above problems, embodiments of this application propose utilizing the combined effects of three image feature extractions, spectral feature extractions, a first cross-attention mechanism, a second cross-attention mechanism, a gated fusion mechanism, and a self-attention mechanism to achieve effective feature fusion between the near-infrared spectral and image domains. This results in a highly accurate target agricultural product detection vector, leading to a more accurate agricultural product detection and classification result, thus improving the accuracy of agricultural product detection.

[0087] The agricultural product testing method in this application embodiment can be executed by a server alone, by a terminal alone, or by both a terminal and a server. Furthermore, the agricultural product testing method provided in this application embodiment can also be software running on a server. The server can be configured as a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The software can be an application that implements the agricultural product testing method, but is not limited to the above forms.

[0088] This application provides a method for detecting agricultural products, an electronic device, and a computer-readable storage medium, which are specifically described through the following embodiments. First, the method for detecting agricultural products in the embodiments of this application is described.

[0089] See Figure 1 According to one embodiment of this application, the method for detecting agricultural products includes:

[0090] Step 101: Obtain color images and two-dimensional near-infrared spectral data of agricultural products;

[0091] Step 102: Extract global features from the color image of agricultural products using a global feature extractor to obtain a global feature vector;

[0092] Step 103: Perform deep feature extraction on the color image of agricultural products using a deep feature extractor to obtain a deep feature vector;

[0093] Step 104: Extract discrete features from the color image of agricultural products based on the two-dimensional discrete wavelet transform function to obtain discrete feature vectors;

[0094] Step 105: Extract spectral features from the two-dimensional near-infrared spectral data of agricultural products using a spectral feature extractor to obtain near-infrared spectral vectors;

[0095] Step 106: Based on the near-infrared spectral vector, determine the near-infrared spectral query vector, the near-infrared spectral key vector, and the near-infrared spectral numerical vector;

[0096] Step 107: Based on the first cross-attention mechanism, attention is calculated on the near-infrared spectral query vector, global feature vector, deep feature vector, and discrete feature vector to obtain the first agricultural product detection vector;

[0097] Step 108: Based on the second cross-attention mechanism, attention calculation is performed on the near-infrared spectral key vector, near-infrared spectral numerical vector, global feature vector, deep feature vector and the discrete feature vector to obtain the second agricultural product detection vector;

[0098] Step 109: Based on the gating fusion mechanism, the first agricultural product detection vector and the second agricultural product detection vector are fused to obtain the target fused vector;

[0099] Step 110: Based on the self-attention mechanism, perform attention calculation on the target fusion vector to obtain the target agricultural product detection vector;

[0100] Step 111: Classify the detection vector of the target agricultural product to obtain the agricultural product detection classification result.

[0101] Reference Figure 2 , Figure 2 This is a general framework diagram of the agricultural product testing method according to an embodiment of this application.

[0102] Figure 2 On the far left are the color image and two-dimensional near-infrared spectral data of the agricultural product from step 101. The color image of the agricultural product refers to a color image of the agricultural product (such as an apple), also known as an RGB image. This color image can be obtained by capturing images of the agricultural product using a camera, mobile phone, or other image acquisition devices. The two-dimensional near-infrared spectral data of the agricultural product refers to its two-dimensional near-infrared spectral data. This data can be obtained by first detecting the agricultural product using a near-infrared spectrometer to obtain its one-dimensional near-infrared spectral data; then using the Gram angle field (... Figure 2The GAF (Gramian Angual Field) shown here transforms one-dimensional near-infrared spectral data of agricultural products into two-dimensional near-infrared spectral data.

[0103] In one example, the color image of the agricultural product is obtained through the following steps: First, place the apple on a black background. Hold a smartphone 25cm away from the apple and capture images of its top, bottom, and sides. Take one top image, one bottom image, and three side images. Ensure that the background of each image is black and the apple is centered. Each image has a resolution of 3000*3000. Then, use the cv2 toolkit in Python to reduce the image size from 3000*3000 to 500*500 to obtain the color image X1 of the agricultural product.

[0104] In one example, one-dimensional near-infrared spectral data of agricultural products are obtained through the following steps:

[0105] (1) First, wipe the apple surface clean and place it at 25 degrees Celsius for 24 hours. (2) Turn on the near-infrared spectrometer and preheat for 30 minutes. (3) Adjust the intensity of the HL2000 halogen light source to level 5, perform dark spectrum and total reflectance spectrum calibration in OceanView software, and set the following parameters: integration time 50ms, take the average of 5 scans as the result, and the sliding average width is 2. (4) Perform spectrum acquisition: When acquiring the spectrum, the fiber optic reflectance probe is perpendicular to the apple surface and 1cm away from the apple surface. Select 3 points 120 degrees apart along the apple equator to measure the near-infrared reflectance spectrum, and take the average of the 3 points as the measured value of the apple's near-infrared reflectance spectrum. (5) The spectrum acquisition range is 900nm-1720nm, and the resolution is 1.60nm. There are a total of 512 sample points. The exported spectral data is a .txt file, which records the spectrometer sampling parameters, spectral sampling points, and corresponding reflectance in detail.

[0106] In one example, the process of processing one-dimensional near-infrared spectral data of agricultural products to obtain two-dimensional near-infrared spectral data of agricultural products may include:

[0107] (1) First, use the numpy toolkit in Python to read the collected .txt file. After reading, the data format is numpy array with a dimension of 512.

[0108] (2) Next, Savitzky-Golay (SG) filtering is used to filter noise in the near-infrared spectrum: SG filtering is a filtering method based on local polynomial least squares fitting in the time domain. The biggest feature of this method is that it can ensure that the shape of the spectrum remains unchanged while filtering out noise.

[0109] (3) Considering that the large differences in the size of different features in the near-infrared spectrum are not conducive to the convergence of subsequent classification model training, after SG filtering, maximum and minimum normalization processing was performed, and the final one-dimensional near-infrared spectral data ranged from 0 to 1.

[0110] (4) Since one-dimensional near-infrared spectral data is one-dimensional data, it is difficult to build a deep learning model for one-dimensional data. Convolutional networks have achieved great success in processing two-dimensional image classification tasks. Therefore, Gram angle field can be used to convert one-dimensional near-infrared spectrum into two-dimensional near-infrared spectral data of agricultural products x2.

[0111] In step 102, the global feature extractor is a neural network model used to extract global features of the image. The global feature extractor can be a VIT-Base (Vision Transformer) model.

[0112] In one example, a pre-trained Vi T-Base model is used as the global feature extractor for color images of agricultural products, as shown in the following formula:

[0113] F1=f1(x1) Formula (1)

[0114] Where X1 is a color image of agricultural products, f1 is a pre-trained ViT-Base model, and F1 is a global feature vector. Figure 2 As shown, the color image X1 of agricultural products is input into the VIT model for global feature extraction, resulting in a global feature vector F1.

[0115] In step 103, the deep feature extractor is a neural network model used to extract deep features from an image. The deep feature extractor can be a ResNet (Res Dual Networks) model.

[0116] In one example, a pre-trained ResNet is used as a deep feature extractor, as shown in the following formula:

[0117] F2=f2(x1) Formula (2)

[0118] Where X1 is a color image of agricultural products, f2 is a pre-trained ResNet model, and F2 is a deep feature vector. Figure 2 As shown, the color image X1 of agricultural products is input into the ResNet model for deep feature extraction, resulting in a deep feature vector F2.

[0119] In step 104, the two-dimensional discrete wavelet transform (2D-DWT) function is a function used to extract discrete features of an image. For example... Figure 2 As shown, the color image X1 of agricultural products is input into the 2D-DWT function for discrete feature extraction, resulting in a discrete feature vector F3.

[0120] In one embodiment, reference is made to Figure 3 Step 104 includes:

[0121] Step 301: Determine the mean and variance of the three channels of the color image of agricultural products, and obtain the channel mean and channel variance;

[0122] Step 302: Binarize the color image of the agricultural product to obtain a grayscale image of the agricultural product;

[0123] Step 303: Perform a two-dimensional decomposition on the grayscale image of agricultural products based on the two-dimensional discrete wavelet transform function to obtain the image's two-dimensional decomposed sub-bands;

[0124] Step 304: Determine the mean, variance, and energy of the sub-bands of the second-order image decomposition to obtain the sub-band mean, sub-band variance, and sub-band energy;

[0125] Step 305: Perform feature fusion based on channel mean, channel variance, subband mean, subband variance, and subband energy to obtain discrete feature vectors.

[0126] In one embodiment, step 305 includes: first, concatenating the channel mean, channel variance, sub-band mean, sub-band variance, and sub-band energy into a vector concatenation to obtain a color-stitched vector for agricultural products; then, normalizing the color-stitched vector to obtain a discrete feature vector. Concatenating first and then normalizing allows for the fusion of multiple features while enhancing the correlation between features, thus helping to improve the accuracy of agricultural product detection.

[0127] For example, first, the channel mean of the three channels of the agricultural product color image is calculated (dimension 3), and the channel variance of the three channels of the RGB image is also calculated. Next, the color transformation of the agricultural product color image is implemented using the cv2 toolkit in Python, converting it from a color image to a grayscale image. After performing a two-dimensional discrete wavelet transform on the grayscale image, subbands LH1, HL1, HH1, LH2, HL2, and HH2 represent the detail wavelet coefficients. The mean, variance, and energy of subbands HH1 and HH2 are calculated respectively. Then, the channel mean, channel variance, subband HH1 mean, subband HH1 variance, subband HH1 energy, subband HH2 mean, subband HH2 variance, and subband HH2 energy of the agricultural product color image are calculated. These features are concatenated to obtain the agricultural product color spliced ​​vector. Then, min-max normalization is used to ensure that the vector element size of the agricultural product color spliced ​​vector is between 0 and 1, finally obtaining the discrete feature vector F3.

[0128] The benefit of the embodiments of steps 301 to 305 is that they can improve the ability of discrete feature vectors to carry potential information of images in channels and sub-bands, thereby helping to improve the accuracy of agricultural product detection.

[0129] In step 105, the spectral feature extractor is a neural network model used to extract spectral features from two-dimensional near-infrared spectral data. The spectral feature extractor can be a ResNet (Res Identical Networks, ResNets) model.

[0130] In one example, a pre-trained ResNet is used as the spectral feature extractor, as shown in the following formula:

[0131] F4 = f(x2) Formula (3)

[0132] Where X2 represents two-dimensional near-infrared spectral data, f is a pre-trained ResNet model, and F4 is a near-infrared spectral vector. Figure 2 As shown, the two-dimensional near-infrared spectral data X2 is input into the ResNet model for deep feature extraction, resulting in the near-infrared spectral vector F4.

[0133] In step 106, based on the near-infrared spectral vector, the near-infrared spectral query vector, the near-infrared spectral key vector, and the near-infrared spectral numerical vector can be determined respectively.

[0134] In one embodiment, step 106 includes: obtaining a near-infrared spectral query weight matrix, a near-infrared spectral key weight matrix, and a near-infrared spectral numerical weight matrix; multiplying the near-infrared spectral query weight matrix and the near-infrared spectral vector to obtain a near-infrared spectral query vector; multiplying the near-infrared spectral key weight matrix and the near-infrared spectral vector to obtain a near-infrared spectral key vector; and multiplying the near-infrared spectral numerical weight matrix and the near-infrared spectral vector to obtain a near-infrared spectral numerical vector.

[0135] The near-infrared spectroscopy query weight matrix, near-infrared spectroscopy key weight matrix, and near-infrared spectroscopy numerical weight matrix are all preset weight matrices.

[0136] Specifically, such as Figure 4 and Figure 5 As shown, the near-infrared spectral query weight matrix W 41 Multiplying the near-infrared spectral feature F4 yields the near-infrared spectral query vector Q. 41 The key weight matrix W of the near-infrared spectrum 42 Multiplying the near-infrared spectral feature F4 with the near-infrared spectral feature yields the near-infrared spectral key vector Q. 42 The key numerical matrix W of the near-infrared spectrum 43 Multiplying the near-infrared spectral feature F4 by the near-infrared spectral feature yields the near-infrared spectral numerical vector Q. 43 .

[0137] In step 107, the first cross-attention mechanism is a commonly used mechanism in deep learning to enhance the expressive power of neural network models. When processing sequential data or performing semantic association tasks, the first cross-attention mechanism can help the model capture the correlation information between different inputs. In this embodiment, the first cross-attention mechanism includes a first cross-attention function, a first key weight matrix and a first numerical weight matrix, a second key weight matrix and a second numerical weight matrix, and a third key weight matrix and a third numerical weight matrix.

[0138] In one embodiment, step 107 includes:

[0139] The first image key vector is obtained by multiplying the first key weight matrix and the global feature vector.

[0140] The first image numerical vector is obtained by multiplying the first numerical weight matrix and the global feature vector.

[0141] Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectral query vector, the first image key vector, and the first image numerical vector to obtain the first cross-attention vector;

[0142] The second key vector of the image is obtained by multiplying the second key weight matrix and the deep feature vector.

[0143] The second image numerical vector is obtained by multiplying the second numerical weight matrix and the deep feature vector.

[0144] Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectral query vector, the second image key vector, and the second image numerical vector to obtain the second cross-attention vector;

[0145] The third key vector of the image is obtained by multiplying the third key weight matrix and the discrete feature vector.

[0146] The third image numerical vector is obtained by multiplying the third numerical weight matrix and the discrete feature vector.

[0147] Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectral query vector, the third image key vector, and the third image numerical vector to obtain the third cross-attention vector;

[0148] The first agricultural product detection vector is obtained by concatenating the first cross-attention vector, the second cross-attention vector, and the third cross-attention vector.

[0149] The first cross-attention function is shown below:

[0150]

[0151] Where Q represents the query vector, K represents the key vector, V represents the numerical vector, and D is the number of dimensions of Q.

[0152] Specifically, such as Figure 4 and Figure 5 As shown, the process of determining the first agricultural product detection vector may include:

[0153] (1) The first key weight matrix W 12 Multiplying it with the global feature vector F1 yields the first image key vector K. 12 The first key numerical matrix W 13 Multiplying it by the global feature vector F1 yields the first image numerical vector V. 13 Near-infrared spectral query vector Q 41 First image key vector K 12 and the first image numerical vector V 13 Substituting into formula (3), we can obtain the first cross-attention vector F. A1 .

[0154] (2) The second key weight matrix W 22Multiplying the deep feature vector F2 with the second image key vector K yields the second image key vector K. 22 The second numerical weight matrix W 23 Multiplying the deep feature vector F2 with the image yields the second image numerical vector V. 23 Near-infrared spectral query vector Q 41 Second image key vector K 22 and the second image numerical vector V 23 Substituting into formula (3), we can obtain the second cross-attention vector F. A2 .

[0155] (3) The third key weight matrix W 32 Multiplying the discrete feature vector F3 with the third image key vector K yields the third image key vector K. 22 The third numerical weight matrix W 33 Multiplying the discrete feature vector F3 by the third image numerical vector V yields the third image numerical vector V. 33 Near-infrared spectral query vector Q 41 Key vector K of the third image 32 and the third image numerical vector V 33 Substituting into formula (3), we can obtain the third cross-attention vector F. A3 .

[0156] (4) The first cross attention vector F A1 The second cross-attention vector F A2 and the third cross-attention vector F A3 By splicing the data, we obtain the first agricultural product detection vector F. A .

[0157] The advantage of the above embodiments is that the first cross-attention mechanism is used to achieve effective feature fusion among near-infrared spectral query vector, global feature vector, deep feature vector and discrete feature vector, which can fully reflect the degree of preference of the first agricultural product detection vector for global feature vector, deep feature vector and discrete feature vector.

[0158] In step 108, the second cross-attention mechanism, similar to the first cross-attention mechanism, is also a commonly used mechanism in deep learning to enhance the expressive power of neural network models. When processing sequential data or implementing semantic association tasks, the second cross-attention mechanism can help the model capture the correlation information between different inputs. In this embodiment, the second cross-attention mechanism includes a second cross-attention function, a first query weight matrix, a second query weight matrix, and a third query weight matrix.

[0159] In one embodiment, step 108 includes:

[0160] The first image query vector is obtained by multiplying the first query weight matrix and the global feature vector.

[0161] Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectral key vector, the near-infrared spectral numerical vector, and the first image query vector to determine the fourth cross-attention vector;

[0162] The second image query vector is obtained by multiplying the second query weight matrix and the deep feature vector.

[0163] Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectral key vector, the near-infrared spectral numerical vector, and the second image query vector to obtain the fifth cross-attention vector;

[0164] The third image query vector is obtained by multiplying the third query weight matrix and the discrete feature vector.

[0165] Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectral key vector, the near-infrared spectral numerical vector, and the third image query vector to determine the sixth cross-attention vector;

[0166] The second agricultural product detection vector is obtained by concatenating the fourth, fifth, and sixth cross-attention vectors.

[0167] The second cross-attention function is shown below:

[0168]

[0169] Where Q represents the query vector, K represents the key vector, V represents the numerical vector, and D is the number of dimensions of Q.

[0170] Specifically, such as Figure 4 and Figure 5 As shown, the process of determining the second agricultural product detection vector may include:

[0171] (1) The first query weight matrix W 11 Multiplying it by the global feature vector F1 yields the first image query vector K. 11 Near-infrared spectral key vector Q 42 Near-infrared spectral numerical vector Q 43 and the first image query vector K 11 Substituting into formula (4), we can obtain the fourth cross-attention vector F. B1 .

[0172] (2) The second query weight matrix W 21 Multiplying the deep feature vector F2 with the deep feature vector yields the second image query vector K. 21Near-infrared spectral key vector Q 42 Near-infrared spectral numerical vector Q 43 and the second image query vector K 21 Substituting into formula (4), we can obtain the fifth cross-attention vector F. B2 .

[0173] (3) The third query weight matrix W 31 Multiplying the discrete feature vector F3 with the third image query vector K yields the third image query vector K. 31 Near-infrared spectral key vector Q 42 Near-infrared spectral numerical vector Q 43 and the third image query vector K 31 Substituting into formula (4), we can obtain the sixth cross-attention vector F. B2 .

[0174] (4) The fourth cross attention vector F B1 The fifth cross-attention vector F B2 and the sixth cross attention vector F B3 By splicing the data, we obtain the second agricultural product detection vector F. B .

[0175] The advantage of the above embodiments is that the second cross-attention mechanism is used to achieve effective feature fusion among the near-infrared spectral query vector, global feature vector, deep feature vector and discrete feature vector, which can fully reflect the degree of preference of the second agricultural product detection vector for the near-infrared spectral vector.

[0176] In step 109, a common application of gating fusion mechanisms is to control the information interaction and fusion between different inputs. The gating fusion mechanism in this embodiment includes a target concatenation weight matrix and a target activation function.

[0177] In one embodiment, step 109 includes:

[0178] The target agricultural product splicing vector is obtained by concatenating the first agricultural product detection vector and the second agricultural product detection vector.

[0179] The initial weight vector is obtained by multiplying the target splicing weight matrix and the target agricultural product splicing vector.

[0180] The target weight vector is calculated based on the target activation function and the initial weight vector.

[0181] The first dot product vector is obtained by multiplying the first agricultural product detection vector and the target weight vector.

[0182] Subtract each element of the target weight vector from 1 to obtain the difference vector, and multiply the difference vector by the second agricultural product detection vector to obtain the second dot product vector;

[0183] The target fusion vector is obtained by adding the first dot product vector and the second dot product vector.

[0184] Specifically, such as Figure 6 As shown, the process of determining the target fusion vector may include:

[0185] (1) Based on the first agricultural product detection vector F A Second agricultural product detection vector F B By splicing the vectors, we obtain the spliced ​​vector F of the target agricultural product. AB =[F A :F B ].

[0186] (2) Based on the target splicing weight matrix W p splicing vector F with the target agricultural product AB Multiplying them together yields the initial weight vector F. w1 .

[0187] (3) The calculation process of the target weight vector is as follows:

[0188] G=σ(W p [F A :F B Formula (5) + b)

[0189] Where G represents the target weight vector, σ(·) represents the target activation function, and b represents the network bias.

[0190] (4) The calculation process of the target fusion vector is as follows:

[0191] F C =G·F A +(1-G)·F B Formula (6)

[0192] Among them, F C This represents the target fusion vector.

[0193] The advantage of the above embodiments is that by using a gating fusion mechanism to perform vector fusion to obtain a target fusion vector, the ability of the target fusion vector to carry the feature information of agricultural products is improved, which helps to improve the accuracy of agricultural product detection.

[0194] In step 110, self-attention is a mechanism used to capture global dependencies within sequence data. In self-attention, each input element (such as a feature vector) can interact with other elements in the sequence and learn the correlations between them. The self-attention mechanism in this embodiment includes an agricultural product query weight matrix, an agricultural product key weight matrix, and an agricultural product query numerical matrix.

[0195] In one embodiment, step 110 includes:

[0196] The agricultural product query vector is obtained by multiplying the agricultural product query weight matrix and the target fusion vector.

[0197] The key vector of agricultural products is obtained by multiplying the key weight matrix of agricultural products and the target fusion vector.

[0198] The agricultural product numerical vector is obtained by multiplying the agricultural product query numerical matrix and the target fusion vector.

[0199] The target agricultural product detection vector is obtained by performing self-attention calculation based on the agricultural product query vector, agricultural product key vector, and agricultural product numerical vector.

[0200] Specifically, such as Figure 7 As shown, the process of determining the detection vector for the target agricultural product may include:

[0201] (1) Based on the agricultural product query weight matrix W s1 Filtration vector F with target C Multiplying them together yields the agricultural product query vector Q. s1 .

[0202] (2) Based on the key weight matrix W of agricultural products s2 Filtration vector F with target C Multiplying them together yields the agricultural product query vector K. s2 .

[0203] (3) Based on the numerical weight matrix W of agricultural products s3 Filtration vector F with target C Multiplying them together yields the agricultural product query vector K. s3 .

[0204] (4) The self-attention function is shown below:

[0205]

[0206] Where Q represents the query vector, K represents the key vector, V represents the numerical vector, and D is the number of dimensions in Q. The agricultural product query vector Q... s1 Key vector K for agricultural products s2 and agricultural product numerical vector Vs3 Substituting into formula (7), we obtain the target agricultural product detection vector F. final .

[0207] The advantage of the above embodiments is that, based on the first cross-attention mechanism, the second cross-attention mechanism, and the gated fusion mechanism, the self-attention mechanism is further used to explore the correlation between the vector elements of the target fusion vector, thereby increasing the energy of the target agricultural product detection vector to carry agricultural product feature information and improving the accuracy of agricultural product detection.

[0208] In step 111, the target agricultural product detection vector can be classified using a classification model to obtain the agricultural product detection classification result. The agricultural product detection classification result can be the place of origin category. For example, if the agricultural product is apples, after classifying the target agricultural product detection vector corresponding to apples, the place of origin category is obtained as City A.

[0209] In one embodiment, the classification model is trained through the following steps:

[0210] (1) Collection of agricultural products: This embodiment has apples from four different origins: A, B, C and D. A total of 324 apples were collected, and the number of apples from each origin is shown in Table 1. When purchasing apples, select fruits that are uniformly ripe, of moderate size, free from pests and diseases, and free from mechanical damage.

[0211] Origin A place Location B C location D location total Number 90 88 98 48 324

[0212] Table 1

[0213] (2) Dataset partitioning: Obtain color images of agricultural products and two-dimensional near-infrared spectral data of agricultural products to obtain the dataset. Divide the dataset into training set and validation set according to a 4:1 ratio, where training set: validation set = 259:65.

[0214] (3) Label the training set and validation set as shown in Table 2.

[0215] Origin A place Location B C location D location Label 0 1 2 3

[0216] Table 2

[0217] (4) Determine the classification model: As shown in Table 3, the classification model is a multilayer perceptron, the activation function is the linear correction unit ReLU, and it is initialized as xavier_uniform.

[0218] Fully connected layer 1 Dimensions (512, 128) Fully connected layer 2 Dimension (128, 128) Fully connected layer 3 Dimensions (128, 64) Fully connected layer 4 Dimension (64,4) Activation function ReLU initialization xavier_uniform

[0219] (4) Training parameters: The number of training iterations is 10,000, the batch size is 256, the optimizer is Adam, the learning rate decreases exponentially (initially 0.005, and then decreases to 90% of the original every 2,000 iterations), and the loss function is the cross-entropy loss function.

[0220] This application utilizes non-destructive testing (NDT) technology to achieve efficient agricultural product origin identification by combining near-infrared spectroscopy and RGB image information while maintaining sample integrity. Compared with traditional destructive methods, NDT is simple and fast to operate, suitable for large-scale real-time detection, reduces detection costs, and lessens reliance on professional operators. By fusing chemical information provided by near-infrared spectroscopy and appearance information provided by RGB images, a more comprehensive feature description is obtained, significantly improving detection and classification accuracy and enhancing robustness under different environmental conditions. Furthermore, the introduction of attention mechanisms (including cross-attention and self-attention mechanisms) enables the model to focus on the most critical features, thereby improving the effectiveness of feature fusion and prediction accuracy. The intuitive visual information provided by RGB images not only enhances detection robustness but also improves the interpretability of detection results, facilitating user understanding and application. These innovations, compared to existing technologies, not only overcome the limitations of single-modality and traditional fusion methods but also achieve deeper feature understanding and interaction.

[0221] This application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the above-described agricultural product detection method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0222] Please see Figure 8 , Figure 8 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0223] The processor 801 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0224] The memory 802 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called and executed by the processor 801 to execute the agricultural product detection method of the embodiments of this application.

[0225] The 803 input / output interface is used to implement information input and output.

[0226] The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0227] Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804);

[0228] The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0229] This application also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, which can be executed by one or more processors to implement the above-mentioned agricultural product detection method.

[0230] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0231] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0232] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0233] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0234] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0235] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0236] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0237] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0238] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0239] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for detecting agricultural products, characterized in that, The method includes: Acquire color images and two-dimensional near-infrared spectral data of agricultural products; The global feature vector is obtained by performing global feature extraction on the color image of the agricultural product using a global feature extractor. The deep feature vector is obtained by extracting deep features from the color image of the agricultural product using a deep feature extractor. Discrete feature vectors are obtained by performing discrete feature extraction on the color image of the agricultural products based on the two-dimensional discrete wavelet transform function. The spectral features of the two-dimensional near-infrared spectral data of the agricultural products are extracted using a spectral feature extractor to obtain near-infrared spectral vectors. Based on the near-infrared spectral vector, determine the near-infrared spectral query vector, the near-infrared spectral key vector, and the near-infrared spectral numerical vector; Based on the first cross-attention mechanism, attention is calculated on the near-infrared spectral query vector, the global feature vector, the deep feature vector, and the discrete feature vector to obtain the first agricultural product detection vector; Based on the second cross-attention mechanism, attention calculation is performed on the near-infrared spectral key vector, the near-infrared spectral numerical vector, the global feature vector, the deep feature vector, and the discrete feature vector to obtain the second agricultural product detection vector; Based on the gating fusion mechanism, the first agricultural product detection vector and the second agricultural product detection vector are fused to obtain the target fusion vector; Based on the self-attention mechanism, attention calculation is performed on the target fusion vector to obtain the target agricultural product detection vector; The detection vectors of the target agricultural products are classified to obtain the agricultural product detection classification results.

2. The method according to claim 1, characterized in that, The discrete feature extraction of the agricultural product color image based on the two-dimensional discrete wavelet transform function to obtain a discrete feature vector includes: The mean and variance of the three channels of the color image of the agricultural product are determined to obtain the channel mean and channel variance; The color image of the agricultural product is binarized to obtain a grayscale image of the agricultural product; The grayscale image of the agricultural product is decomposed into two levels based on the two-dimensional discrete wavelet transform function to obtain image two-level decomposed sub-bands. The mean, variance, and energy of the sub-bands of the image's second-order decomposition are determined to obtain the sub-band mean, sub-band variance, and sub-band energy. The discrete feature vector is obtained by fusing features based on the channel mean, channel variance, sub-band mean, sub-band variance, and sub-band energy.

3. The method according to claim 2, characterized in that, The step of fusing features based on the channel mean, channel variance, sub-band mean, sub-band variance, and sub-band energy to obtain the discrete feature vector includes: The channel mean, channel variance, sub-band mean, sub-band variance, and sub-band energy are vectorized and concatenated to obtain the agricultural product color splicing vector; The discrete feature vector is obtained by normalizing the color splicing vector of the agricultural products.

4. The method according to any one of claims 1 to 3, characterized in that, The gated fusion mechanism includes a target splicing weight matrix and a target activation function; The method based on the gated fusion mechanism fuses the first agricultural product detection vector and the second agricultural product detection vector to obtain the target fusion vector, including: The first agricultural product detection vector and the second agricultural product detection vector are concatenated to obtain the target agricultural product concatenation vector. The initial weight vector is obtained by multiplying the target splicing weight matrix and the target agricultural product splicing vector. The target weight vector is calculated based on the target activation function and the initial weight vector. The first dot product vector is obtained by performing a dot product between the first agricultural product detection vector and the target weight vector; Subtracting each element of the target weight vector from 1 yields a difference vector, and a dot product is performed between the difference vector and the second agricultural product detection vector to obtain a second dot product vector; The target fusion vector is obtained by adding the first dot product vector and the second dot product vector.

5. The method according to any one of claims 1 to 3, characterized in that, The self-attention mechanism includes an agricultural product query weight matrix, an agricultural product key weight matrix, and an agricultural product query numerical matrix. The step of performing attention calculation on the target fusion vector based on a self-attention mechanism to obtain the target agricultural product detection vector includes: The agricultural product query vector is obtained by multiplying the agricultural product query weight matrix and the target fusion vector. The key weight matrix of agricultural products and the target fusion vector are multiplied together to obtain the key vector of agricultural products; The agricultural product numerical vector is obtained by multiplying the agricultural product query numerical matrix and the target fusion vector. The target agricultural product detection vector is obtained by performing self-attention calculation based on the agricultural product query vector, the agricultural product key vector, and the agricultural product numerical vector.

6. The method according to any one of claims 1 to 3, characterized in that, The step of determining the near-infrared spectral query vector, the near-infrared spectral key vector, and the near-infrared spectral numerical vector based on the near-infrared spectral vector includes: Obtain the near-infrared spectral query weight matrix, the near-infrared spectral key weight matrix, and the near-infrared spectral numerical weight matrix; The near-infrared spectral query vector is obtained by multiplying the near-infrared spectral query weight matrix and the near-infrared spectral vector. The near-infrared spectral key weight matrix and the near-infrared spectral vector are multiplied to obtain the near-infrared spectral key vector; The near-infrared spectral numerical vector is obtained by multiplying the near-infrared spectral numerical weight matrix and the near-infrared spectral vector.

7. The method according to any one of claims 1 to 3, characterized in that, The first cross-attention mechanism includes a first cross-attention function, a first key weight matrix and a first numerical weight matrix, a second key weight matrix and a second numerical weight matrix, and a third key weight matrix and a third numerical weight matrix; The first agricultural product detection vector is obtained by performing attention calculations on the near-infrared spectral query vector, the global feature vector, the deep feature vector, and the discrete feature vector based on the first cross-attention mechanism, including: The first image key vector is obtained by multiplying the first key weight matrix and the global feature vector. The first image numerical vector is obtained by multiplying the first numerical weight matrix and the global feature vector. Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectral query vector, the first image key vector, and the first image numerical vector to obtain the first cross-attention vector; The second image key vector is obtained by multiplying the second key weight matrix and the deep feature vector. The second image numerical vector is obtained by multiplying the second numerical weight matrix and the deep feature vector. Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectral query vector, the second image key vector, and the second image numerical vector to obtain the second cross-attention vector; The third image key vector is obtained by multiplying the third key weight matrix and the discrete feature vector. The third image numerical vector is obtained by multiplying the third numerical weight matrix and the discrete feature vector. Based on the first cross-attention function, cross-attention calculation is performed on the near-infrared spectral query vector, the third image key vector, and the third image numerical vector to obtain the third cross-attention vector; The first agricultural product detection vector is obtained by concatenating the first cross-attention vector, the second cross-attention vector, and the third cross-attention vector.

8. The method according to any one of claims 1 to 3, characterized in that, The second cross-attention mechanism includes a second cross-attention function, a first query weight matrix, a second query weight matrix, and a third query weight matrix; The second agricultural product detection vector is obtained by performing attention calculations on the near-infrared spectral key vector, the near-infrared spectral numerical vector, the global feature vector, the deep feature vector, and the discrete feature vector based on the second cross-attention mechanism, including: The first image query vector is obtained by multiplying the first query weight matrix and the global feature vector. Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectral key vector, the near-infrared spectral numerical vector, and the first image query vector to determine the fourth cross-attention vector; The second image query vector is obtained by multiplying the second query weight matrix and the deep feature vector. Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectral key vector, the near-infrared spectral numerical vector, and the second image query vector to obtain the fifth cross-attention vector; The third image query vector is obtained by multiplying the third query weight matrix and the discrete feature vector. Based on the second cross-attention function, cross-attention calculation is performed on the near-infrared spectral key vector, the near-infrared spectral numerical vector, and the third image query vector to determine the sixth cross-attention vector; The second agricultural product detection vector is obtained by concatenating the fourth cross-attention vector, the fifth cross-attention vector, and the sixth cross-attention vector.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images