Fresh corn appearance detection method based on image fusion and deep learning

Abstract

The invention discloses a fresh corn appearance detection method based on image fusion and deep learning, and belongs to the technical field of image recognition. The method comprises the following steps: acquiring an RGB image and an NIR image of the appearance of the fresh corn, and fusing the RGB image and the NIR image; and detecting the fused image by using an HMM-GELAN detection model to obtain a fresh corn appearance detection result. According to the method, feature extraction and appearance detection are performed by using the fused image, so that the overall detection performance of the network can be effectively improved; in addition, the first RepNCSPELAN4 module in the backbone network is replaced by the HAM module, so that the detection precision of the weak target is effectively improved; and a PConv module and an MSSA module are further introduced, so that the detection precision and the detection efficiency can be further improved. Experiments prove that the fresh corn appearance detection method based on image fusion and deep learning can complete high-precision detection of the appearance of the fresh corn on the premise of few parameters.

Description

Technical Field

[0001] This invention relates to a method for detecting the appearance of fresh corn based on image fusion and deep learning, belonging to the field of image recognition technology. Background Technology

[0002] Sweet corn is young corn harvested during its milk stage. It is tender, sweet, crisp, and juicy, making it very popular with consumers. The appearance quality of the corn is an important factor for consumers when choosing sweet corn. Detecting corn with appearance problems and corn with incompletely removed husks is a crucial step in the processing of sweet corn.

[0003] Currently, most enterprises use manual visual inspection methods, which have many drawbacks: 1. Low inspection efficiency; sorting results are easily affected by human subjectivity and prone to missed inspections; 2. High workload and long hours, causing visual fatigue for inspectors; 3. Processing lines require multiple workers, resulting in high long-term labor costs. Therefore, adopting accurate and efficient non-destructive testing methods is a favorable way to reduce costs and increase efficiency in fresh corn processing.

[0004] In recent years, with the rapid development of image processing and deep learning technologies, scholars at home and abroad have gradually begun to use visual inspection and deep learning methods to detect corn quality. Gao Xinhao et al. solved the entropy values ​​of texture features in the horizontal, vertical, and diagonal directions of corn and combined them with weight criteria to achieve high-precision classification of fresh corn of different varieties, sizes, and degrees of damage (Gao Xinhao et al. Design and Experiment of Fresh Corn Quality Detection Classifier Based on Machine Vision. Transactions of the Chinese Society of Agricultural Engineering. 2016, Vol. 32 (Issue 1), pp. 298-303). Li Qi et al. used SVM to identify four abnormal corn ears: discoloration, missing kernels, insect infestation, and disordered kernels, further improving the identification types and accuracy of corn appearance quality (Li Qi et al. Classification and Identification Method of Abnormal Corn Ears Based on Color and Texture Features. Jiangsu Journal of Agricultural Sciences. 2020, Vol. 36 (Issue 1), pp. 24-31). Liu Guorong et al. improved the Ghost network to achieve efficient identification of five types of ears: normal, moldy, discolored, missing kernels, and disordered kernels (Liu Guorong et al. Lightweight Maize Ear Quality Identification Based on Spatial Information Augmentation. *Chinese Cereals and Oils Journal*. 2024, Vol. 39 (No. 5), pp. 165-171). Although deep learning methods have made significant progress in maize appearance detection tasks, existing research focuses more on dry maize detection, and it is difficult to balance detection accuracy and speed. Secondly, the appearance characteristics of dry maize differ significantly from those of fresh maize, making it difficult to guarantee efficient detection of the appearance of fresh maize.

[0005] Patent CN119399564B discloses "A method for detecting corn ear damage based on deep learning and embedded systems," which acquires real-time image data of the ear collection box inside the harvester and obtains damage classification results from the image data through a pre-trained deep learning model. However, the damage classification results predicted by this method only show the damaged and undamaged areas of all corn, making it difficult to obtain the damage status of each individual corn ear. Furthermore, in actual production, there are many types of corn defects that need to be detected, and relying solely on breakage is insufficient to meet practical needs. In summary, current methods for detecting the appearance of fresh corn are insufficient to meet the detection requirements of multiple defects, and it is difficult to balance detection speed and accuracy. Summary of the Invention

[0006] To improve the accuracy and speed of fresh corn appearance detection, and to meet the need for detecting various defects, this invention provides a fresh corn appearance detection method based on image fusion and deep learning. The technical solution is as follows:

[0007] The first objective of this invention is to provide a method for detecting the appearance of fresh corn based on image fusion and deep learning, the method comprising:

[0008] Step 1: Acquire RGB and NIR images of the appearance of fresh corn and fuse them;

[0009] Step 2: Use the HPM-GELAN detection model to detect the fused image and obtain the appearance detection results of fresh corn;

[0010] The HPM-GELAN detection model is an improvement on the GELAN-t network, replacing the first RepNCSPELAN4 module in the GELAN-t network with a hierarchical attention module HAM.

[0011] Optionally, the HPM-GELAN detection model is an improvement based on the GELAN-t network, and the improvement also includes replacing the original basic convolution AConv with the windmill convolution module PConv.

[0012] Optionally, the HPM-GELAN detection model is an improvement based on the GELAN-t network, and the improvement further includes adding a multi-scale self-attention module (MSSA) before each of the three detection heads of the GELAN-t model.

[0013] Optionally, the hierarchical attention module HAM uses the wavelet attention fusion module WAFM for feature extraction and aggregation. The WAFM module obtains information from each layer through two branches: wavelet convolution and strip convolution. The feature extraction process includes:

[0014] Step 21: Perform pointwise convolution on the input fresh corn feature map;

[0015] Step 22: Perform channel-by-channel separation on the feature map obtained in step 21;

[0016] Step 23: Merge the feature maps of odd-numbered sequences and then sequentially pass them through wavelet transform, standard convolution, and inverse wavelet transform to obtain the first fresh corn feature map. Merge the feature maps of even-numbered sequences and then sequentially pass them through average pooling, pointwise convolution, and two strip convolutions in the horizontal and vertical directions to obtain the attention factor map of the fresh corn feature map. Multiply the attention factor map with the input fresh corn feature map of this path to obtain the second fresh corn feature map after attention weighting.

[0017] Step 24: Combine the first and second fresh corn feature images;

[0018] Step 25: Add the feature map obtained in step 24 to the original input feature map and then perform feature fusion through pointwise convolution.

[0019] Optionally, the processing flow of the multi-scale self-attention module (MSSA) includes:

[0020] Step 31: Divide the input feature map into two equal parts. Both parts are processed by the same convolutional group consisting of three dilated convolutions of different sizes to obtain the features of fresh corn at multiple scales.

[0021] Step 32: Fuse the two feature sets using pointwise convolution;

[0022] Step 33: The fresh corn feature map containing multi-scale information is obtained by sequentially performing average pooling, fully connected layer, and channel expansion operations;

[0023] Step 34: Add the fresh corn feature map obtained in step 33 to the original input feature map to output the scale-enhanced fresh corn feature map.

[0024] Optionally, the formula for fusing the RGB image and the NIR image in step 1 is as follows:

[0025] R channel (x) i ,y i Pixel value I at ) R (x i ,y i )=(1-α)·I RGB-R (x i ,y i )+α·I NIR (x i ,y i )

[0026] G channel (x) i ,y i Pixel value I at ) G (x i ,y i )=(1-α)·I RGB-G (x i ,y i )+α·I NIR (x i ,y i )

[0027] Channel B (x) i ,y i Pixel value I at ) B (x i ,y i )=(1-α)·I RGB-B (x i ,y i )+α·I NIR (x i ,y i )

[0028] Wherein, 1-α and α are the weight proportions of RGB image and NIR image in the pixel values ​​of each point in each channel, respectively, and the value of α ranges from [0,1].

[0029] Optionally, the types of appearance of the fresh corn include: mechanical nibbling, impact damage, incomplete removal of husks, missing kernels, and insect infestation.

[0030] A second objective of this invention is to provide a fresh corn appearance detection system based on image fusion and deep learning, the system being used to implement the fresh corn appearance detection method based on image fusion and deep learning as described in any of the preceding claims, the system comprising:

[0031] The image acquisition module is configured to acquire RGB and NIR images of the appearance of fresh corn.

[0032] An image fusion module is configured to fuse the RGB image and the NIR image;

[0033] The detection module is configured to use the HPM-GELAN detection model to detect the fused image and obtain the appearance detection results of fresh corn.

[0034] The third objective of this invention is to provide a fresh corn appearance detection device based on image fusion and deep learning, including a memory and a processor;

[0035] The memory is used to store computer programs;

[0036] The processor is configured to, when executing the computer program, implement the image fusion and deep learning-based fresh corn appearance detection method as described above.

[0037] A fourth objective of this invention is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image fusion and deep learning-based fresh corn appearance detection method as described in any of the preceding claims.

[0038] The beneficial effects of this invention are:

[0039] The fresh corn appearance detection method of the present invention uses a fused image of RGB and NIR. Experimental results show that using the fused image for feature extraction and appearance detection can effectively improve the overall detection performance of the network. In addition, the present invention replaces the first RepNCSPELAN4 module in the backbone network with the HPM module, which effectively improves the network's ability to extract and preserve weak target information in the feature extraction stage, thereby improving the detection accuracy of weak targets.

[0040] In one embodiment of the present invention, PConv modules are further introduced into the backbone network and the neck network to significantly increase the receptive field with only a few parameters, establish long-distance dependencies, improve the network's semantic understanding ability, and thus improve the target recognition and discrimination performance.

[0041] In one embodiment of the present invention, an MSSA module is further added in front of the detection head to acquire multi-scale target information, optimize the feature representation capability of the model, and thereby improve the detection capability of targets at different scales.

[0042] Experiments have shown that the image fusion and deep learning-based fresh corn appearance detection method of this invention can achieve high-precision detection of the appearance of fresh corn with fewer parameters. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart of the fresh corn appearance detection method based on image fusion and deep learning according to the present invention.

[0045] Figure 2 This is a schematic diagram of the RGB-NIR image fusion method for fresh corn according to the present invention.

[0046] Figure 3 This is a schematic diagram illustrating the types of appearance detection for fresh corn in this invention.

[0047] Figure 4 This is a structural diagram of the original GELAN-t network model.

[0048] Figure 5 This is a structural diagram of the HPM-GELAN network model of the present invention.

[0049] Figure 6A This is a structural diagram of the HAM module of the present invention.

[0050] Figure 6B This is a structural diagram of the WAFM module of the present invention.

[0051] Figure 7 This is a structural diagram of the PConv module of the present invention.

[0052] Figure 8 This is a structural diagram of the MSSA module of the present invention.

[0053] Figure 9 This is a schematic diagram of the detection results of the HPM-GELAN network model of the present invention on a self-made RGB-NIR image fusion dataset. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0055] This embodiment provides a method for detecting the appearance of fresh corn based on image fusion and deep learning. See [link to relevant documentation]. Figure 1 The method includes:

[0056] S1: Images of the appearance of fresh sweet corn to be sorted during processing were acquired using RGB and NIR cameras, respectively. Two sets of image data were obtained: RGB and NIR images, containing five categories of targets on the fresh sweet corn: mechanical damage, collision damage, incomplete husk removal, missing kernels, and insect infestation, denoted as Bro, Bre, Le, Lo, and WE, respectively. Partial datasets are shown below. Figure 3 As shown, the first row is the fused image, and the second row is the label.

[0057] S2: Determine the fusion ratio of RGB and NIR images in the fresh corn appearance image fusion dataset through experiments, and construct an RGB-NIR fusion dataset for fresh corn appearance detection.

[0058] S3: Construct the HPM-GELAN detection model, which is an improvement on the GELAN-t network. The first RepNCSPELAN4 module in the GELAN-t network is replaced with the hierarchical attention module HAM, the original convolution module AConv is replaced with the windmill convolution module PConv, and a multi-scale self-attention module MSSA is added before each of the three detection heads of the GELAN-t model.

[0059] S4: The dataset is trained using the constructed HPM-GELAN detection model. The image to be detected is then input into the trained fresh corn appearance detection model to obtain the final detection result.

[0060] In S2, image fusion is achieved through pixel fusion of each channel, and the fusion process is as follows: Figure 2 As shown. First, connect the single-channel I... NIR The image is expanded into a three-channel image, and then the pixel values ​​of the two sets of images are added and fused together according to a certain ratio for each channel, that is:

[0061] R channel (x) i , y i Pixel value I at ) R (x i , y i )=(1-α)·I RGB-R (x i , y i )+α·I NIR (x i , y i (1)

[0062] G channel (x) i , y i Pixel value I at ) G (x i , y i )=(1-α)·I RGB-G (x i , y i )+α·I NIR (x i , y i (2)

[0063] Channel B (x) i , y i Pixel value I at ) B (x i , y i )=(1-α)·I RGB-B (x i , y i )+α·I NIR (x i , yi (3)

[0064] Among them, I R (x i ,y i ), I G (x i ,y i ), I B (x i ,y i ) represent the R channel (x) in the fused image, respectively. i ,y i Pixel value at ) and G channel (x) i ,y i Pixel value at position (x) and B channel (x) i ,y i The pixel value at position ) is where α is the proportion of NIR image pixels in each channel of the fused image.

[0065] To further verify the beneficial effects of the present invention, a series of experiments were conducted in this embodiment. The operating system used in the experiments was Ubuntu 20.04.4LTS, and GPU acceleration training was performed using an NVIDIA GeForce RTX3090. The PyTorch version was 1.7.1, and the CUDA version was 11.0. The training parameters for the experiments were as follows: input image size was 640×640×3, network optimizer was stochastic gradient descent (SGD), momentum parameter was 0.937, initial learning rate was 0.01, batch size was 16, epoch was 300, dropout was 0.2, and patience was 100.

[0066] To verify the effectiveness of the improved model, this embodiment selects five target detection evaluation metrics, including:

[0067] (1) Average Precision (AP) for each category;

[0068] (2) The mean average precision (mAP) of all categories, when IoU is set to 0.5, is expressed as mAP@0.5;

[0069] (3) The size of the model parameters;

[0070] (4) One billion floating-point operations per second (GFLOPs);

[0071] (5) Latency refers to the time required for the model to process the input image and generate the output result.

[0072] To obtain the optimal α value, the values ​​of α in the fusion dataset were changed, creating five fusion datasets with α = 0, α = 0.25, α = 0.5, α = 0.75, and α = 1. The comparative experimental results for each dataset are shown in Table 1, using mean precision (AP) and mean mean precision (mAP) as evaluation metrics. Analysis of Table 1 shows that when α = 0.75, the mAP@0.5 value is the highest, exceeding the results of single RGB detection and single NIR detection by 2% and 0.8%, respectively. The AP values ​​for all five target classes are not the lowest, indicating the best overall detection performance.

[0073] Table 1. Detection results for different α values

[0074]

[0075] This embodiment constructs an HPM-GELAN fresh corn appearance detection model based on the original GELAN-t network model. The original GELAN-t network model is as follows: Figure 4 As shown, the structure of the improved HPM-GELAN model is as follows: Figure 5 As shown.

[0076] To address the issue of insufficient feature extraction in the shallow stages of the backbone network, which easily leads to the loss of important details, this embodiment replaces the first RepNCSPELAN4 module of the original GELAN-t model with a HAM module in the backbone network part of HPM-GELAN. The structure of the HAM module is shown in Figure 6. The overall module design retains the GELAN design architecture, such as... Figure 6A As shown, WAFM is used as the computational center to propagate feature information at different levels and complete feature extraction and aggregation.

[0077] Specific details of the WAFM module are as follows: Figure 6BAs shown, by processing the input WAFM features with different frequency domain features and applying strip convolution, the module can effectively capture local and global features, obtain global context information, and enhance feature extraction capabilities. First, the fresh corn feature map is processed by pointwise convolution to output the convolutional fresh corn feature map. Second, this feature map is separated channel by channel. The odd-numbered sequence feature maps are merged and then processed by wavelet transform, standard convolution, and inverse wavelet transform to obtain the first-path fresh corn feature map. The even-numbered sequence feature maps are merged and then processed by average pooling, pointwise convolution, and two strip convolutions in the horizontal and vertical directions to obtain the attention factor map of the fresh corn feature map. The attention factor map is multiplied by the input fresh corn feature map of this path to obtain the attention-weighted second-path fresh corn feature map. Next, the two-path fresh corn feature maps are concatenated to obtain a comprehensive fresh corn feature map containing rich global, local, and spatial information. Finally, the comprehensive fresh corn feature map is added to the original input fresh corn feature map to retain some of the original input features, and then pointwise convolution is used to perform feature fusion to obtain the output fresh corn feature map of the module.

[0078] Analysis of the detected objects reveals that Bro and Bre targets exhibit relatively weak features, and Bre shares some similarity with the row and column spacing of corn kernels, making them easily confused. GELAN primarily uses AConv in its convolution operation, employing an average pooling followed by convolution for feature extraction. While computationally efficient, this approach is prone to losing detailed information and resulting in insufficient feature extraction.

[0079] To address the aforementioned issues, PConv modules were used to replace all AConv modules in the original model's backbone and neck network portions of HPM-GELAN. The specific structure of the PConv module is as follows: Figure 7 As shown, PConv primarily creates horizontal and vertical convolutional kernels through asymmetric padding, aligning them with the Gaussian spatial distribution of pixels of small targets to adapt to different regions of the image. First, the input feature map is padded according to four specific padding parameters to obtain X1, X2, X3, and X4. Then, X1, X2, X3, and X4 are convolved in their respective convolutional layers W... i After the operation (i = 1, 2, 3, 4), batch normalization (BN) and sigmoid linear units (SiLU) are applied to enhance the stability and speed of training. Finally, the four sets of feature maps obtained by directly concatenating them using Cat are then processed by convolution W. out To further enhance the extracted features, the receptive field size is adjusted to obtain the output feature map X. out The specific processing flow is as follows:

[0080]

[0081] X Cat =Cat(X1′,X2′,X3′,X4′) (8)

[0082]

[0083] In the formula, Represents the convolution operation; W1, W2, W3, W4, W out The first two numbers in the upper right corner represent the kernel size, and the third number represents the number of output channels after the convolution operation.

[0084] Among the detected targets, Bre and Bro are relatively small compared to Lo, Le, and WE. Furthermore, even within the same target category, such as Le, there are significant scale differences. The original GELAN model uses three detection heads (P3, P4, and P5) to enable the network to detect targets of different scales; however, the detection head structure is relatively simple, resulting in weak perception of feature maps of input targets at different scales. To address these issues, an MSSA module is added to each of the three detection heads in HPM-GELAN, with the module structure as follows: Figure 8 As shown, the input fresh corn feature map is divided into two parts, and both parts are processed by the same convolutional group consisting of three dilated convolutions of different sizes to obtain multi-scale fresh corn features. Next, the two feature parts are fused by pointwise convolution. Then, the features are processed by average pooling, fully connected layers and channel expansion operations to obtain multi-scale fresh corn feature maps. Finally, the multi-scale fresh corn feature maps are added to the original input fresh corn feature maps to output scale-enhanced fresh corn feature maps.

[0085] This embodiment further verified the impact of each component on the model performance through ablation experiments. The experimental evaluation metrics were AP, mAP, Params, and GFLOPs. The experimental results are shown in Table 2.

[0086] Table 2 Ablation Experiment Results

[0087]

[0088] Experiments 1, 2, and 3 show that the PConv and MSSA modules enhance the model's detection performance for weak targets such as Bre and Bro. Experiments 1 and 4 show that applying the HAM module to the backbone network improves mAP@0.5 by 1.2%, indicating that the HAM module effectively extracts and retains the feature information of the input targets, enhancing the network's feature extraction capability. When all three modules are integrated, the improved model's mAP@0.5 is 2.2% higher than the baseline, despite reducing the number of model parameters, indicating an overall improvement in detection performance. Experiments 5, 6, and 7 completed experiments without using the HAM, MSSA, and PConv modules, respectively. The results show that the mAP@0.5 values ​​under any two modules are lower than the detection results of the integrated application of all three modules in Experiment 8, indicating that the detection performance is optimal when HAM, MSSA, and PConv are applied simultaneously to the baseline.

[0089] To visually illustrate the detection performance of the HPM-GELAN network model, a heatmap visualization was used to compare some detection results before and after the improvement, such as... Figure 9 As shown in the heatmap, the stronger the red color, the higher the network's attention level to that part; conversely, the lighter the blue, the lower the network's attention level to that part. The four sets of comparison images show that after the improvement, the network model's heatmap color for the defective part is redder, indicating a higher level of attention to the defect. Less attention is paid to the background area, suggesting that the network's focus on the defective target object is more concentrated. Figure 9 (b) The improved network model has a higher attention area to the corn leaves and a more complete overall attention to detail. Figure 9 In groups (c) and 9(d), the heatmap of the improved network model HPM-GELAN shows that the high-interest regions only exist in the region of the target object, indicating that the feature extraction capability of HPM-GELAN has been improved and the network attention has been increased.

[0090] To further verify the superiority and effectiveness of the improved HPM-GELAN model in this embodiment, comparative experiments were conducted on a self-made fresh corn appearance fusion dataset using RetinaNet, EfficientDet-d1, D-FINE-n, RT-DETR-r50, YOLOv8, YOLOv10, YOLOv11, Hyper-YOLO-v1.1, GELAN-t, and the method proposed in this embodiment, under the same experimental parameters. The evaluation metrics were mAP, Params, GFLOPs, and Latency. The specific experimental results are shown in Table 3.

[0091] Table 3 Comparison of experimental results

[0092]

[0093] As shown in Table 3, the HPM-GELAN model of this invention exhibits the highest mAP@0.5 and the best detection performance, exceeding RetinaNet, EfficientDet-d1, D-FINE-n, RT-DETR-r50, YOLOv8, YOLOv10, YOLOv11, Hyper-YOLO-v1.1, and GELAN-t by 13.2%, 11.2%, 3.2%, 5.3%, 4.0%, 3.6%, and 2.2%, respectively. Comparing the number of parameters required by each model, the HPM-GELAN model requires the fewest parameters, representing 5.3%, 29.1%, 51.6%, 4.6%, 64.0%, 71.1%, 74.4%, 76.8%, and 97.5% of the aforementioned models. Furthermore, the improved HPM-GELAN model also demonstrates excellent performance in inference time, meeting the real-time requirements of actual production.

[0094] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting the appearance of fresh corn based on image fusion and deep learning, characterized in that, The method includes: Step 1: Acquire RGB and NIR images of the appearance of fresh corn and fuse them; Step 2: Use the HPM-GELAN detection model to detect the fused image and obtain the appearance detection results of fresh corn; The HPM-GELAN detection model is an improvement on the GELAN-t network, replacing the first RepNCSPELAN4 module in the GELAN-t network with a hierarchical attention module HAM.

2. The method for detecting the appearance of fresh corn based on image fusion and deep learning according to claim 1, characterized in that, The HPM-GELAN detection model is an improvement on the GELAN-t network. The improvements also include replacing the original convolutional module AConv with a windmill-shaped convolutional module PConv.

3. The method for detecting the appearance of fresh corn based on image fusion and deep learning according to claim 1, characterized in that, The HPM-GELAN detection model is an improvement on the GELAN-t network. The improvement also includes adding a multi-scale spatial attention module (MSSA) before each of the three detection heads of the GELAN-t model.

4. The method for detecting the appearance of fresh corn based on image fusion and deep learning according to claim 1, characterized in that, The hierarchical attention module (HAM) uses the wavelet attention fusion module (WAFM) for feature extraction and aggregation. The WAFM module obtains information from each layer through two branches: wavelet convolution and strip convolution. The feature extraction process includes: Step 21: Perform pointwise convolution on the input fresh corn feature map; Step 22: Perform channel-by-channel separation on the feature map obtained in step 21; Step 23: Merge the feature maps of odd-numbered sequences and then sequentially pass them through wavelet transform, standard convolution, and inverse wavelet transform to obtain the first fresh corn feature map. Merge the feature maps of even-numbered sequences and then sequentially pass them through average pooling, pointwise convolution, and two strip convolutions in the horizontal and vertical directions to obtain the attention factor map of the fresh corn feature map. Multiply the attention factor map with the input fresh corn feature map of this path to obtain the second fresh corn feature map after attention weighting. Step 24: Combine the first and second fresh corn feature images; Step 25: Add the feature map obtained in step 24 to the original input feature map and then perform feature fusion through pointwise convolution.

5. The method for detecting the appearance of fresh corn based on image fusion and deep learning according to claim 3, characterized in that, The processing flow of the Multi-Scale Spatial Attention Module (MSSA) includes: Step 31: Divide the input feature map into two equal parts. Both parts are processed by the same convolutional group consisting of three dilated convolutions of different sizes to obtain the features of fresh corn at multiple scales. Step 32: Fuse the two feature sets using pointwise convolution; Step 33: The fresh corn feature map containing multi-scale information is obtained by sequentially performing average pooling, fully connected layer, and channel expansion operations; Step 34: Add the feature map obtained in step 33 to the original input feature map to output a scale-enhanced fresh corn feature map.

6. The method for detecting the appearance of fresh corn based on image fusion and deep learning according to claim 1, characterized in that, The formula for fusing the RGB and NIR images in step 1 is as follows: R channel (x) i ,y i Pixel value I at ) R (x i ,y i )=(1-α)·I RGB-R (x i ,y i )+α·I NIR (x i ,y i ) G channel (x) i ,y i Pixel value I at ) G (x i ,y i )=(1-α)·I RGB-G (x i ,y i )+α·I NIR (x i ,y i ) Channel B (x) i ,y i Pixel value I at ) B (x i ,y i )=(1-α)·I RGB-B (x i ,y i )+α·I NIR (x i ,y i ) Wherein, 1-α and α are the weight proportions of RGB image and NIR image in the pixel values ​​of each point in each channel, respectively, and the value of α ranges from [0,1].

7. The method for detecting the appearance of fresh corn based on image fusion and deep learning according to claim 1, characterized in that, The types of appearance of fresh corn include: mechanical nibbling, impact damage, incomplete removal of husks, missing kernels, and insect infestation.

8. A fresh corn appearance detection system based on image fusion and deep learning, characterized in that, The system is used to implement the fresh corn appearance detection method based on image fusion and deep learning as described in any one of claims 1-7, and the system comprises: The image acquisition module is configured to acquire RGB and NIR images of the appearance of fresh corn. An image fusion module is configured to fuse the RGB image and the NIR image; The detection module is configured to use the HPM-GELAN detection model to detect the fused image and obtain the appearance detection results of fresh corn.

9. A device for detecting the appearance of fresh corn based on image fusion and deep learning, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the fresh corn appearance detection method based on image fusion and deep learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the fresh corn appearance detection method based on image fusion and deep learning as described in any one of claims 1 to 7.

Images